This is a longitudinal and cross-sectional project designed to generate associative or correlational [quasi-experimental], descriptive [online behavior], and causal [experimental, quasi-experimental] evidence.
For the lab studies, participants first complete an online screening questionnaire that includes questions about their backgrounds in computer programming, their majors, year in school, personal interests in computer science and in other domains, and basic demographic information (age, gender, ethnicity, GPA). Individual difference measures to control for different orientations which students may bring to the lessons are also completed, including measures of characteristic achievement goals and learning orientations (Achievement Goal Questionnaire (Elliott & McGregor, 2001); Achievement Goal Inventory (Grant & Dweck, 2003); Academic Motivation Scale (Vallerand, et al., 1992)), and characteristic self-regulatory responses (Brief Self Control Scale (Tangney, Baumeister & Boone, 2004), regulatory focus (Higgins, et al., 2001), Action Control Scale (Kuhl & Beckmann, 1994)). As controls, the questionnaire also includes a measure of self-esteem (Rosenberg, 1989) and a brief measure of the basic, Big 5 (extraversion, agreeableness, conscientiousness, neuroticism, openness to experience) personality traits (Gosling, Rentfrow & Swann, 2003). In the lab, participants complete an introductory HTML programming lesson online, drawn from a real online class which includes lesson content, examples, practice exercises, and ends with a particular assignment assessing their learning. After completing the assignment, the computer administers a quiz on what they learned, and several self-report questionnaires assessing their interest and enjoyment of the lesson, how valuable the lesson was, and how well they thought they did. All of these questions are rated on 5-point scales. They are also given an incidental recognition measure, in which words that appeared in different examples and exercises are listed along with other words similar in frequency in the language but which did not appear in the lessons. This incidental recognition measure will allow us an indirect measure of how much attention students paid to the content of the examples and exercises (Sansone, Weir, Harpster & Morgan, 1992). In addition to these self-report measures, we are also using software to record during the lesson how long students spend on each page, example, and exercise, whether they experimented with different examples (e.g., changed HTML codes to see what the results would be), and whether they accessed any other web pages (e.g., Google, email applications). At the end of the lab session, students are given the opportunity to learn more about HTML programming by requesting an access code that will allow them to access the full set of lessons from the real online class taught at the University of Utah. We will track both whether students asked for this access code, and whether they actually accessed these materials anytime during the following 3 months. (We obtain this information from the online class server).
At the beginning of the data analysis phase we plan to assess relationships among the different “boxes” in our model using combinations of repeated measures (to assess changes) and regression techniques. The second stage is exploratory, with the goal of identifying different profiles of interaction with the task over time. In particular, we will attempt to identify profiles that are associated with optimal learning and motivation outcomes, as well as profiles associated with trade-offs.