The project uses a comparative research design and will generate evidence that is descriptive [observational, open-ended survey items], associative/correlational [analytic methods involving data mining methodologies], and causal [experimental, factorial designs to examine the effects of interventions on behavior and performance]. Original data are being collected on undergraduate learners in STEM courses, with a particular focus on female students and those from underrepresented minority groups, using assessments of learning and performance, observation [server logs of user behavior], and survey research [self-report questionnaire].
Study 1 is observational. Study 2 compares effects of motivational prompting and skill training interventions (and a combined condition) against a "business as usual" condition reflecting typical coursework in the assigned class. Effects on students’ tool use, learning, performance, and motivation are observed. Study 3 tests whether sending notification messages to students who evidence behaviors associated with failure or dropout (i.e. in Study 1 sample) that point out such behavior and recommend alternatives can induce behavioral change and superior performance outcomes, compared to a control group who evidences such behavior and receives no message.
Data sources include log files of learning events obtained from the learning management system via Splunk and surveys administered via the LMS, as well as course grades (including withdrawal) and exam scores. Studies 1, 2, and 3 will employ a combination of comparative analyses, structural modeling, and data mining techniques.
