Analyzing Student Usage and Outcome Data from a Randomized Experiment of Cognitive Tutor Algebra

Pennsylvania
Causal
Math
Middle
Secondary
School Personnel
Principal Investigator: 
John Pane
Project Overview
Background & Purpose: 

This study follows up a successful experiment that demonstrated the effectiveness of the Cognitive Tutor Algebra I program. It seeks to understand the causal mechanisms that drive CTAI's success, by examining usage data generated by program participants, along with covariate and outcome data from both treatment and control groups.

Setting: 

We will use data from the original CTAI study, which took place in middle schools and high schools in seven states throughout the US.

Research Design: 

The project will generate evidence that is causal. The Cognitive Tutor Algebra I program is being compared to a standard Algebra I curricula by analyzing secondary data that was generated by the CTAI effectiveness experiment.

We intend to use a principal-stratification framework to analyze student usage data from the CTAI experiment. We will begin by formulating hypotheses regarding subsets of students, defined by their measured or potential CTAI usage, who may experience differential effects from the program. Next, we will build statistical models to attempt to identify these subgroups among both treatment and control students; finally, we will test for differential program effects in these groups.

Findings: 

Findings will be published as they become available.

Publications & Presentations: 

Results from the original CTAI study were published as:

Pane, J. F., Griffin, B. A., McCaffrey, D. F., & Karam, R. (2014). Effectiveness of Cognitive Tutor Algebra I at Scale. Educational Evaluation and Policy Analysis, 36(2), 127-144. doi: 10.3102/0162373713507480. http://epa.sagepub.com/content/36/2/127