A Phase II Trial of the Systems Evaluation Protocol for Assessing and Improving STEM Education Evaluation

This study will be conducted using a longitudinal-sequential (or cohort-sequential) design with four cohorts over five years. The first cohort is comprised of eight pilot sites (seven CCE, one MRSEC) already engaged in the SEP process. Data from the initiation stage of this process (pilot years 1 and 2) have already been collected. The second cohort 20 CCE and three MRSEC sites who are working through the facilitated SEP. The Third cohort will consist of 10 sites who will be used to develop a virtual version of the SEP, and the fourth cohort will use only the virtual SEP. Over the next five years we will follow these sites through their implementation and utilization stages in an effort to determine the degree to which any changes in evaluation capacity are sustained over time. In addition, we will monitor the evaluation activity of five CCE sites and five MRSEC sites who have not participated in the SEP process in REESE year 2 as a control group. The longitudinal-sequential design will allow us to make a number of different comparisons both between and within the four cohorts. Primarily, we will use change score analysis to determine a) if there are any changes over time that are associated with exposure to the SEP process, b) if these changes differ between groups, and c) if these changes are sustained over time.