Staging the Design and Work of Evaluation: Piloting a Stage-One Evaluation of a Complex Educational Initiative

California
Science
Research Methods
Principal Investigator: 
Mark St. John
Project Overview
Background & Purpose: 

This study examines and tests the premise that conducting evaluations in stages is a sound and cost effective way to structure the evaluations of large complex initiatives. The work has two highly inter-related goals: The first is to study and describe a “staged approach” to designing and implementing evaluations of complex educational initiatives. The second goal is to conduct a prototype stage-one evaluation as a way to instantiate and test the basic concept. This prototype evaluation—its processes and products—will be reviewed to better understand and articulate the general design principles and parameters of the staged evaluation approach.

Setting: 

There are five sites: 1) Undergraduate Research Center: Center for Authentic Science Practice in Ed (CASPiE) at Purdue University; 2) Exploring New Models for Authentic Undergraduate Research with Two Year College Students -- City Colleges of Chicago, IL State University, Youngstown State University, and Hope College; 3) Research Experiences to Enhance Learning (REEL) at Ohio State University, Columbus, Ohio with 15 collaborating institutions; 4) The Northern Plains Undergraduate Research Center at the University of South Dakota, Vermillion, SD with Augustana College, Mount Marty College, Dordt College, Buena Vista University, Sinte Gleska University, and Ft. Berthold Community College; and 5) UT-URC: A new model for teaching through research at University of TX, Austin.

Research Design: 

The research design for this project is cross-sectional, and is designed to generate evidence which is descriptive (design research and observational) and synthetic (meta-analysis). This project collects original data using personal observation and informal face-to-face and telephone interviews. This project also includes secondary source analysis of annual reports and evaluation reports. This project gathers data from a review of documents, interviews and site visits. Collectively, it is reviewed and interpreted in light of our project protocols and goals.

Findings: 

Findings will be added as they become available.

Other Products: 

The research findings from this study will be presented in a publishable article outlining the staged evaluation approach and illustrating its application in a particular setting.