Cross-State Study of Quality Math and Science Professional Development

Analysis of the Quality of Professional Development Programs for Mathematics and Science Teachers: Findings from a Cross-State Study (February 2007)

The findings from the CCSSO-led study provide a picture of the status and prospects for math and science teacher professional development, and particularly initiatives supported through federal and state funding.

• Content Focus: The CCSSO analysis found that current leading professional development is providing content knowledge development for teachers in math and science, especially for elementary and middle grades teachers. In 22 of 25 programs reviewed, the activities were rated as significantly focused on content knowledge in math or science. Additionally, a majority of programs reviewed were rated positively for providing important pedagogical content knowledge in math or science for teachers.

• Active Learning: The professional development activities are using active methods of learning for teachers in a large majority of programs. In comparison to the findings from research on professional development in the mid-1990s, the sample of programs in this study were surprising in the prevalence of active roles by teachers, including developing and presenting sample lessons, use of coaching and mentoring, developing new lessons or assessments, and interaction among teachers about ways to improve their practice.

• Collective Participation: While most of the 25 reviewed programs did organize teachers and activities by common subject area and grade level, only a minority of the sample programs focused on delivery of professional development to teachers through a school-based strategy where teachers were learning with their school colleagues. The predominant organizing pattern was to plan teacher development for a treatment group drawn a large number of schools and districts, with only a small number of teachers from each school.

• Coherence: In almost all programs rated, the reviewers found a description of how the program was designed to be aligned to state content standards. Additionally, a majority of program materials described how the development was consistent with local curriculum or with curriculum materials teachers were intended to use.

• Sufficient Time: The average time for professional development activities including follow-up work in schools was found to be significantly greater than the typical math and science professional development documented and described in the mid-1990s.

• Evaluation: The reviewed programs included evaluation designs with a number of evaluation objectives and tools in most programs. The CCSSO review covered four specific evaluation targets, and a majority of programs included at least one measure in each area: (a) quality of implementation of development activities, (b) gains in teacher knowledge, (c) change in classroom practices, and (d) increase in student achievement. Thus in general the programs were very ambitious in emphasizing methods of evaluation. The second phase of the study is now analyzing the evaluation results being reported from the 25 programs, and then comparing planned evaluations with what is actually completed and reported to educators and decision-makers.
  

Does Teacher Professional Development Have Effects on Teaching and Learning? Analysis of Evaluation Findings from Programs for Mathematics and Science Teachers in 14 states (February 2008)

CCSSO reviewed evaluation studies from 25 professional development programs for teachers of mathematics and science from programs nominated by 14 states. The evaluation study reports and papers served as the data sources for the present analysis and paper. The reports primarily address evaluation findings from professional development activities conducted during the period 2004 through 2007. Following are several key findings from the study:

• One-third of evaluation studies reported measurable effects of teacher professional development. Seven of the evaluation studies of teacher professional development reviewed by CCSSO reported measurable effects of the teacher development activities on subsequent student outcomes. A total of 10 of the studies reported measurable effects on increasing teacher content knowledge, and four studies reported measurable effects on instructional practices of teachers.

• Content focus plus sufficient time plus in-school component equals significant effects. The cross-program review of studies showed significant effects of professional development programs for teachers of math and science when the programs include focus on content knowledge in the math and science subject areas plus training and follow-up pedagogical content knowledge. The total time in professional development for the studies with significant effects was 50 hours or more.

• Purposeful evaluations yield measurable effects. The evidence from the CCSSO review of evaluation studies shows that one-third of the programs reviewed had well-developed evaluations that produced findings with measurable effects on student achievement or change in instructional practices. Our analysis of evaluation findings emphasized scientific study design, and these kinds of designs could have been implemented across more programs.

• Teacher vs. school-based professional development designs provide differing data on success. Many designs for professional development are based on selection through teacher-based, voluntary models. The use of teacher-based professional development makes important follow-up activities harder to schedule and implement, and alignment to school curriculum more difficult to accomplish. For evaluation, the use of student assessment scores and tracking change over time appear to be facilitated with use of the school-based model for professional development.

• Include outcome measures in allocation of evaluation resources. Smaller professional development projects typically cannot afford ambitious, multi-stage evaluations or research. Allocation of more funds to evaluation would mean fewer participating teachers or fewer resources for the program implementation. In the cross-state program review we observe that smaller programs typically had to choose a few measures and methods of evaluation.

• Plan for use of data systems and experimental designs. Evaluations that will measure effects over time require access to data collection instruments or data systems, and advance planning with school officials. About one-fourth of program evaluations in the study did include comparison of a treatment group with a control group of teachers. State managers should consider evaluation designs that can be completed by linking data from state student assessments or local assessments with data on professional development for teachers.

• Link teacher knowledge gains to change in classroom practices. One type of evaluation finding identified in this review of studies showing promise for further use and expansion to other PD studies was measurement of change in teaching practices in the classroom. Four of the studies implemented well-tested instruments for comparing classroom practices across samples of teachers and classrooms. With advance planning, teachers and classrooms can be selected so that change in practices can be measured at the baseline point when teacher development begins and after implementation of activities and a period of implementation has been experienced.

• Use findings in program decisions. With the recent attention to scientific designs to provide measurement of impact of professional development on learning, we would like to see greater focus on how results from evaluations will be provided to decision-makers at specific points of time in the course of a project, and not long after the program activities have concluded. This is particularly important if a specific model is being considered for replication or expansion to other districts, schools, or additional teacher groups.

• Value partnerships for evaluation. Our analysis of evaluation findings across a number of programs and studies indicates that partnerships between higher education institutions and school districts have generally not added to the capacity for evaluation of professional development. For partnerships led by higher education institutions, the key partners with regard to data and measures for evaluation are local school district decision-makers and state education agency officials.

Current Models for Evaluating Effectiveness of Teacher Professional Development: Recommendations to State Leaders from Leading Experts (December 2008)

This summary report provides an overview of the proceedings and links to the presentations made at the Workshop for State Leader and Evaluators of Teacher Professional Development held in Washington, DC, April 24-25, 2008.