Oklahoma

The research design for this project is comparative, and is designed to generate evidence that is descriptive using observation, associative/correlational and causal using quasi-experimental methods. This project includes the intervention of attendance at the KREST institute, which is a two-week summer research program designed to provide authentic research experiences for teachers, lesson study and creating professional learning communities. The comparison condition is attendance at a one-day traditional professional development workshop.

This project collects original data using a variety of methods: diaries/journals/records kept by study subjects, assessments of learning/achievement tests, personal observation, and survey research including both paper and pencil and online self-completion questionnaires, as well as face-to-face semi-structured/informal interviews.

Instruments and measures used include:

Teacher Attitude and Motivation to Teach Science and Integrate Technology Pre/Post Survey
Teacher attitude and motivation to teach science and integrate technology into science teaching was measured through the use of an on-line pre/post survey. This survey was developed from an instrument previously tested by the K20 (the PDI used by Janis Slater), work compiled by Yager (2005), and items from the motivation research of one of the PI’s for measuring self-efficacy (e.g., Greene, Dillon, Crynes, 2003 & Greene et al. 2004). Therefore, all the items included on the survey instrument had been tested in other research and found to be reliable and valid for measuring attitudes and efficacy as they were being used in the present research.

Motivation and Aspiration Survey
All students, who were enrolled in the participating teachers’ classroom were asked to complete a short survey which included measures of: 1) Attitudes and efficacy for science and the inquiry method, 2) Self-efficacy for learning science, 3) Self-efficacy for using technology for science, 4) Use of constructivist approaches in the classroom, 4) Collaborative practices to support constructivist approaches in the classroom, 5) Expository and didactic approaches in classroom, and 6) Technology use in the classroom. This survey was modeled from the pre/post survey used to measure teacher attitudes, motivation and technology integration, thereby incorporating and modifying previous work by Slater (2007, 2008); Yager (2005); Greene, Dillon, Crynes (2003); and Greene et al., (2004).

Inquiry Process Test
To obtain a measure of student achievement, all students from selected comparison and experimental group teacher participants were asked to take a multiple choice scientific inquiry literacy test. Along with the student science motivation and aspiration survey, the instrument is administered at the beginning and end of the school year in Year 3 of the study. The K20 science team adapted the Scientific Literacy Inquiry Test (ScInqLiT) developed by Carl Wenning (2007) for use with Middle and High school students participating in the REESE study.

Classroom Observation
For the Spring 2008 pilot test of the classroom observation instrument, 11 teacher classrooms each from comparison and experimental group participants were selected for participation in classroom observations. One member from the evaluation/research team and one member from the program team or other qualified K20 staff visited the selected teacher classrooms and observed a lab or activity lesson in its entirety. The REESE observation teams observed and recorded student engagement and interactions with the teacher during the lesson. A teacher and student inquiry observation guide was developed jointly by the evaluation and program teams. Staff involved in classroom observations included members of the K20 REESE evaluation/research team, K20 REESE program team and members of the K20 staff who are certified science teachers in the State of Oklahoma and are trained in the use and teaching of the inquiry method. All individuals who conducted the classroom observations had completed the one day training given by Dr. Cate. The actual tool used in the classroom observations was adapted from Zubrowski, B. (2007) and Horizon Research, Inc (2000, 2005). The K20 observation tool includes measures for teacher use of inquiry methods and student level of engagement.

Concept Mapping
We also used Concept Maps with the KREST participants to capture and score teacher knowledge of the science concepts that were taught during the professional development sessions. These data are showing that the concept maps seem to be a good way to understand how teacher knowledge changed. Working mostly with the research of Graff (2005) and ideas of Shavelson, Ruiz-Primo, & Wiley (2005) a subgroup of the research and evaluation team developed a series of scores to use with the concept maps. They used the following metrics: Number of nodes minus one for the central concept; number of links or propositions, for a complexity score; links / nodes – 1 for a map density score; the longest link for a depth index; and the level with the greatest number of nodes for a width index. These metrics capture the complexity and structure of the knowledge represented in the maps. We are hoping that they will also capture change over time. At least three people scored and checked each map to ensure accuracy and reliability.

Return to Science (RTS) Semi-structured Interview
Interviews were conducted with the RTS KREST participants from the 2007 institute. The Interview protocol included questions about their participation in the institute, professional learning community development, on-line learning communities, implementation of inquiry teaching methods in the classroom, and the collaborative lesson study process.

This is a mixed methods study. For quantitative data analysis we are using correlational techniques (e.g., factor analysis) and multivariate analysis of variance for many of the tests or similar parametric statistics. We may also need non-parametric is some cases. Data and questions will determine actual tests used. Some of our data will likely require HLM procedures. For the qualitative data, standard constant comparison methods will be used. All data will be kept for five years in the event that other researchers ask to analyze our data.