Georgia

Does it Work: Building Methods for Understanding the Effects of Professional Development

Principal Investigator:

Co-Investigator:

Project Overview

Background & Purpose:

This study seeks to develop and refine methods for empirically examining relationships among teacher professional development, teacher learning, teacher practice, and student achievement. To acheive this, we focus on the following three questions as we offer the NSF-funded InterMath professional development program to middle-grades teachers:

What do the teachers learn from InterMath experiences?
If the teachers learn from InterMath, do their instructional practices change as a result?
If the teachers’ practices change, are there measurable changes in students’ achievement?

Setting:

This study has taken place in 2 large urban school districts in Georgia. Both districts have low SES, high free and reduced lunch rates, and a number of schools failing to meet AYP. Teachers in grades 5-8 were invited to participate from across each district.

Research Design:

The research design for this project is longitudinal, and is designed to generate evidence that is descriptive using case study and observation, associative/ correlational using quasi-experimental methods, and causal through statistical modeling. This project includes an intervention which is participation in a 40-hour professional development opportunity. The PD was content-focused. Teachers in the comparison group participated in whatever was “normal” – which was documented and amounted to no particular professional development in mathematics during the time data were collected.

This project has collected original data through a variety of methods including assessments of learning/achievement tests, observation through videography, and survey research including paper and pencil self-completion questionnaires, and semi-structured/ informal interviews, both face-to-face and telephone. We are using several instruments. First, a teacher assessment instrument designed to measure teacher mathematical knowledge for teaching around fraction and decimal operations and proportional sense. This instrument includes items from the University of Michigan’s Learning Mathematics for Teaching instrument as well as items developed by the Does it Work project team. The instrument has been validated through the use of cognitive instruments. We have also developed a student assessment of fraction ideas based on materials from published textbooks as well as some items developed by team members. We are using an abbreviated version of The 2001 RAND Teacher Survey to capture teacher practices. We have developed a short pre/post course survey to gauge whether participation in the professional development has changed teacher attitudes about using open-ended problems and technology in their classrooms as well as their confidence in teaching aspects of the content covered in the PD.

The assessment data are being analyzed using a Mixture Rasch Model to identify not only growth in ability but also movement between latent classes as identified through the statistical analysis. For qualitative data, including interviews with the teachers and videotape of the PD and the teachers’ classrooms, we are using a variety of standard approaches (e.g., coding and sorting, memoing, etc.). Survey data are analyzed using descriptive statistics to determine changes from one collection of the data to the next.

Findings:

We analyzed the dichotomously scored responses using the mixture Rasch model and found two subgroups of teachers. The teachers fell into one of the two classes based on patterns in their responses to the test items. Initial analysis of the crosstabs suggested a high correlation between class membership and teacher preparation. This result is consistent with LMT findings reported by Hill (2007). Analysis of the raw response data and interviews allowed us to extend Hill’s results significantly by uncovering differences in reasoning between the two subgroups. The first difference was attention to the role of multiplying or dividing by 1 in standard computation procedures. The second was the capacity to reason about referent units for numbers and parts-of-parts of quantities appropriately across a range of situations. Given that Class 1 teachers tend to outscore Class 2 teachers and that Class 2 teachers show a lower incidence of reasoning with referent units appropriately, it seems that our focus on referent units as one of the major themes in the revised InterMath course is critical.

Teachers who did not attend to referent units typically used one of four strategies to solve items that included drawn representations. Many would try to identify the requisite parts in the drawing (e.g., locate each of the factors and the product) to determine whether the drawing was a viable model of the problem. Another common approach was to calculate the solution using a traditional algorithm, then find a response that matched. The third observed approach was to attempt to identify the item that showed the process of solving using the traditional algorithm. The final strategy was to use measurement to find responses (e.g., using informal measuring devices to determine accuracy of drawings). Because of our test construction, many of these strategies led teachers to inaccurate conclusions about the items on which they were working.

We found that teachers often have limited ability to apply their algorithmic understandings of mathematics to drawn representations. In our analysis, most teachers demonstrated deficiencies in making sense of any given representation – particularly if it was one they had not used before. Interestingly, even when they had familiarity with a representation (e.g., area model for multiplication) they were unable to transform what they understood about it to a similar model for a different operation (e.g., area model for fraction division) or to translate it from the model they were looking at to another model (e.g., number line model for multiplication of fractions).

Publications & Presentations:

Articles

Brown, R. E. (2010). Guest editorial: Tensions faced by mathematics professional developers. The Mathematics Educator, 20(1), 3-7.

Izsák, A., Orrill, C. H., Cohen, A., & Brown, R. E. (2010). Measuring middle grades teachers’ understanding of rational numbers with the mixture Rasch model. Elementary School Journal, 110(3), 279-300.

Lee, S., Brown, R. E., & Orrill, C. H. (accepted for publication). Mathematics teachers’ reasoning about fractions and decimals using drawn representations. To appear in Mathematical Thinking and Learning.

Published Proceedings

Orrill, C. H., Izsák, A., Jacobson, E., & de Araujo, Z. (2010). Teachers’ understanding of representations: The role of partitioning when modeling fraction arithmetic. In K. Gomez, L. Lyons, & J. Radinsky (Eds.) Learning in the disciplines: ICLS 2010 conference proceedings (Vol. 2, pp. 338-340). Chicago, IL: University of Illinois at Chicago.

Izsák, A., Lobato, J., Orrill, C. H., Cohen, A. S., & Templin, J. (2009). Psychometric models and assessments of teacher knowledge. Proceedings for the Twelfth Special Interest Group of the Mathematical Association of America on Research in Undergraduate Mathematics Education. Paper retrieved July 2, 2009, from http://rume.org/crume2009/proceedings.html.

Lee, S-J. & Orrill, C. H. (2009). Middle grades teachers’ reorganization of measurement fraction division concepts. In S. L. Swars, D. W. Stinson, & S. Lemons-Smith (Eds.), Proceedings of the 31^st annual meeting of the North American chapter of the International Group for the Psychology of Mathematics Education (pp. 1370-1377). Atlanta, GA: Georgia State University.

Sexton, S. & Orrill, C. H. (2009). The impact of professional development on two teachers’ understanding and use of representations. In S. L. Swars, D. W. Stinson, & S. Lemons-Smith (Eds.), Proceedings of the 31^st annual meeting of the North American chapter of the International Group for the Psychology of Mathematics Education (pp. 735-782). Atlanta, GA: Georgia State University.

Orrill, C.H., Sexton, S., Lee, S-J, & Gerde, C. (2008). Mathematics teachers’ abilities to use and make sense of drawn representations. In The International Conference of the Learning Sciences 2008: Proceedings of ICLS 2008. Mahwah, NJ: International Society of the Learning Sciences.

Presentations

Cohen, A. S. (March, 2008). Analyzing tests with mixture IRT models. Presented at Center for Educational Assessment, the University of Massachusetts, Amherst, MA.

Izsák, A. (March 1, 2010). Measuring middle grades mathematics teachers’ understanding of fractions. Invited presentation at UCLA.

Izsák, A., Confrey, J. E., Orrill, C., McCrory, R., & Kelly, A. (April, 2010). Using psychometrics to advance assessment in mathematics education. Symposium presented at the Research Presession of the 88^th Annual Meeting of the National Council of Teachers of Mathematics: San Diego.

Izsák, A., Lobato, J., Orrill, C. H., Cohen, A. S., & Templin, J. (February, 2009). Psychometric models and assessments of teacher knowledge. Paper presented at the Conference on Research in Undergraduate Mathematics Education: Raleigh, NC.

Izsák, A., Orrill, C. H., Cohen, A. S., Brown, R. E., (2009, April). Assessing middle grades teachers’ capacities to reason about arithmetic with rational numbers. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.

Lee, S. J., Brown, R.E., Orrill, C.H., & Sexton, S. (April, 2009). Middle school teachers’ problem solving strategies for interpreting rational number items using drawn representations. Poster to be presented at the Research Presession of the 87th Annual Meeting of the National Council of Teachers of Mathematics: Washington, D.C.

Lee, S.J. & Orrill, C. H. (September, 2009). Middle grades teachers’ reorganization of measurement fraction division concepts. Paper presented at The 31st annual meeting of the North American chapter of the International Group for the Psychology of Mathematics Education: Atlanta, GA.

Orrill, C. H. (April, 2009). Assessing teachers’ knowledge in mathematics: Considering new approaches. Invited presentation to The International Center for Learning, Education and Performance Systems. Athens, GA.

Orrill, C. (November, 2008). Does it work?: Building methods for understanding effects of professional development. Paper presented at 2008 Association for Educational Communications and Technology Convention. Orlando, FL.

Orrill, C. H., Izsák, A., Jacobson, E., & de Araujo, Z. (June, 2010). Teachers’ understanding of representations: The role of partitioning when modeling fraction arithmetic. Poster presented at the 9^th International Conference of the Learning Sciences: Chicago.

Orrill, C. H., Jacobson, E., & de Araujo, Z. (April, 2010). Teachers’ emerging understanding of fraction division as proportional reasoning in professional development. Paper presented at the Annual Meeting of the American Educational Research Association: Denver.

Orrill, C.H., Lee, S.J., & Brown, R.E. (April, 2009). Mathematics teachers’ abilities to interpret fraction operations with drawn representations. Paper presented at the Annual Meeting of the American Educational Research Association: San Diego, CA.

Orrill, C.H., Sexton, S., Lee, S., Gerde, C. (June, 2008). Mathematics teachers’ abilities to use and make sense of drawn representations. Paper presented at the International Conference of the Learning Sciences. Utrecht, Netherlands.

Sexton, S. & Orrill, C. H. (September, 2009). The impact of professional development on two teachers’ understanding and use of representations. Paper presented at The 31^st annual meeting of the North American chapter of the International Group for the Psychology of Mathematics Education: Atlanta, GA.

Sexton, S., Orrill, C., & Gerde, C. (March, 2008). Middle grades teachers’ flexibility with drawn representations. Paper presented at the Annual Meeting of the American Educational Research Association. New York.

Dissertations

Brown, R. E. (2009). Community building in mathematics professional development. (Doctoral dissertation). University of Georgia, Athens, GA.

Lee, S. J. (2010). Exploring middle grade teachers’ knowledge of partitive and quotitive fraction division. (Doctoral dissertation). University of Georgia, Athens, GA.

On the Web:

Discipline:

State:

Target Population:

Research Design:

Read more about Does it Work: Building Methods for Understanding the Effects of Professional Development

Pages

Search form