CAREER: A Comprehensive Modeling Approach to Cognitively Diagnostic Assessment: Methodological Developments and Practical Implementations

New Jersey
Descriptive
Associative
Math
Cognitive Science
Research Methods
Middle
Secondary
Higher Education
Principal Investigator: 
Jimmy de la Torre
Project Overview
Background & Purpose: 

This project will develop a cognitively-based assessment of proportional reasoning that can provide information deemed diagnostic, highly informative, and potentially prescriptive, therefore more relevant to learning and useful in applied instructional settings. In addition, this proposal will introduce a comprehensive framework that will make construction and analysis of such assessments more practicable. Finally, a computer code for estimation of model parameters based on this framework will also be developed.

Setting: 

For the different stages of the study, the project will involve middle school and college students. The college students will be recruited from a northeastern university, whereas the middle school students from several school district within the same state.

Research Design: 

This cross-sectional project is both descriptive and quasi-experimental in design, collecting original data using assessments of learning/ achievement tests, paper and pencil surveys, and a verbal protocol. The project involves development of new psychometric models that will provide finer-grained inferences about what students know and do not know. Consequently, one of the major tasks of this project is to develop an estimation code for the appropriate inferences to be extracted from assessment data using these models. The project will also involve several validation studies to determine the nature of the constructs being measured, and whether they reflect what students can and cannot do. In addition, it will also be determined whether these constructs are predictive of their future performance.

Findings: 

One new psychometric model has been developed. This is a general model that subsumes several existing models for diagnostic purposes.

Publications & Presentations: 

de la Torre, J. (2009, March). The generalized DINA model framework. Paper presented at the Department of Development Colloquium, Teachers College, Columbia University.

de la Torre, J. (2009, February). The generalized DINA model framework. Paper presented at the Department of Measurement, Statistics, & Evaluation Colloquium, University of Maryland.

de la Torre, J. (2008, December). New models for cognitive diagnosis. Paper presented at the Psychometrics & Quantitative Psychology Colloquium, Department of Psychology, Fordham University.

de la Torre, J. (2008, November). A general framework for estimating and testing cognitive diagnosis models. Paper presented at the Annual Meeting of the Korean Society for Educational Evaluation, Seoul National University, Seoul, South Korea.

Other Products: 

This project expects to produce a grade 8 Proportional Reasoning Assessment that can provide finer-grained inferences. The data will be referenced in the presentations and papers that will come out of this project, and can be made available by contacting the PI.