The project uses a cross-sectional and comparative research design and will generate evidence that is descriptive [observational], associative/correlational [quasi-experimental], and causal [statistical adjustment]. Original data are being collected on a purposive sample of 40 8th grade public school science teachers and their students using diaries, observation [personal observation, videography], and survey research [self-completion questionnaire, structured interviewer-administered questionnaire, semi-structured or informal interview].
The tablet portfolio developed for this study is the central instrument examined. Teachers use the portfolio to collect evidence of instructional practice that is then reviewed and scored by trained raters. Measures to be collected from teachers include a) portfolio scores assigned by trained raters; b) background and demographic information; c) teacher and student surveys of instructional practice; d) AT-LAST (Assessing Teacher Learning About Science Teaching, Smith, 2004) test scores of teacher pedagogical content knowledge; and the ratings in the district’s own classroom observation rubric. Measures (outcomes) for students include state test scores in mathematics and science; periodic assessment scores used at the district, along with course grades and demographics; a survey of classroom practice and an “interest in science” inventory adapted from the TSSI survey (Cobern, 2000).
We will make use of Generalizability theory to analyze the reliability of the portfolio ratings. The dimensionality of the portfolio will be analyzed using a combination of Exploratory –Confirmatory factor analysis. To investigate the relation between instructional practice (portfolio ratings) and student outcomes we will employ a multi-level modelling approach with 8th grade Mathematics scores as the dependent variable and adjustments for prior scores (7th grade Mathematics scores). Finally, extensive qualitative evidence will be collected through observations and interviews to investigate teacher uses and perceptions about the portfolio, and reflection and learning processes triggered by portfolio collection.
