Students’ Attempts at Understanding the Unobservable: A Multi-Method Approach to Visualization Analysis and Design

Illinois
Descriptive
Associative
Causal
Science
Cognitive Science
Other
Higher Education
Principal Investigator: 
David Rapp
Project Overview
Background & Purpose: 

The direct aim of this empirical project is to understand the processes that underlie, and the learning consequences that result from, student experiences with visualizations of scientific phenomena at an unobservable scale. The proposed project implements a multi-method approach to evaluate students’ moment-by-moment processing and resulting comprehension of instructional materials conveying core chemistry concepts. The goal is to identify whether students’ individual differences influence the effectiveness of visualizations for learning scientific concepts as measured with process and product-based metrics of evaluation; based on the profiles of performance derived from these measures, the project will also develop design principles for the use of visualizations as instructional tools in chemistry, and STEM fields more broadly.

Setting: 

We will study first and second-year, college-level chemistry undergraduates at four institutions (Northwestern University, Texas A&M, Tufts University, and the University of New Brunswick).

Research Design: 

The research design for this project is comparative, and is designed to generate research which is descriptive (observational), associative and/or correlational (quasi-experimental), and causal (experimental). This project collects original data using: paper and pencil and online self-completion questionnaires; structured and informal face-to-face interviews; eye tracking analyses; and reaction time and recall performance. Data will be analyzed using eye tracking analytical packages, coded interview protocols, and memory recall protocols. The data will be analyzed using ANOVAs, correlational methods, and regressions.

Findings: 

Before developing a formal proposal, members of our team examined the viability of a multi-method approach to the examination of visualization activity at a much smaller scale. This pilot study evaluated students’ processing of image- and animation-based chemistry presentations. Eye tracking data were collected in combination with interview protocols and question-based testing to probe students’ use of visualizations during chemistry problem solving. An important finding from the work was that the mixed methods approach provided an informative analysis of participants’ attention to and consideration of visualization content during attempts to answer chemistry questions. Overall, the data served to demonstrate the utility of coupling eye tracking with interviews to assess moment-by-moment processing, and of the potential for utilizing a larger-scale, multi-method approach to consider the contributions of individual differences during chemistry visualization experiences.

Other Products: 

We hope to generate a set of guidelines in the design of chemistry, and more generally scientific, visualizations for undergraduate chemistry students.