S-CASTS: A System for Collaboration Among Students, Teachers and Software
We developed and evaluated four original algorithms for recognizing user’s activities within solving probability problems using TInkerPlots a commercial pedagogical software used internationally to support statistics education. We carried out empirical evaluations of the four algorithms, based on 43 user interactions with TinkerPlots,. We found an expected tradeoff between computational complexity and accuracy; the polynomial algorithm identified user activities in 27 of 43 user interactions (63%), while the other three increasingly exponentially algorithms identified activities in 33 (77%), 40 (93%), and 40 (93%) interactions, respectively. We also examined the problem of expressing potential user strategies, or recipes, in a compact recipe language, and discussed the limitations arising from various recipe language assumptions.
Our laboratory study showed that under controlled conditions, plan recognition algorithms can be used pragmatically despite theoretical worst-case exponential complexity. Our next step is to see whether their performance can generalize to classroom settings involving middle school students using TinkerPlots to solve statistics problems of varying kinds and complexities. These conditions are more challenging in that students' interaction process with TinkerPlots is longer and more involved, and that the size of the recipe data base will be considerably larger than in our lab study.
Examining the results of the plan recognition process has also led us to consider two branches to our work that were not included in our original conceptualization of the project.
- Plan recognition, while it can accurately identify a student’s plan, essentially hides student actions that are not part of the plan. We also know that as the problems that students are solving become more complex, the recipe library becomes unwieldy, difficult to assemble and difficult to use. These two realizations have led us to investigate other representations of students’ work that do not rely on plan recognition. This will be the focus of the next year of the project.
- We have spoken to a number of people who do research on statistical thinking using TinkerPlots and they have expressed an interest in the kinds of results we can generate. It may be, in fact, that researchers will be able to take advantage of our work at least as much as teachers. We will thus be expanding our work to consider researchers as an audience as well as teachers.
Y. Gal, E. Yamangil, A. Rubin, S. Shieber, and B.J. Grosz. Towards Collaborative Intelligent Tutors: Aautomated Recognition of User's Strategies. Proceedings of the Ninth International Conference on Intelligent Tutoring Systems. Montreal, Quebec, June 2008.
S. Reddy, “Constraint Satisfaction-Aided Plan Recognition,” A.B. Thesis, Harvard University, Cambridge, MA, 2009.
S. Reddy, Y. Gal, and S. Shieber. Recognition of User's Activities Using Constraint Satisfaction. Proceedings of the Seventeenth International Conference on User Modeling, Adaptation, and Personalization. Trento, Italy, June 2009.
By the end of the project, we will have a set of “analysis and presentation” tools that will provide teachers and researchers with organized, useful information about students’ work with educational software. This information will be available both for individual students and for the students as a group (i.e. aggregate information).
