Considering Alternate Futures to Classify Off-Task Behavior as Emotion Self-Regulation: A Supervised Learning Approach



Published May 1, 2013
Jennifer L. Sabourin Jonathan P. Rowe Bradford W. Mott James C. Lester


Over the past decade, there has been growing interest in real-time assessment of student engagement andmotivation during interactions with educational software. Detecting symptoms of disengagement, such as off-taskbehavior, has shown considerable promise for understanding students motivational characteristics duringlearning. In this paper, we investigate the affective role of off-task behavior by analyzing data from studentinteractions with CRYSTAL ISLAND, a narrative-centered learning environment for middle school microbiology.We observe that off-task behavior is associated with reduced student learning, but preliminary analyses ofstudents affective transitions suggest that off-task behavior may also serve a productive role for some studentscoping with negative affective states such as frustration. Empirical findings imply that some students may useoff-task behavior as a strategy for self-regulating negative emotional states during learning.Based on these observations, we introduce a supervised machine learning procedure for detecting whetherstudents off-task behaviors are cases of emotion self-regulation. The method proceeds in three stages. Duringthe first stage, a dynamic Bayesian network (DBN) is trained to model the valence of students emotion self reportsusing collected data from interactions with the learning environment. In the second stage, a novelsimulation process uses the DBN to generate alternate futures by modeling students affective trajectories as ifthey had engaged in fewer off-task behaviors than they did during their actual learning interactions. Thealternate futures are compared to students actual traces to produce labels denoting whether students off-taskbehaviors are cases of emotion self-regulation. In the final stage, the generated emotion self-regulation labelsare predicted using off-the-shelf classifiers and features that can be computed in run-time settings. Resultssuggest that this approach shows promise for identifying cases of off-task behavior that are emotion selfregulation.Analyses of the first two phases suggest that trained DBN models are capable of accuratelymodeling relationships between students off-task behaviors and self-reported emotional valence in CRYSTALISLAND. Additionally, the proposed simulation process produces emotion self-regulation labels with high levelsof reliability. Preliminary analyses indicate that support vector machines, bagged trees, and random forestsshow promise for predicting the generated emotion self-regulation labels, but room for improvement remains.The findings underscore the methodological potential of considering alternate futures when modeling students'emotion self-regulation processes in narrative-centered learning environments.

How to Cite

Sabourin, J., Rowe, J., Mott, B., & Lester, J. (2013). Considering Alternate Futures to Classify Off-Task Behavior as Emotion Self-Regulation: A Supervised Learning Approach. JEDM | Journal of Educational Data Mining, 5(1), 9-38. Retrieved from
Abstract 235 | PDF Downloads 176