Assessing the Performance of Online Students - New Data, New Approaches, Improved Accuracy (Journal of Educational Data Mining, 2022)
We consider the problem of assessing the changing performance levels of individual students as they gothrough online courses. This student performance modeling problem is a critical step for building adaptiveonline teaching systems. Specifically, we conduct a study of how to utilize various types and large amountsof log data from earlier students to train accurate machine learning models that predict the performance offuture students. This study is the first to use four very large sets of student data made available recentlyfrom four distinct intelligent tutoring systems.Our results include a new machine learning approach that defines a new state of the art for logisticregression based student performance modeling, improving over earlier methods in several ways: First, weachieve improved accuracy of student modeling by introducing new features that can be easily computedfrom conventional question-response logs (e.g., features such as the pattern in the student’s most recentanswers). Second, we take advantage of features of the student history that go beyond question-responsepairs (e.g., features such as which video segments the student watched, or skipped) as well as backgroundinformation about prerequisite structure in the curriculum. Third, we train multiple specialized studentperformance models for different aspects of the curriculum (e.g., specializing in early versus later segmentsof the student history), then combine these specialized models to create a group prediction of the studentperformance. Taken together, these innovations yield an average AUC score across these four datasets of0.808 compared to the previous best logistic regression approach score of 0.767, and also outperformingstate-of-the-art deep neural net approaches. Importantly, we observe consistent improvements from each ofour three methodological innovations, in each diverse dataset, suggesting that our methods are of generalutility and likely to produce improvements for other online tutoring systems as well.