In this paper, we present a data-based user modeling framework that uses both unsupervised and supervised classification to build student models for exploratory learning environments. We apply the framework to build student models for two different learning environments and using two different data sources (logged interface and eye-tracking data). Despite limitations due to the size of our datasets, we provide initial evidence that the framework can automatically identify meaningful student interaction behaviors and can be used to build user models for the online classification of new student behaviors online. We also show framework transferability across applications and data types.
How to Cite
data mining, unsupervised and supervised classification, user modeling, intelligent learning environments, exploratory learning environments
AMERSHI, S., ARKSEY, N., CARENINI, G., CONATI, C., MACKWORTH, A., MACLAREN, H., AND POOLE, D. 2005. Designing CIspace: Pedagogy and Usability in a Learning Environment for AI. In Proceedings of the ACM SIGCSE Conference on Innovation and Technology in Computer Science Education, 178-182.
AMERSHI, S., CARENINI, C., CONATI, C., MACKWORTH, A., POOLE, D. 2008. Pedagogy and Usability in Interactive Visualizations - Designing and Evaluating CIspace. Interacting with Computers - The Interdisciplinary Journal of Human-Computer Interaction 20 (1), 64-96.
AMERSHI, S., AND CONATI, C. 2006. Automatic Recognition of Learner Groups in ELEs. In Proceedings of Intelligent Tutoring Systems, 463-472.
AMERSHI, S., AND CONATI, C. 2007. Unsupervised and Supervised Machine Learning in User Modeling for Intelligent Learning Environments. In Proceedings of Intelligent User Interfaces, 72-81.
ARROYO, I., BECK, J., BEAL, C., WING, R., AND WOOLF, B. P. 2001. Analyzing Students' Response to Help Provision in an Elementary Mathematics Intelligent Tutoring System. In Proceedings of the AIED Workshop on Help Provision and Help Seeking in Interactive Learning Environments.
ARROYO, I., FERGUSON, K., JOHNS, J., DRAGON, T., MEHERANIAN, H., FISHER, D., BARTO, A., MAHADEVAN, S., AND WOOLF. B.P. 2007. Repairing disengagement with non-invasive interventions. In Proceedings of the 13 th International Conference on Artificial Intelligence in Education, 195–202.
AUMANN, Y., AND LINDELL, Y. 2005. A Statistical Theory For Quantitative Association Rules. Journal of Intelligent Information Systems 20 (3), 255-283.
AYERS, E., NUGENT, R., AND DEAN, N. 2008. Skill Set Profile Clustering Based on Weighted Student Responses. In Proceedings of the 1 st International Conference on Educational Data Mining, 210-217.
BAKER, R.S.J.D., CORBETT, A.T., KOEDINGER, K.R., EVENSON, E., ROLL, I., WAGNER, A.Z., NAIM, M., RASPAT, J., BAKER, D.J., AND BECK, J. 2006. Adapting to When Students Game an Intelligent Tutoring System. In Proceedings of the 8th International Conference on Intelligent Tutoring Systems, 392-401.
BAKER, R.S.J.D., CORBETT, A.T., ROLL, I. AND KOEDINGER, K.R. 2008. Developing a Generalizable Detector of When Students Game the System. User Modeling and User-Adapted Interaction 18 (3), 287-314.
BAKER, R.S.J.D. AND DE CARVALHO, A.M.J.A. 2008. Labeling Student Behavior Faster and More Precisely with Text Replays. In Proceedings of the 1st International Conference on Educational Data Mining, 38-47.
BECK, J. 2005. Engagement Tracing: Using Response Times to Model Student Disengagement. In Proceedings of the International Conference on Artificial Intelligence in Education.
BECK, J., AND WOOLF, B. P. 2000. High-Level Student Modeling with Machine Learning. In Proceedings of Intelligent Tutoring Systems.
BELLMANN, R. 1961. Adaptive Control Processes: A Guided Tour. Princeton University Press. BEN-ARI, M. 1998. Constructivism in Computer Science Education. In Proceedings of the ACM SIGSCE Conference.
BUNT, A., AND CONATI, C. 2002. Assessing Effective Exploration in Open Learning Environments Using Bayesian Networks. In Proceedings of the International Conference on Intelligent Tutoring Systems.
BUNT, A., AND CONATI, C. 2003. Probabilistic Student Modeling to Improve Exploratory Behavior. UMUAI 13 (3), 269-309.
BUNT, A., CONATI, C., HUGGETT, M., AND MULDNER, K. 2001. On Improving the Effectiveness of Open Learning Environments through Tailored Support for Exploration. In Proceedings of the International Conference on Artificial Intelligence in Education.
CARBONETTO, P., DE FREITAS, N., GUSTAFSON, P., AND THOMPSON, N. 2003. Bayesian Feature Weighting for Unsupervised Learning with Application to Object Recognition. In Proceedings of the International Workshop on Artificial Intelligence and Statistics.
CHI, M. T. H., BASSOK, M., LEWIS, M., REIMANN, P., AND GLASER, R. 1989. Self- explanations: How students study and use examples in learning to solve problems. Cognitive Science 13, 145-182.
COHEN, J. 1988. Statistical Power Analysis for the Behavioral Sciences (2 nd ed.). Hillsdale: Lawrence Earlbaum Associates.
CONATI, C., AND MERTEN, C. (To Appear). Gaze-Tracking for User Modeling in Intelligent Learning Environments: an Empirical Evaluation. Knowledge Based Systems (Techniques and Advances in IUIs).
CONATI, C., MERTEN, C., MULDNER, K., AND TERNES, D. 2005. Exploring Eye Tracking to Increase Bandwidth in User Modeling. In Proceedings of the International Conference on User Modeling.
CONATI, C., AND VANLEHN, K. 2000. Toward Computer-Based Support of Meta-Cognitive Skills: A Computational Framework to Coach Self-Explanation. Artificial Intelligence in Education 11, 398-415.
CONATI, C., AND VANLEHN, K. 2002. Using Bayesian Networks to Manage Uncertainty in Student Modeling. User Modeling and User-Adapted Interaction 12 (4), 371-417.
CORBETT, A. T., MCLAUGHLIN, M. S., AND SCARPINATTO, K. C. 2000. Modeling student knowledge: Cognitive tutors in high school and college. User Modeling and User-Adapted Interaction 10, 81-108.
DASH, M., CHOI, K., SCHEUERMANN, P., AND LIU, H. 2002. Feature Selection for Clustering - A Filter Solution. In Proceedings of the IEEE International Conference on Data Mining.
DASH, M., AND LIU, H. 2000. Feature Selection for Clustering. In Proceedings of PACKDDM.
DASH, M., LIU, H., AND YAO, J. 1997. Dimensionality Reduction for Unsupervised Data. In Proceedings of the IEEE International Conference on Tools with Artificial Intelligence.
DE VICENTE, A. AND PAIN, H. 2002. Informing the Detection of the Students' Motivational State: An Empirical Study. In Proceedings of Intelligent Tutoring Systems, 933-943.
D'MELLO, S.K., CRAIG, S.D., WITHERSPOON, A. W., MCDANIEL, B. T., AND GRAESSER, A. C. 2008. Automatic Detection of Learner’s Affect from Conversational Cues. User Modeling and User-Adapted Interaction, 18(1).
DUDA, R. O., HART, P. E., AND STORK, D. G. 2001. Pattern Classification (2 nd ed.). New York: Wiley-Interscience.
FARAWAY, J. J. 2002. Practical Regression and Anova using R. FERGUSON-HESSLER, M., AND JONG, T. D. 1990. Studying Physics Texts: Differences in Study Processes Between Good and Poor Performers. Cognition and Instruction 7 (1), 41-54.
FISHER, R. A. 1936. The Use of Multiple Measurements in Taxonomic Problems. Annals of Eugenics 7 (2), 179-188.
FRIEDMAN, J. H., AND MEULMAN, J. J. 2004. Clustering Objects on Subsets of Attributes. Journal of the Royal Statistical Society, Series B, 66, 815-849.
GAMA, C. 2004. Metacognition in Interactive Learning Environments: The Reflection Assistant Model. In Proceedings of Intelligent Tutoring Systems.
GORNIAK, P. J., AND POOLE, D. 2000. Building a Stochastic Dynamic Model of Application Use. In Proceedings of UAI.
HUNDHAUSEN, C. D., DOUGLAS, S. A., AND STASKO, J. T. 2002. A Meta-Study of Algorithm Visualization Effectiveness. Visual Languages and Computing 13(3), 259-290.
HUNT, E., AND MADHYASTHA, T. 2005. Data Mining Patterns of Thought. In Proceedings of the AAAI Workshop on Educational Data Mining.
JAIN, A. K., DUIN, R. P. W., AND MAO, J. 2000. Statistical Pattern Recognition: A Review. IEEE Trans. on Pattern Analysis and Machine Intelligence 22(1), 4-37.
JAIN, A. K., MURTY, M. N., AND FLYNN, P. J. 1999. Data Clustering: A Review. ACM Computing Surveys 31(3), 264-323.
JOHNS, J., AND WOOLF, B. 2006. A dynamic mixture model to detect student motivation and proficiency. In Proceedings of the 21 st National Conference on Artificial Intelligence, 163-168.
KEARNS, M., AND RON, D. 1997. Algorithmic Stability and Sanity-Check Bounds for Leave- One-Out Cross-Validation. In Proceedings of Computational Learning Theory.
KIRA, K. AND RENDELL, L. 1992 A practical approach to feature selection. In Proceedings of the Ninth International Conference on Machine learning, 249-256.
KIRSCHNER, P., SWELLER, J., AND CLARK, R. 2006. Why minimal guidance during instruction does not work: an analysis of the failure of constructivist, discovery, problem-based, experimental and inquiry-based teaching. Educational Psychologist 41 (2), 75-86.
KOHAVI, R. AND JOHN, G.H. 1997 Wrappers for feature subset selection. Artificial Intelligence 1-2, 273-324
KUSHMERICK, N., AND LAU, T. 2005. Automated Email Activity Management: An Unsupervised Learning Approach. In Proceedings of the Intelligent User Interfaces.
LANGE, T., BRAUN, M. L., ROTH, V., AND BUHMANN, J. M. 2003. Stability-Based Model Selection. In Proceedings of NIPS.
LAW, M., FIGUEIREDO, M., AND JAIN, A. K. 2004. Simultaneous Feature Selection and Clustering Using Mixture Models. IEEE Trans. on Pattern Analysis and Machine Intelligence 26(9), 1154-1166.
MAYO, M., AND MITROVIC, A. 2001. Optimizing ITS Behavior with Bayesian Networks and Decision Theory. Artificial Intelligence in Education 12, 124-153.
MERCERON, A., AND YACEF, K. 2005. TADA-Ed for Educational Data Mining. Interactive Mulitmedia Electronic Journal of Computer-Enhanced Learning 7(1).
MERTEN, C., AND CONATI, C. 2006. Eye-Tracking to Model and Adapt to User Meta-Cognition in Intelligent Learning Environments. In Proceedings of Intelligent User Interfaces.
MIKSATKO, J. AND MCLAREN, B. 2008. What's in a Cluster? Automatically Detecting Interesting Interactions in Student E-Discussions. In Proceedings of Intelligent Tutoring Systems, 333-342.
MOBASHER, B., COOLEY, R., AND SRIVASTAVA, J. 2000. Automatic Personalization Based on Web Usage Mining. Communications of the ACM 43(8), 142-151.
MAVRIKIS, M. 2008. Data-driven modeling of students' interactions in an ILE. In Educational Data Mining 2008: 1 st International Conference on Educational Data Mining, 87-96.
MURRAY C. AND VANLEHN K. 2005 Effects of dissuading unnecessary help requests while providing proactive help. In Proceedings of AIED.
MUTTER, S. 2004. Classification Using Association Rules. Freidburg im Breisgau, Germany.
NAKAGAWA, M., AND MOBASHER, B. 2003. Impact of Site Characteristics on Recommendation Models Based on Association Rules and Sequential Patterns. In Proceedings of the IJCAI'03 Workshop on Intelligent Techniques for Web Personalization.
NAPS, T. L., RODGER, S., VELZQUEZ-ITURBIDE, J., RÖßLING, G., ALMSTRUM, V., DANN, W., ET AL. 2003. Exploring the Role of Visualization and Engagment in Computer Science Education. ACM SIGCSE Bulletin 35(2), 131-152.
OLEJNIK, S., AND ALGINA, J. 2000. Measures of Effect Size for Comparative Studies: Applications, Interpretations, and Limitations. Contemporary Educational Psychology 25, 241- 286.
PAZZANI, M.J., AND BILLSUS, D. 2007. Content-Based Recommendation Systems. The Adaptive Web, 325-341.
PERERA, D., J. KAY, K. YACEF, I. KOPRINSKA, AND O. ZAIANE. (In Press) Clustering and Sequential Pattern Mining of Online Collaborative Learning Data, To Appear in Proceedings of the IEEE Transactions on Knowledge and Data Engineering.
PERKOWITZ, M., AND ETZIONI, O. 2000. Towards Adaptive Web Sites: Conceptual Framework and Case Study. AI 118 (1-2), 245-275.
PIAGET, J. 1954. The Construction of Reality in the Child. New York: Basic Books.
POOLE, D., MACKWORTH, A., AND GOEBEL, R. 1998. Computational Intelligence: A Logical Approach. New York: Oxford University Press.
ROBARDET, C., CREMILLIEUX, B., AND BOULICAUT, J. 2002. Characterization of Unsupervised Clustering with the Simplest Association Rules: Application for Child's Meningitis. In Proceedings of the International Intelligent Workshop on Data Analysis in Biomedicine and Pharmacology, Co-located with the European Conference on Artificial Intelligence.
ROMERO, C., VENTURA, S., ESPEJO, P.G., AND HERVAS, C. 2008. Data Mining Algorithms to Classify Students. In Proceedings of Educational Data Mining, 8-17.
SCHAFER, J.B., FRANKOWSKI, D., HERLOCKER, J., AND SEN, S. 2007. Collaborative Filtering Recommender Systems. The Adaptive Web.
SCHNEIDERMAN, B. 2003. Promoting Universal Usability with Multi-Layer Interface Design. In Proceedings of the ACM Conference on Universal Usability.
SHIH, B., KOEDINGER, K., AND SCHEINES, R. 2008. A Response Time Model for Bottom-Out Hints as Worked Examples. In Proceedings of the 1 st International Conference on Educational Data Mining.
SHUTE, V. 1994. Discovery learning environments: Appropriate for all? In Proceedings of the American Educational Research Association, New Orleans, LA.
SHUTE, V., AND GLASER, V. 1990. A large-scale evaluation of an intelligent discovery world. Interactive Learning Environments 1, 51-76.
SHUTE, V. J. 1993. A comparison of learning environments: All that glitters... In S. Lajoie, P. & S. Derry (Eds.), Computers as Cognitive Tools (pp. 47-73). Hillsdale, NJ: Lawrence Erlbaum Associates.
SISON, R., NUMAO, M., AND SHIMURA, M. 2000. Multistrategy Discovery and Detection of Novice Programmer Errors. Machine Learning 38, 157-180.
STERN, L., MARKHAM, S., AND HANEWALD, R. 2005. You Can Lead a Horse to Water: How Students Really Use Pedagogical Software. In Proceedings of the ACM SIGCSE Conference on Innovation and Technology in Computer Science Education.
SOLLER, A. 2004. Computational Modeling and Analysis of Knowledge Sharing in Collaborative Distance Learning. User Modeling and User-Adapted Interaction 14(4), 351-381.
SUAREZ M AND SISON R. 2008. Automatic Construction of a Bug Library for Object Oriented Novice Java Programming Errors. In Proceedings of Intelligent Tutoring Systems.
TALAVERA, L., AND GAUDIOSO, E. 2004. Mining Student Data to Characterize Similar Behavior Groups in Unstructured Collaboration Spaces. In Proceedings of the European Conference on AI Workshop on AI in CSCL.
TANG, T., AND MCCALLA, G., 2005. Smart recommendation for an evolving e-learning system. International Journal on E-Learning 4 (1), 105-129.
WALONOSKI, J.A., AND HEFFERNAN, N.T. 2006. Detection and analysis of off-task gaming behavior in intelligent tutoring systems. In Proceedings of the 8th International Conference on Intelligent Tutoring Systems, 382–391.
WEISS, G. M., AND PROVOST, F. 2001. The Effect of Class Distribution on Classifier Learning: An Empirical Study. (Technical No. ML-TR-44): Rutgers Univ.
ZAIANE, O. 2002. Building a Recommender Agent for e-Learning Systems. In Proceedings of the International Conference on Computers in Education.
ZAIANE, O., AND LUO, J. 2001. Towards Evaluating Learners' Behaviour in a Web-based Distance Learning Environment. In Proceedings of the IEEE International Conference on Advanced Learning Technologies.
ZUKERMAN, I., AND ALBRECHT, D. W. 2001. Predictive Statistical Models for User Modeling. In User Modeling and User-Adapted Interaction.
Authors who publish with this journal agree to the following terms:
- The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
- Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
- The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons 4.0 License (Attribution-Noncommercial-No Derivatives 4.0 International), or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
- Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
- Noncommercial—other users (including Publisher) may not use this Work for commercial purposes;
- No Derivative Works—other users (including Publisher) may not alter, transform, or build upon this Work,with the understanding that any of the above conditions can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
- The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
- Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
- The Author represents and warrants that:
- the Work is the Author’s original work;
- the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
- the Work is not pending review or under consideration by another publisher;
- the Work has not previously been published;
- the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
- the Work contains no libel, invasion of privacy, or other unlawful matter.
- The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.