Combining Unsupervised and Supervised Classification to Build User Models for Exploratory Learning Environments

##plugins.themes.bootstrap3.article.main##

##plugins.themes.bootstrap3.article.sidebar##

Published Oct 1, 2009
Saleema Amershi Cristina Conati

Abstract

In this paper, we present a data-based user modeling framework that uses both unsupervised and supervised classification to build student models for exploratory learning environments. We apply the framework to build student models for two different learning environments and using two different data sources (logged interface and eye-tracking data). Despite limitations due to the size of our datasets, we provide initial evidence that the framework can automatically identify meaningful student interaction behaviors and can be used to build user models for the online classification of new student behaviors online. We also show framework transferability across applications and data types.

How to Cite

Amershi, S., & Conati, C. (2009). Combining Unsupervised and Supervised Classification to Build User Models for Exploratory Learning Environments. Journal of Educational Data Mining, 1(1), 18–71. https://doi.org/10.5281/zenodo.3554659
Abstract 1930 | PDF Downloads 1566

##plugins.themes.bootstrap3.article.details##

Keywords

data mining, unsupervised and supervised classification, user modeling, intelligent learning environments, exploratory learning environments

References
AGRAWAL, R., IMILINSKI, T., AND SWAMI, A. N. 1993. Mining Association Rules between Sets of Items in Large Databases In Proceedings of the ACM SIGMOD Conference on the Management of Data.

AMERSHI, S., ARKSEY, N., CARENINI, G., CONATI, C., MACKWORTH, A., MACLAREN, H., AND POOLE, D. 2005. Designing CIspace: Pedagogy and Usability in a Learning Environment for AI. In Proceedings of the ACM SIGCSE Conference on Innovation and Technology in Computer Science Education, 178-182.

AMERSHI, S., CARENINI, C., CONATI, C., MACKWORTH, A., POOLE, D. 2008. Pedagogy and Usability in Interactive Visualizations - Designing and Evaluating CIspace. Interacting with Computers - The Interdisciplinary Journal of Human-Computer Interaction 20 (1), 64-96.

AMERSHI, S., AND CONATI, C. 2006. Automatic Recognition of Learner Groups in ELEs. In Proceedings of Intelligent Tutoring Systems, 463-472.

AMERSHI, S., AND CONATI, C. 2007. Unsupervised and Supervised Machine Learning in User Modeling for Intelligent Learning Environments. In Proceedings of Intelligent User Interfaces, 72-81.

ARROYO, I., BECK, J., BEAL, C., WING, R., AND WOOLF, B. P. 2001. Analyzing Students' Response to Help Provision in an Elementary Mathematics Intelligent Tutoring System. In Proceedings of the AIED Workshop on Help Provision and Help Seeking in Interactive Learning Environments.

ARROYO, I., FERGUSON, K., JOHNS, J., DRAGON, T., MEHERANIAN, H., FISHER, D., BARTO, A., MAHADEVAN, S., AND WOOLF. B.P. 2007. Repairing disengagement with non-invasive interventions. In Proceedings of the 13 th International Conference on Artificial Intelligence in Education, 195–202.

AUMANN, Y., AND LINDELL, Y. 2005. A Statistical Theory For Quantitative Association Rules. Journal of Intelligent Information Systems 20 (3), 255-283.

AYERS, E., NUGENT, R., AND DEAN, N. 2008. Skill Set Profile Clustering Based on Weighted Student Responses. In Proceedings of the 1 st International Conference on Educational Data Mining, 210-217.

BAKER, R.S.J.D., CORBETT, A.T., KOEDINGER, K.R., EVENSON, E., ROLL, I., WAGNER, A.Z., NAIM, M., RASPAT, J., BAKER, D.J., AND BECK, J. 2006. Adapting to When Students Game an Intelligent Tutoring System. In Proceedings of the 8th International Conference on Intelligent Tutoring Systems, 392-401.

BAKER, R.S.J.D., CORBETT, A.T., ROLL, I. AND KOEDINGER, K.R. 2008. Developing a Generalizable Detector of When Students Game the System. User Modeling and User-Adapted Interaction 18 (3), 287-314.

BAKER, R.S.J.D. AND DE CARVALHO, A.M.J.A. 2008. Labeling Student Behavior Faster and More Precisely with Text Replays. In Proceedings of the 1st International Conference on Educational Data Mining, 38-47.

BECK, J. 2005. Engagement Tracing: Using Response Times to Model Student Disengagement. In Proceedings of the International Conference on Artificial Intelligence in Education.

BECK, J., AND WOOLF, B. P. 2000. High-Level Student Modeling with Machine Learning. In Proceedings of Intelligent Tutoring Systems.

BELLMANN, R. 1961. Adaptive Control Processes: A Guided Tour. Princeton University Press. BEN-ARI, M. 1998. Constructivism in Computer Science Education. In Proceedings of the ACM SIGSCE Conference.

BUNT, A., AND CONATI, C. 2002. Assessing Effective Exploration in Open Learning Environments Using Bayesian Networks. In Proceedings of the International Conference on Intelligent Tutoring Systems.

BUNT, A., AND CONATI, C. 2003. Probabilistic Student Modeling to Improve Exploratory Behavior. UMUAI 13 (3), 269-309.

BUNT, A., CONATI, C., HUGGETT, M., AND MULDNER, K. 2001. On Improving the Effectiveness of Open Learning Environments through Tailored Support for Exploration. In Proceedings of the International Conference on Artificial Intelligence in Education.

CARBONETTO, P., DE FREITAS, N., GUSTAFSON, P., AND THOMPSON, N. 2003. Bayesian Feature Weighting for Unsupervised Learning with Application to Object Recognition. In Proceedings of the International Workshop on Artificial Intelligence and Statistics.

CHI, M. T. H., BASSOK, M., LEWIS, M., REIMANN, P., AND GLASER, R. 1989. Self- explanations: How students study and use examples in learning to solve problems. Cognitive Science 13, 145-182.

COHEN, J. 1988. Statistical Power Analysis for the Behavioral Sciences (2 nd ed.). Hillsdale: Lawrence Earlbaum Associates.

CONATI, C., AND MERTEN, C. (To Appear). Gaze-Tracking for User Modeling in Intelligent Learning Environments: an Empirical Evaluation. Knowledge Based Systems (Techniques and Advances in IUIs).

CONATI, C., MERTEN, C., MULDNER, K., AND TERNES, D. 2005. Exploring Eye Tracking to Increase Bandwidth in User Modeling. In Proceedings of the International Conference on User Modeling.

CONATI, C., AND VANLEHN, K. 2000. Toward Computer-Based Support of Meta-Cognitive Skills: A Computational Framework to Coach Self-Explanation. Artificial Intelligence in Education 11, 398-415.

CONATI, C., AND VANLEHN, K. 2002. Using Bayesian Networks to Manage Uncertainty in Student Modeling. User Modeling and User-Adapted Interaction 12 (4), 371-417.

CORBETT, A. T., MCLAUGHLIN, M. S., AND SCARPINATTO, K. C. 2000. Modeling student knowledge: Cognitive tutors in high school and college. User Modeling and User-Adapted Interaction 10, 81-108.

DASH, M., CHOI, K., SCHEUERMANN, P., AND LIU, H. 2002. Feature Selection for Clustering - A Filter Solution. In Proceedings of the IEEE International Conference on Data Mining.

DASH, M., AND LIU, H. 2000. Feature Selection for Clustering. In Proceedings of PACKDDM.

DASH, M., LIU, H., AND YAO, J. 1997. Dimensionality Reduction for Unsupervised Data. In Proceedings of the IEEE International Conference on Tools with Artificial Intelligence.

DE VICENTE, A. AND PAIN, H. 2002. Informing the Detection of the Students' Motivational State: An Empirical Study. In Proceedings of Intelligent Tutoring Systems, 933-943.

D'MELLO, S.K., CRAIG, S.D., WITHERSPOON, A. W., MCDANIEL, B. T., AND GRAESSER, A. C. 2008. Automatic Detection of Learner’s Affect from Conversational Cues. User Modeling and User-Adapted Interaction, 18(1).

DUDA, R. O., HART, P. E., AND STORK, D. G. 2001. Pattern Classification (2 nd ed.). New York: Wiley-Interscience.

FARAWAY, J. J. 2002. Practical Regression and Anova using R. FERGUSON-HESSLER, M., AND JONG, T. D. 1990. Studying Physics Texts: Differences in Study Processes Between Good and Poor Performers. Cognition and Instruction 7 (1), 41-54.

FISHER, R. A. 1936. The Use of Multiple Measurements in Taxonomic Problems. Annals of Eugenics 7 (2), 179-188.

FRIEDMAN, J. H., AND MEULMAN, J. J. 2004. Clustering Objects on Subsets of Attributes. Journal of the Royal Statistical Society, Series B, 66, 815-849.

GAMA, C. 2004. Metacognition in Interactive Learning Environments: The Reflection Assistant Model. In Proceedings of Intelligent Tutoring Systems.

GORNIAK, P. J., AND POOLE, D. 2000. Building a Stochastic Dynamic Model of Application Use. In Proceedings of UAI.

HUNDHAUSEN, C. D., DOUGLAS, S. A., AND STASKO, J. T. 2002. A Meta-Study of Algorithm Visualization Effectiveness. Visual Languages and Computing 13(3), 259-290.

HUNT, E., AND MADHYASTHA, T. 2005. Data Mining Patterns of Thought. In Proceedings of the AAAI Workshop on Educational Data Mining.

JAIN, A. K., DUIN, R. P. W., AND MAO, J. 2000. Statistical Pattern Recognition: A Review. IEEE Trans. on Pattern Analysis and Machine Intelligence 22(1), 4-37.

JAIN, A. K., MURTY, M. N., AND FLYNN, P. J. 1999. Data Clustering: A Review. ACM Computing Surveys 31(3), 264-323.

JOHNS, J., AND WOOLF, B. 2006. A dynamic mixture model to detect student motivation and proficiency. In Proceedings of the 21 st National Conference on Artificial Intelligence, 163-168.

KEARNS, M., AND RON, D. 1997. Algorithmic Stability and Sanity-Check Bounds for Leave- One-Out Cross-Validation. In Proceedings of Computational Learning Theory.

KIRA, K. AND RENDELL, L. 1992 A practical approach to feature selection. In Proceedings of the Ninth International Conference on Machine learning, 249-256.

KIRSCHNER, P., SWELLER, J., AND CLARK, R. 2006. Why minimal guidance during instruction does not work: an analysis of the failure of constructivist, discovery, problem-based, experimental and inquiry-based teaching. Educational Psychologist 41 (2), 75-86.

KOHAVI, R. AND JOHN, G.H. 1997 Wrappers for feature subset selection. Artificial Intelligence 1-2, 273-324

KUSHMERICK, N., AND LAU, T. 2005. Automated Email Activity Management: An Unsupervised Learning Approach. In Proceedings of the Intelligent User Interfaces.

LANGE, T., BRAUN, M. L., ROTH, V., AND BUHMANN, J. M. 2003. Stability-Based Model Selection. In Proceedings of NIPS.

LAW, M., FIGUEIREDO, M., AND JAIN, A. K. 2004. Simultaneous Feature Selection and Clustering Using Mixture Models. IEEE Trans. on Pattern Analysis and Machine Intelligence 26(9), 1154-1166.

MAYO, M., AND MITROVIC, A. 2001. Optimizing ITS Behavior with Bayesian Networks and Decision Theory. Artificial Intelligence in Education 12, 124-153.

MERCERON, A., AND YACEF, K. 2005. TADA-Ed for Educational Data Mining. Interactive Mulitmedia Electronic Journal of Computer-Enhanced Learning 7(1).

MERTEN, C., AND CONATI, C. 2006. Eye-Tracking to Model and Adapt to User Meta-Cognition in Intelligent Learning Environments. In Proceedings of Intelligent User Interfaces.

MIKSATKO, J. AND MCLAREN, B. 2008. What's in a Cluster? Automatically Detecting Interesting Interactions in Student E-Discussions. In Proceedings of Intelligent Tutoring Systems, 333-342.

MOBASHER, B., COOLEY, R., AND SRIVASTAVA, J. 2000. Automatic Personalization Based on Web Usage Mining. Communications of the ACM 43(8), 142-151.

MAVRIKIS, M. 2008. Data-driven modeling of students' interactions in an ILE. In Educational Data Mining 2008: 1 st International Conference on Educational Data Mining, 87-96.

MURRAY C. AND VANLEHN K. 2005 Effects of dissuading unnecessary help requests while providing proactive help. In Proceedings of AIED.

MUTTER, S. 2004. Classification Using Association Rules. Freidburg im Breisgau, Germany.

NAKAGAWA, M., AND MOBASHER, B. 2003. Impact of Site Characteristics on Recommendation Models Based on Association Rules and Sequential Patterns. In Proceedings of the IJCAI'03 Workshop on Intelligent Techniques for Web Personalization.

NAPS, T. L., RODGER, S., VELZQUEZ-ITURBIDE, J., RÖßLING, G., ALMSTRUM, V., DANN, W., ET AL. 2003. Exploring the Role of Visualization and Engagment in Computer Science Education. ACM SIGCSE Bulletin 35(2), 131-152.

OLEJNIK, S., AND ALGINA, J. 2000. Measures of Effect Size for Comparative Studies: Applications, Interpretations, and Limitations. Contemporary Educational Psychology 25, 241- 286.

PAZZANI, M.J., AND BILLSUS, D. 2007. Content-Based Recommendation Systems. The Adaptive Web, 325-341.

PERERA, D., J. KAY, K. YACEF, I. KOPRINSKA, AND O. ZAIANE. (In Press) Clustering and Sequential Pattern Mining of Online Collaborative Learning Data, To Appear in Proceedings of the IEEE Transactions on Knowledge and Data Engineering.

PERKOWITZ, M., AND ETZIONI, O. 2000. Towards Adaptive Web Sites: Conceptual Framework and Case Study. AI 118 (1-2), 245-275.

PIAGET, J. 1954. The Construction of Reality in the Child. New York: Basic Books.

POOLE, D., MACKWORTH, A., AND GOEBEL, R. 1998. Computational Intelligence: A Logical Approach. New York: Oxford University Press.

ROBARDET, C., CREMILLIEUX, B., AND BOULICAUT, J. 2002. Characterization of Unsupervised Clustering with the Simplest Association Rules: Application for Child's Meningitis. In Proceedings of the International Intelligent Workshop on Data Analysis in Biomedicine and Pharmacology, Co-located with the European Conference on Artificial Intelligence.

ROMERO, C., VENTURA, S., ESPEJO, P.G., AND HERVAS, C. 2008. Data Mining Algorithms to Classify Students. In Proceedings of Educational Data Mining, 8-17.

SCHAFER, J.B., FRANKOWSKI, D., HERLOCKER, J., AND SEN, S. 2007. Collaborative Filtering Recommender Systems. The Adaptive Web.

SCHNEIDERMAN, B. 2003. Promoting Universal Usability with Multi-Layer Interface Design. In Proceedings of the ACM Conference on Universal Usability.

SHIH, B., KOEDINGER, K., AND SCHEINES, R. 2008. A Response Time Model for Bottom-Out Hints as Worked Examples. In Proceedings of the 1 st International Conference on Educational Data Mining.

SHUTE, V. 1994. Discovery learning environments: Appropriate for all? In Proceedings of the American Educational Research Association, New Orleans, LA.

SHUTE, V., AND GLASER, V. 1990. A large-scale evaluation of an intelligent discovery world. Interactive Learning Environments 1, 51-76.

SHUTE, V. J. 1993. A comparison of learning environments: All that glitters... In S. Lajoie, P. & S. Derry (Eds.), Computers as Cognitive Tools (pp. 47-73). Hillsdale, NJ: Lawrence Erlbaum Associates.

SISON, R., NUMAO, M., AND SHIMURA, M. 2000. Multistrategy Discovery and Detection of Novice Programmer Errors. Machine Learning 38, 157-180.

STERN, L., MARKHAM, S., AND HANEWALD, R. 2005. You Can Lead a Horse to Water: How Students Really Use Pedagogical Software. In Proceedings of the ACM SIGCSE Conference on Innovation and Technology in Computer Science Education.

SOLLER, A. 2004. Computational Modeling and Analysis of Knowledge Sharing in Collaborative Distance Learning. User Modeling and User-Adapted Interaction 14(4), 351-381.

SUAREZ M AND SISON R. 2008. Automatic Construction of a Bug Library for Object Oriented Novice Java Programming Errors. In Proceedings of Intelligent Tutoring Systems.

TALAVERA, L., AND GAUDIOSO, E. 2004. Mining Student Data to Characterize Similar Behavior Groups in Unstructured Collaboration Spaces. In Proceedings of the European Conference on AI Workshop on AI in CSCL.

TANG, T., AND MCCALLA, G., 2005. Smart recommendation for an evolving e-learning system. International Journal on E-Learning 4 (1), 105-129.

WALONOSKI, J.A., AND HEFFERNAN, N.T. 2006. Detection and analysis of off-task gaming behavior in intelligent tutoring systems. In Proceedings of the 8th International Conference on Intelligent Tutoring Systems, 382–391.

WEISS, G. M., AND PROVOST, F. 2001. The Effect of Class Distribution on Classifier Learning: An Empirical Study. (Technical No. ML-TR-44): Rutgers Univ.

ZAIANE, O. 2002. Building a Recommender Agent for e-Learning Systems. In Proceedings of the International Conference on Computers in Education.

ZAIANE, O., AND LUO, J. 2001. Towards Evaluating Learners' Behaviour in a Web-based Distance Learning Environment. In Proceedings of the IEEE International Conference on Advanced Learning Technologies.

ZUKERMAN, I., AND ALBRECHT, D. W. 2001. Predictive Statistical Models for User Modeling. In User Modeling and User-Adapted Interaction.
Section
Articles