Clustering and Profiling Students According to their Interactions with an Intelligent Tutoring System Fostering Self-Regulated Learning



Published May 1, 2013
François Bouchet Jason M. Harley Gregory J. Trevors Roger Azevedo


In this paper, we present the results obtained using a clustering algorithm (Expectation-Maximization) on data collected from 106 college students learning about the circulatory system with MetaTutor, an agent-based Intelligent Tutoring System (ITS) designed to foster self-regulated learning (SRL). The three extracted clusters were validated and analyzed using multivariate statistics (MANOVAs) in order to characterize three distinct profiles of students, displaying statistically significant differences over all 12 variables used for the clusters formation (including performance, use of note-taking and number of sub-goals attempted). We show through additional analyses that variations also exist between the clusters regarding prompts they received by the system to perform SRL processes. We conclude with a discussion of implications for designing a more adaptive ITS based on an identification of learners' profiles

How to Cite

Bouchet, F., Harley, J. M., Trevors, G. J., & Azevedo, R. (2013). Clustering and Profiling Students According to their Interactions with an Intelligent Tutoring System Fostering Self-Regulated Learning. Journal of Educational Data Mining, 5(1), 104–146.
Abstract 1498 | PDF Downloads 945



profiling, cluster, expectation-maximization, intelligent tutoring system, agent-based system, self-regulated learning, metacognition, adaptivity

ALEVEN, V., ROLL, I., MCLAREN, B.M., AND KOEDINGER, K.R. 2010. Automated, unobtrusive, action-by-action assessment of self-regulation during learning with an intelligent tutoring system. Educational Psychologist, 45 (4) 224-233.

AMERSHI, S., AND CONATI, C. 2007. Unsupervised and supervised machine learning in user modeling for intelligent learning environments. In Proceedings of the 12th international conference on Intelligent user interfaces, IUI '07, New York, NY, USA: ACM, 72-81.

ARROYO, I., MURRAY, T., WOOLF, B., AND BEAL, C. 2004. Inferring Unobservable Learning Variables from Students' Help Seeking Behavior. In Lecture Notes in Computer Science, Vol. 3220. Springer, Berlin Heidelberg New York, 782-784.

AZEVEDO, R., AND ALEVEN, V, Eds. in press. International handbook of metacognition and learning technologies. Amsterdam, The Netherlands: Springer.

AZEVEDO, R., BEHNAGH, R., DUFFY, M., HARLEY, J., AND TREVORS, G. 2012. Metacognition and self-regulated learning in student-centered leaning environments. D. JONASSEN AND S. LAND, Eds. Theoretical foundations of student-center learning environments (2 nd ed.), New York: Routledge, 171-197.

AZEVEDO, R., CROMLEY, J.G., MOOS, D.C., GREENE, J.A., AND WINTERS, F.I. 2011. Adaptive content and process scaffolding: A key to facilitating students' self-regulated learning with hypermedia. Psychological Testing and Assessment Modeling, 53, 106-140.

AZEVEDO, R., JOHNSON, A., BURKETT, C., FIKE, A., LINTEAN, M., CAI, Z., AND RUS, V. 2010b. The role of prompting and feedback in facilitating students' learning about science with MetaTutor. In Proceedings of the AAAI Fall Symposium on Cognitive and Metacognitive Educational Systems, R. PIRRONE, R. AZEVEDO, AND G. BISWAS, Eds, Menlo Park, CA: Association for the Advancement of Artificial Intelligence (AAAI) Press, 11-16.

AZEVEDO, R., LANDIS, R.S., FEYZI-BEHNAGH, R., DUFFY, M., TREVORS, G., HARLEY, J., BOUCHET, F., BURLISON, J., TAUB, M., PACAMPARA, N., YEASIN, M., RAHMAN, A.K.M.M., TANVEER, M.I., AND HOSSAIN, G. 2012b. The effectiveness of pedagogical agents' prompting and feedback in facilitating co-adapted learning with MetaTutor. In Proceedings of the 11th International Conference on Intelligent Tutoring Systems, ITS2011. LNCS Vol. 7315, S. A. CERRI ET AL., Eds., Springer, Berlin Heidelberg, 212-221.

AZEVEDO, R., MOOS, D., JOHNSON, A., AND CHAUNCEY, A. 2010. Measuring cognitive and metacognitive regulatory processes used during hypermedia learning: Issues and challenges. Educational Psychologist, 45 (4), 210-223.

BAKER, R.S.J.D., GOLDSTEIN, A.B., AND HEFFERNAN, N.T. in press. Detecting Learning Moment-by-Moment. To appear in International Journal of Artificial Intelligence in Education.

BAKER, R.S.J.d., and YACEF, K. 2009. The State of Educational Data Mining in 2009: A Review and Future Visions. Journal of Educational Data Mining, 1 (1), 3-17.

BISWAS, G., JEONG, H., KINNEBREW, J., SULCER, B., AND ROSCOE, R. 2010. Measuring Self-regulated Learning Skills through Social Interactions in a Teachable Agent Environment. Research and Practice in Technology-Enhanced Learning, 5 (2), 123-152.

BOUCHET, F., KINNEBREW, J. S., BISWAS, G., & AZEVEDO, R. (2012). Identifying Students' Characteristic Learning Behaviors in an Intelligent Tutoring System Fostering Self-Regulated Learning. In Proceedings of the 5th International Conference on Educational Data Mining, K. YACEF, O. ZAÏANE, A. HERSHKOVITZ, M. YUDELSON, AND J. STAMPER, Eds., Chania, Greece, 65-72.

CHANG, K., BECK, J., MOSTOW, J., CORBETT, A., AND A BAYES. 2006. Net Toolkit for Student Modeling in Intelligent Tutoring Systems. In Proceedings of the 8 th International Conference on Intelligent Tutoring Systems, ITS2006, LNCS Vol. 4053, IKEDA, M., ET AL., Eds. Springer, Berlin Heidelberg New York, 104-113.

D'MELLO, S., AND GRASSER, A. in press. Dynamics of affective states during complex learning. Learning and Instruction.

D'MELLO, S.K., LEHMAN, B., SULLINS, J., DAIGLE, R., COMBS, R., VOGT, K., PERKINS, L., AND GRAESSER, A.C. 2010. A time for emoting: When affect-sensitivity is and isn't effective at promoting deep learning. In Proceedings of the 10th Annual Conference of Intelligent Tutoring Systems Pittsburg, USA: Springer, 245-254.

FERGUSON, K., ARROYO, I., MAHADEVAN, S., WOOLF, B., AND BARTO, A. 2006. Improving Intelligent Tutoring Systems: Using Expectation Maximization to Learn Student Skill Levels. In Proceedings of the 8th International Conference on Intelligent Tutoring Systems, ITS2006, LNCS Vol. 4053, IKEDA, M., ET AL., Eds. Springer, Berlin Heidelberg New York, 453-462.

GRAESSER, A.C., D'MELLO, S.K., HU, X., CAI, Z., OLNEY, A., AND MORGAN, B. in press. AutoTutor. Applied natural language processing and content analysis: Advances in identification, investigation and resolution, P. M. MCCARTHY, AND C. BOONTHUM, Eds. Hershey, PA: IGI Global.

GRAESSER, A.C., AND MCNAMARA, D.S. 2010. Self-regulated learning in learning environments with pedagogical agents that interact in natural language. Educational Psychologist, 45, 234-244.

HADWIN, A., JARVELA, S., AND MILLER, M. 2011. Self-regulated , co-regulated, and socially shared regulation of learning. Handbook of self-regualtion of learning and performance, B. ZIMMERMAN AND D. SCHUNK, Eds. New York: Routledge, 65-84.

HALL, M., FRANK, E., HOLMES, G., PFAHRINGER, B., REUTEMANN, P., AND WITTEN, I.H. 2009. The WEKA data mining software: an update. SIGKDD Explor. Newsl., 11 (1), 10-18.

HARLEY, J., BOUCHET, F., AND AZEVEDO, R. 2011. Examining Learners' Emotional Responses to Virtual Pedagogical Agents' Tutoring Strategies. In Proceedings of 11th International Conference on Intelligent Virtual Agents, Lecture Notes in Computer Science (Vol. 6895), H.H. VILHJÁLMSSON, S. KOPP, S. MARSELLA, AND K.R. THÓRISSON, Eds. Reykjavik, Iceland: Springer, 449-450.

KINNEBREW, J.S., AND BISWAS, G. 2011. Modeling and Measuring Self-Regulated Learning in Teachable Agent Environments. Journal of e-Learning and Knowledge Society, 7 (2), 19-35.

LEELAWONG, K., AND BISWAS, G. 2008. Designing Learning by Teaching Agents: The Betty's Brain System. Int. J. Artif. Intell. Ed., 18 (3), 181-208.

MANIKANDAN, C., SUNDARAM, M.A.S., AND MAHESH, B.M. 2006. Collaborative E-Learning for Remote Education; An Approach For Realizing Pervasive Learning Environments. In Proceedings of the 2nd International Conference on Information and Automation (ICIA'2006), Colombo, Sri Lanka: IEEE, 274-278

MCQUIGGAN, S., AND LESTER, J. (2009). Modeling affect expression and recognition in an interactive learning environment. International Journal of Learning Technology, 4 (3-4), 216-233.

RODRIGO, M.M.T., ANGLO, E.A., SUGAY, J.O., AND BAKER, R.S.J. 2008. Use of Unsupervised Clustering to Characterize Learner Behaviors and Affective States while Using an Intelligent Tutoring System. In Proceedings of International Conference on Computers in Education (pp. 49-56). Asia Pacific Society for Computers in Education.

SHUTE, V.J., AND ZAPATA-RIVERA, D. 2012. Adaptive educational systems. Adaptive technologies for training and education, P. DURLACH, Ed. New York, NY: Cambridge University Press, 7-27.

TABACHNICK, B.G., AND FIDELL, L.S. 2007. Using multivariate statistics (5th ed.). New York: Harper Collins.

TALAVERA, L., AND GAUDIOSO, E. 2004. Mining Student Data To Characterize Similar Behavior Groups In Unstructured Collaboration Spaces. Proc. of the ECAI 2004 Workshop on Artificial Intelligence in Computer Supported Collaborative Learning, Valencia, Spain, 17-22.

TENG, C.-W., LIN, C.-S., CHENG, S.-Y., AND HEH, J.-S. 2004. Analyzing User Behavior Distribution on E-Learning Platform with Techniques of Clustering. Society for Information Technology & Teacher Education International Conference, 2004 (1), 3052-3058.

TIAN, F., WANG, S., ZHENG, C., AND ZHENG, Q. 2008. Research on e-learner personality grouping based on fuzzy clustering analysis. In Proceedings of 12th International Conference on Computer Supported Cooperative Work in Design (CSCWD'2008), Xi'an, China: IEEE, 1035-1040. TOBII STUDIO TM (Version 1.5) [Computer Software]. 2009. Stockholm, Sweden: Tobii Technology. TOBII T60 [Apparatus]. 2009. Stockholm, Sweden: Tobii Technology.

VANLEHN, K. 2011. The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring system. Educational Psychologist, 46 (4), 197-221.

VELLIDO, A., CASTRO, F., AND NEBOT, A. 2010. Clustering Educational Data. Handbook of Educational Data Mining, Data Mining and Knowledge Discovery, C. ROMERO, S. VENTURA, M. PECHENIZKIY, AND R.S.J.D. BAKER, Eds. CRC Press, 75-92.

VELLIDO, A., CASTRO, F., NEBOT, A., AND MUGICA, F. 2006. Characterization of atypical virtual campus usage behavior through robust generative relevance analysis. In Proceedings of the 5th IASTED international conference on Web-based education. Anaheim, CA, USA: ACTA Press, 183–188.

VICARVISION. 2010. FaceReader (3.1) [Computer software]. Wageningen, The Netherlands: Noldus Information Technology.

WHITE, B.Y., FREDERIKSEN, J.R., AND COLLINS, J. 2009. The interplay of scientific inquiry and metacognition. Handbook of Metacognition, D.J. HACKER, J. DUNLOSKY, AND A.C. GRAESSER, Eds. New York, NY: Routledge, 175-205.

WINNE, P., AND HADWIN, A. 1998. Studying as self-regulated learning. Metacognition in educational theory and practice, D. HACKER, J. DUNLOSKY, AND A. GRAESSER, Eds. Mahwah, NJ: Erlbaum, 227-304.

WINNE, P., AND HADWIN, A. 2008. The weave of motivation and self-regulated learning. Motivation and self-regulated learning: Theory, research, and applications, D. SCHUNK & B. ZIMMERMAN, Eds. Mahwah, NJ: Erlbaum, 297-314.

WINNE, P.H., AND NESBIT, J.C. 2009. Supporting self-regulated learning with cognitive tools. Handbook of metacognition in education, D.J. HACKER, J. DUNLOSKY, AND A.C. GRAESSER, Eds. Mahwah, NJ: Erlbaum, 259-277.

WOOLF, B. 2009. Building intelligent interactive tutors: Student centered strategies for revolutionizing e-learning. Burlington, MA: Morgan Kaufmann.

ZAKRZEWSKA, D. 2008. Using Clustering Technique for Students' Grouping in Intelligent E-Learning Systems. HCI and Usability for Education and Work, LNCS (Vol. 5298), A. HOLZINGER, Ed. Berlin / Heidelberg: Springer-Verlag, 403-410.