360-Degree Cameras vs Traditional Cameras in Multimodal Learning Analytics: Comparative Study of Facial Recognition and Pose Estimation
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Mar 4, 2025
Robin Jephthah Rajarathinam
Chris Palaguachi
Jina Kang
https://orcid.org/0000-0002-7063-6982
Abstract
Multimodal Learning Analytics (MMLA) has emerged as a powerful approach within the computer-supported collaborative learning community, offering nuanced insights into learning processes through diverse data sources. Despite its potential, the prevalent reliance on traditional instruments such as tripod-mounted digital cameras for video capture often results in suboptimal data quality for facial expressions and poses captured, which is crucial for understanding collaborative dynamics. This study introduces an innovative approach to overcome this limitation by employing 360-degree camera technology to capture students' facial and body features while collaborating in small working groups. A comparative analysis of 1.5 hours of video data from both traditional tripod-mounted digital cameras and 360-degree cameras evaluated the efficacy of these methods in capturing facial action units (AUs) and face and body keypoints. The use of OpenFace revealed that the 360-degree camera captured high-quality facial features more effectively than the traditional method, significantly enhancing the reliability of facial feature detection. Similarly, OpenPose analysis demonstrated that the 360-degree camera substantially improved the capture of complete body keypoints compared to the traditional setup. These findings suggest that integrating 360-degree camera technology in MMLA can provide richer data for analyzing affect and engagement in collaborative learning environments. Future research will focus on refining this technology to further enhance our understanding of the learning process.
How to Cite
Rajarathinam, R. J., Palaguachi, C., & Kang, J. (2025). 360-Degree Cameras vs Traditional Cameras in Multimodal Learning Analytics: Comparative Study of Facial Recognition and Pose Estimation. Journal of Educational Data Mining, 17(1), 157–182. https://doi.org/10.5281/zenodo.14966499
##plugins.themes.bootstrap3.article.details##
Keywords
facial features, pose estimation, affect detection, multimodal learning analytics, engagement, computer-supported collaborative learning, video recording
References
Alwahaby, H., Cukurova, M., Papamitsiou, Z., and Giannakos, M. 2022. The evidence of impact and ethical considerations of multimodal learning analytics: A systematic literature review. In The Multimodal Learning Analytics Handbook, M. Giannakos, D. Spikol, D. D. Mitri, K. Sharma, X. Ochoa, and R. Hammad, Eds. Springer International Publishing, Cham, Switzerland, Chapter 7, 289–325.
Amos, B., Ludwiczuk, B., and Satyanarayanan, M. 2016. OpenFace: A general-purpose face recognition library with mobile applications. Technical Report CM-CS-16-118, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA.
Andrade, A. 2017. Understanding student learning trajectories using multimodal learning analytics within an embodied-interaction learning environment. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference. LAK ’17. Association for Computing Machinery, New York, NY, USA, 70–79.
Baker, R. S. J. d., Gowda, S. M., Wixon, M., Kalka, J., Wagner, A. Z., Salvi, A., Aleven, V., Kusbit, G. W., Ocumpaugh, J., and Rossi, L. 2012. Towards sensor-free affect detection in cognitive tutor algebra. In Proceedings of the 5th International Conference on Educational Data Mining, K. Yacef, O. Zaïane, A. Hershkovitz, M. Yudelson, and J. Stamper, Eds. International Educational Data Mining Society, Chania, Greece, 126–133.
Baltrusaitis, T., Zadeh, A., Lim, Y. C., and Morency, L.-P. 2018. Openface 2.0: Facial behavior analysis toolkit. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). IEEE, IEEE, Xi’an, China, 59–66.
Bansal, P., Deshpande, P., and Sarawagi, S. 2021. Missing value imputation on multidimensional time series. Proc. VLDB Endow. 14, 11 (July), 2533–2545.
Botelho, A. F., Baker, R. S., and Heffernan, N. T. 2017. Improving sensor-free affect detection using deep learning. In Artificial Intelligence in Education, E. André, R. Baker, X. Hu, M. M. T. Rodrigo, and B. du Boulay, Eds. Springer International Publishing, Cham, 40–51.
Cao, Z., Hidalgo, G., Simon, T., Wei, S., and Sheikh, Y. 2021. OpenPose: Realtime multi-person 2d pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis & Machine Intelligence 43, 01, 172–186.
Chejara, P., Prieto, L. P., Rodríguez-Triana, M. J., Ruiz-Calleja, A., Kasepalu, R., Chounta, I.-A., and Schneider, B. 2023. Exploring indicators for collaboration quality and its dimensions in classroom settings using multimodal learning analytics. In Responsive and Sustainable Educational Futures: 18th European Conference on Technology Enhanced Learning, EC-TEL 2023, Aveiro, Portugal, September 4–8, 2023, Proceedings. Springer-Verlag, Berlin, Heidelberg, 60–74.
Dai, Z., McReynolds, A., and Whitehill, J. 2023. In search of negative moments: Multi-modal analysis of teacher negativity in classroom observation videos. In Proceedings of the 16th International Conference on Educational Data Mining, M. Feng, T. Käser, and P. Talukdar, Eds. International Educational Data Mining Society, Bengaluru, India, 278–285.
D’Mello, S. K., Craig, S. D., and Graesser, A. C. 2009. Multimethod assessment of affective experience and expression during deep learning. International Journal of Learning Technology 4, 3-4, 165–187.
D’Mello, S. K., Olney, A. M., Blanchard, N., Samei, B., Sun, X., Ward, B., and Kelly, S. 2015. Multimodal capture of teacher-student interactions for automated dialogic analysis in live classrooms. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. ICMI ’15. Association for Computing Machinery, New York, NY, USA, 557–566.
Ekman, P., Friesen, W. V., and Hager, J. C. 2002. Facial Action Coding System: The manual: On CD-ROM. Research Nexus.
Evens, M., Empsen, M., and Hustinx, W. 2023. A literature review on 360-degree video as an educational tool: Towards design guidelines. Journal of Computers in Education 10, 2, 325–375.
Fahid, F. M., Lee, S., Mott, B., Vandenberg, J., Acosta, H., Brush, T., Glazewski, K., Hmelo-Silver, C., and Lester, J. 2023. Effects of modalities in detecting behavioral engagement in collaborative game-based learning. In LAK23: 13th International Learning Analytics and Knowledge Conference. LAK2023. Association for Computing Machinery, New York, NY, USA, 208–218.
Graesser, A. C. and D’Mello, S. 2012. Moment-to-moment emotions during reading. The Reading Teacher 66, 3, 238–242.
Gu, Y., Zhang, H., and Kamijo, S. 2020. Multi-person pose estimation using an orientation and occlusion aware deep learning network. Sensors 20, 6.
Hu, S. and Kuh, G. D. 2002. Being (dis)engaged in educationally purposeful activities: The influences of student and institutional characteristics. Research in Higher Education 43, 5, 555–575.
Hur, P. and Bosch, N. 2022. Tracking individuals in classroom videos via post-processing openpose data. In LAK22: 12th International Learning Analytics and Knowledge Conference. LAK22. Association for Computing Machinery, New York, NY, USA, 465–471.
Hur, P., Bosch, N., Paquette, L., and Mercier, E. 2020. Harbingers of collaboration? The role of early-class behaviors in predicting collaborative problem solving. In Proceedings of the 13th International Conference on Educational Data Mining, A. N. Rafferty, J. Whitehill, C. Romero, and V. Cavalli-Sforza, Eds. International Educational Data Mining Society, Online, 104–114.
Lewis, A., Ochoa, X., and Qamra, R. 2023. Instructor-in-the-loop exploratory analytics to support group work. In Proceedings of the 13th International Learning Analytics and Knowledge Conference (LAK23). ACM, Arlington, TX, USA, 284–292.
Li, Q. and Baker, R. 2018. The different relationships between engagement and outcomes across participant subgroups in massive open online courses. Computers & Education 127, 41–65.
Liao, C.-H. and Wu, J.-Y. 2022. Deploying multimodal learning analytics models to explore the impact of digital distraction and peer learning on student performance. Computers & Education 190, 104599.
Liu, S., Liu, S., Liu, Z., Peng, X., and Yang, Z. 2022. Automated detection of emotional and cognitive engagement in mooc discussions to predict learning achievement. Computers & Education 181, 104461.
Ma, Y., Celepkolu, M., and Boyer, K. E. 2022. Detecting impasse during collaborative problem solving with multimodal learning analytics. In Proceedings of the 12th International Conference on Learning Analytics and Knowledge (LAK22). ACM, Online, 45–55.
Ma, Y., Song, Y., Celepkolu, M., Boyer, K. E., Wiebe, E., Lynch, C. F., and Israel, M. 2024. Automatically detecting confusion and conflict during collaborative learning using linguistic, prosodic, and facial cues. arXiv preprint arXiv:2401.15201.
Mallavarapu, A., Lyons, L., and Uzzo, S. 2022. Exploring the utility of social-network-derived collaborative opportunity temperature readings for informing design and research of large-group immersive learning environments. Journal of Learning Analytics 9, 1, 53–76.
Martinez-Maldonado, R., Echeverria, V., Fernandez-Nieto, G., Yan, L., Zhao, L., Alfredo, R., Li, X., Dix, S., Jaggard, H., Wotherspoon, R., Osborne, A., Shum, S. B., and Gašević, D. 2023. Lessons learnt from a multimodal learning analytics deployment in-the-wild. ACM Transactions on Computer-Human Interaction 31, 1.
Mishra, P. K., Mihailidis, A., and Khan, S. S. 2024. Skeletal video anomaly detection using deep learning: Survey, challenges, and future directions. IEEE Transactions on Emerging Topics in Computational Intelligence 8, 2, 1073–1085.
Noël, R., Miranda, D., Cechinel, C., Riquelme, F., Primo, T. T., and Munoz, R. 2022. Visualizing collaboration in teamwork: A multimodal learning analytics platform for non-verbal communication. Applied Sciences 12, 15.
Oviatt, S., Lin, J., and Sriramulu, A. 2021. I know what you know: What hand movements reveal about domain expertise. ACM Transactions on Interactive Intelligent Systems 11, 1 (Mar.).
Padrón-Rivera, G., Rebolledo-Mendez, G., Parra, P. P., and Huerta-Pacheco, N. S. 2016. Identification of action units related to affective states in a tutoring system for mathematics. Journal of Educational Technology & Society 19, 2, 77–86.
Park, J., Müller, J., Arora, B., et al. 2022. Long-term missing value imputation for time series data using deep neural networks. Neural Computing and Applications 35, 9071–9091.
Pekrun, R. 2006. The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review 18, 315–341.
Prata, D., d. Baker, R. S. J., Costa, E., Rosé, C. P., and Cui, Y. 2009. Detecting and understanding the impact of cognitive and interpersonal conflict in computer supported collaborative learning environments. In Proceedings of the 2nd International Conference on Educational Data Mining (EDM 2009). www.educationaldatamining.org, Cordoba, Spain, 131–140.
Radu, I., Tu, E., and Schneider, B. 2020. Relationships between body postures and collaborative learning states in an augmented reality study. In Artificial Intelligence in Education, I. I. Bittencourt, M. Cukurova, K. Muldner, R. Luckin, and E. Millán, Eds. Springer International Publishing, Cham, 257–262.
Rajarathinam, R. J., Palaguachi, C., and Kang, J. 2024. Enhancing multimodal learning analytics: A comparative study of facial features captured using traditional vs 360-degree cameras in collaborative learning. In Proceedings of the 17th International Conference on Educational Data Mining, B. Paaßen and C. D. Epp, Eds. International Educational Data Mining Society, Atlanta, Georgia, USA, 551–558.
Ravi, N., Gabeur, V., Hu, Y.-T., Hu, R., Ryali, C., Ma, T., Khedr, H., Rädle, R., Rolland, C., Gustafson, L., Mintun, E., Pan, J., Alwala, K. V., Carion, N., Wu, C.-Y., Girshick, R., Dollár, P., and Feichtenhofer, C. 2024. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714.
Ronda-Carracao, M. A., Santos, O. C., Fernandez-Nieto, G., and Martinez-Maldonado, R. 2021. Towards exploring stress reactions in teamwork using multimodal physiological data. In Proceedings of the 1st International Workshop on Multimodal Artificial Intelligence in Education (MAIED 2021). CEUR Workshop Proceedings, Utrecht, The Netherlands, 48–59.
Roschelle, J. 1992. Learning by collaborating: Convergent conceptual change. Journal of the Learning Sciences 2, 235–276.
Schneider, B., Sharma, K., Cuendet, S., Zufferey, G., Dillenbourg, P., and Pea, R. 2018. Leveraging mobile eye-trackers to capture joint visual attention in co-located collaborative learning groups. International Journal of Computer-Supported Collaborative Learning 13, 3 (Sept.), 241–261.
Snelson, C. and Hsu, Y.-C. 2020. Educational 360-degree videos in virtual reality: A scoping review of the emerging research. TechTrends 64, 3, 404–412.
Spikol, D., Ruffaldi, E., and Cukurova, M. 2017. Using multimodal learning analytics to identify aspects of collaboration in project-based learning. In Proceedings of the 12th International Conference on Computer Supported Collaborative Learning (CSCL). International Society of the Learning Sciences, Philadelphia, PA, USA, 263–270.
Stewart, A. E. B., Keirn, Z., and D’Mello, S. K. 2021. Multimodal modeling of collaborative problem-solving facets in triads. User Modeling and User-Adapted Interaction 31, 4, 713–751.
Sümer, O., Goldberg, P., D’Mello, S., Gerjets, P., Trautwein, U., and Kasneci, E. 2023. Multimodal engagement analysis from facial videos in the classroom. IEEE Transactions on Affective Computing 14, 2, 1012–1027.
Taylor, R. 2016. The multimodal texture of engagement: Prosodic language, gaze and posture in engaged, creative classroom interaction. Thinking Skills and Creativity 20, 83–96.
Tedre, M., Toivonen, T., Kahila, J., Vartiainen, H., Valtonen, T., Jormanainen, I., and Pears, A. 2021. Teaching machine learning in k–12 classroom: Pedagogical and technological trajectories for artificial intelligence education. IEEE Access 9, 110558–110572.
Tsai, Y.-S., Hsieh, Y.-Z., Lin, S.-S., and Chen, N.-C. 2020. The real-time depth estimation for an occluded person based on a single image and openpose method. Mathematics 8, 8, 1333.
Vrzakova, H., Amon, M. J., Stewart, A., Duran, N. D., and D’Mello, S. K. 2020. Focused or stuck together: Multimodal patterns reveal triads’ performance in collaborative problem solving. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge. LAK ’20. Association for Computing Machinery, New York, NY, USA, 295–304.
Yan, L., Martinez-Maldonado, R., Cordoba, B. G., Deppeler, J., Corrigan, D., Nieto, G. F., and Gasevic, D. 2021. Footprints at school: Modelling in-class social dynamics from students’ physical positioning traces. In LAK21: 11th International Learning Analytics and Knowledge Conference. LAK21. Association for Computing Machinery, New York, NY, USA, 43–54.
Zhou, Q., Bhattacharya, A., Suraworachet, W., Nagahara, H., and Cukurova, M. 2023. Automated detection of students’ gaze interactions in collaborative learning videos: A novel approach. In Responsive and Sustainable Educational Futures, O. Viberg, I. Jivet, P. Muñoz-Merino, M. Perifanou, and T. Papathoma, Eds. Springer Nature Switzerland, Cham, 504–517.
Zhou, Q. and Cukurova, M. 2023. Zoom lens: An mmla framework for evaluating collaborative learning at both individual and group levels. In Proceedings of the Third International Workshop on Multimodal Immersive Learning Systems (MILeS 2023). CEUR Workshop Proceedings, Aveiro, Portugal, 28–35.
Amos, B., Ludwiczuk, B., and Satyanarayanan, M. 2016. OpenFace: A general-purpose face recognition library with mobile applications. Technical Report CM-CS-16-118, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA.
Andrade, A. 2017. Understanding student learning trajectories using multimodal learning analytics within an embodied-interaction learning environment. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference. LAK ’17. Association for Computing Machinery, New York, NY, USA, 70–79.
Baker, R. S. J. d., Gowda, S. M., Wixon, M., Kalka, J., Wagner, A. Z., Salvi, A., Aleven, V., Kusbit, G. W., Ocumpaugh, J., and Rossi, L. 2012. Towards sensor-free affect detection in cognitive tutor algebra. In Proceedings of the 5th International Conference on Educational Data Mining, K. Yacef, O. Zaïane, A. Hershkovitz, M. Yudelson, and J. Stamper, Eds. International Educational Data Mining Society, Chania, Greece, 126–133.
Baltrusaitis, T., Zadeh, A., Lim, Y. C., and Morency, L.-P. 2018. Openface 2.0: Facial behavior analysis toolkit. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). IEEE, IEEE, Xi’an, China, 59–66.
Bansal, P., Deshpande, P., and Sarawagi, S. 2021. Missing value imputation on multidimensional time series. Proc. VLDB Endow. 14, 11 (July), 2533–2545.
Botelho, A. F., Baker, R. S., and Heffernan, N. T. 2017. Improving sensor-free affect detection using deep learning. In Artificial Intelligence in Education, E. André, R. Baker, X. Hu, M. M. T. Rodrigo, and B. du Boulay, Eds. Springer International Publishing, Cham, 40–51.
Cao, Z., Hidalgo, G., Simon, T., Wei, S., and Sheikh, Y. 2021. OpenPose: Realtime multi-person 2d pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis & Machine Intelligence 43, 01, 172–186.
Chejara, P., Prieto, L. P., Rodríguez-Triana, M. J., Ruiz-Calleja, A., Kasepalu, R., Chounta, I.-A., and Schneider, B. 2023. Exploring indicators for collaboration quality and its dimensions in classroom settings using multimodal learning analytics. In Responsive and Sustainable Educational Futures: 18th European Conference on Technology Enhanced Learning, EC-TEL 2023, Aveiro, Portugal, September 4–8, 2023, Proceedings. Springer-Verlag, Berlin, Heidelberg, 60–74.
Dai, Z., McReynolds, A., and Whitehill, J. 2023. In search of negative moments: Multi-modal analysis of teacher negativity in classroom observation videos. In Proceedings of the 16th International Conference on Educational Data Mining, M. Feng, T. Käser, and P. Talukdar, Eds. International Educational Data Mining Society, Bengaluru, India, 278–285.
D’Mello, S. K., Craig, S. D., and Graesser, A. C. 2009. Multimethod assessment of affective experience and expression during deep learning. International Journal of Learning Technology 4, 3-4, 165–187.
D’Mello, S. K., Olney, A. M., Blanchard, N., Samei, B., Sun, X., Ward, B., and Kelly, S. 2015. Multimodal capture of teacher-student interactions for automated dialogic analysis in live classrooms. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. ICMI ’15. Association for Computing Machinery, New York, NY, USA, 557–566.
Ekman, P., Friesen, W. V., and Hager, J. C. 2002. Facial Action Coding System: The manual: On CD-ROM. Research Nexus.
Evens, M., Empsen, M., and Hustinx, W. 2023. A literature review on 360-degree video as an educational tool: Towards design guidelines. Journal of Computers in Education 10, 2, 325–375.
Fahid, F. M., Lee, S., Mott, B., Vandenberg, J., Acosta, H., Brush, T., Glazewski, K., Hmelo-Silver, C., and Lester, J. 2023. Effects of modalities in detecting behavioral engagement in collaborative game-based learning. In LAK23: 13th International Learning Analytics and Knowledge Conference. LAK2023. Association for Computing Machinery, New York, NY, USA, 208–218.
Graesser, A. C. and D’Mello, S. 2012. Moment-to-moment emotions during reading. The Reading Teacher 66, 3, 238–242.
Gu, Y., Zhang, H., and Kamijo, S. 2020. Multi-person pose estimation using an orientation and occlusion aware deep learning network. Sensors 20, 6.
Hu, S. and Kuh, G. D. 2002. Being (dis)engaged in educationally purposeful activities: The influences of student and institutional characteristics. Research in Higher Education 43, 5, 555–575.
Hur, P. and Bosch, N. 2022. Tracking individuals in classroom videos via post-processing openpose data. In LAK22: 12th International Learning Analytics and Knowledge Conference. LAK22. Association for Computing Machinery, New York, NY, USA, 465–471.
Hur, P., Bosch, N., Paquette, L., and Mercier, E. 2020. Harbingers of collaboration? The role of early-class behaviors in predicting collaborative problem solving. In Proceedings of the 13th International Conference on Educational Data Mining, A. N. Rafferty, J. Whitehill, C. Romero, and V. Cavalli-Sforza, Eds. International Educational Data Mining Society, Online, 104–114.
Lewis, A., Ochoa, X., and Qamra, R. 2023. Instructor-in-the-loop exploratory analytics to support group work. In Proceedings of the 13th International Learning Analytics and Knowledge Conference (LAK23). ACM, Arlington, TX, USA, 284–292.
Li, Q. and Baker, R. 2018. The different relationships between engagement and outcomes across participant subgroups in massive open online courses. Computers & Education 127, 41–65.
Liao, C.-H. and Wu, J.-Y. 2022. Deploying multimodal learning analytics models to explore the impact of digital distraction and peer learning on student performance. Computers & Education 190, 104599.
Liu, S., Liu, S., Liu, Z., Peng, X., and Yang, Z. 2022. Automated detection of emotional and cognitive engagement in mooc discussions to predict learning achievement. Computers & Education 181, 104461.
Ma, Y., Celepkolu, M., and Boyer, K. E. 2022. Detecting impasse during collaborative problem solving with multimodal learning analytics. In Proceedings of the 12th International Conference on Learning Analytics and Knowledge (LAK22). ACM, Online, 45–55.
Ma, Y., Song, Y., Celepkolu, M., Boyer, K. E., Wiebe, E., Lynch, C. F., and Israel, M. 2024. Automatically detecting confusion and conflict during collaborative learning using linguistic, prosodic, and facial cues. arXiv preprint arXiv:2401.15201.
Mallavarapu, A., Lyons, L., and Uzzo, S. 2022. Exploring the utility of social-network-derived collaborative opportunity temperature readings for informing design and research of large-group immersive learning environments. Journal of Learning Analytics 9, 1, 53–76.
Martinez-Maldonado, R., Echeverria, V., Fernandez-Nieto, G., Yan, L., Zhao, L., Alfredo, R., Li, X., Dix, S., Jaggard, H., Wotherspoon, R., Osborne, A., Shum, S. B., and Gašević, D. 2023. Lessons learnt from a multimodal learning analytics deployment in-the-wild. ACM Transactions on Computer-Human Interaction 31, 1.
Mishra, P. K., Mihailidis, A., and Khan, S. S. 2024. Skeletal video anomaly detection using deep learning: Survey, challenges, and future directions. IEEE Transactions on Emerging Topics in Computational Intelligence 8, 2, 1073–1085.
Noël, R., Miranda, D., Cechinel, C., Riquelme, F., Primo, T. T., and Munoz, R. 2022. Visualizing collaboration in teamwork: A multimodal learning analytics platform for non-verbal communication. Applied Sciences 12, 15.
Oviatt, S., Lin, J., and Sriramulu, A. 2021. I know what you know: What hand movements reveal about domain expertise. ACM Transactions on Interactive Intelligent Systems 11, 1 (Mar.).
Padrón-Rivera, G., Rebolledo-Mendez, G., Parra, P. P., and Huerta-Pacheco, N. S. 2016. Identification of action units related to affective states in a tutoring system for mathematics. Journal of Educational Technology & Society 19, 2, 77–86.
Park, J., Müller, J., Arora, B., et al. 2022. Long-term missing value imputation for time series data using deep neural networks. Neural Computing and Applications 35, 9071–9091.
Pekrun, R. 2006. The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review 18, 315–341.
Prata, D., d. Baker, R. S. J., Costa, E., Rosé, C. P., and Cui, Y. 2009. Detecting and understanding the impact of cognitive and interpersonal conflict in computer supported collaborative learning environments. In Proceedings of the 2nd International Conference on Educational Data Mining (EDM 2009). www.educationaldatamining.org, Cordoba, Spain, 131–140.
Radu, I., Tu, E., and Schneider, B. 2020. Relationships between body postures and collaborative learning states in an augmented reality study. In Artificial Intelligence in Education, I. I. Bittencourt, M. Cukurova, K. Muldner, R. Luckin, and E. Millán, Eds. Springer International Publishing, Cham, 257–262.
Rajarathinam, R. J., Palaguachi, C., and Kang, J. 2024. Enhancing multimodal learning analytics: A comparative study of facial features captured using traditional vs 360-degree cameras in collaborative learning. In Proceedings of the 17th International Conference on Educational Data Mining, B. Paaßen and C. D. Epp, Eds. International Educational Data Mining Society, Atlanta, Georgia, USA, 551–558.
Ravi, N., Gabeur, V., Hu, Y.-T., Hu, R., Ryali, C., Ma, T., Khedr, H., Rädle, R., Rolland, C., Gustafson, L., Mintun, E., Pan, J., Alwala, K. V., Carion, N., Wu, C.-Y., Girshick, R., Dollár, P., and Feichtenhofer, C. 2024. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714.
Ronda-Carracao, M. A., Santos, O. C., Fernandez-Nieto, G., and Martinez-Maldonado, R. 2021. Towards exploring stress reactions in teamwork using multimodal physiological data. In Proceedings of the 1st International Workshop on Multimodal Artificial Intelligence in Education (MAIED 2021). CEUR Workshop Proceedings, Utrecht, The Netherlands, 48–59.
Roschelle, J. 1992. Learning by collaborating: Convergent conceptual change. Journal of the Learning Sciences 2, 235–276.
Schneider, B., Sharma, K., Cuendet, S., Zufferey, G., Dillenbourg, P., and Pea, R. 2018. Leveraging mobile eye-trackers to capture joint visual attention in co-located collaborative learning groups. International Journal of Computer-Supported Collaborative Learning 13, 3 (Sept.), 241–261.
Snelson, C. and Hsu, Y.-C. 2020. Educational 360-degree videos in virtual reality: A scoping review of the emerging research. TechTrends 64, 3, 404–412.
Spikol, D., Ruffaldi, E., and Cukurova, M. 2017. Using multimodal learning analytics to identify aspects of collaboration in project-based learning. In Proceedings of the 12th International Conference on Computer Supported Collaborative Learning (CSCL). International Society of the Learning Sciences, Philadelphia, PA, USA, 263–270.
Stewart, A. E. B., Keirn, Z., and D’Mello, S. K. 2021. Multimodal modeling of collaborative problem-solving facets in triads. User Modeling and User-Adapted Interaction 31, 4, 713–751.
Sümer, O., Goldberg, P., D’Mello, S., Gerjets, P., Trautwein, U., and Kasneci, E. 2023. Multimodal engagement analysis from facial videos in the classroom. IEEE Transactions on Affective Computing 14, 2, 1012–1027.
Taylor, R. 2016. The multimodal texture of engagement: Prosodic language, gaze and posture in engaged, creative classroom interaction. Thinking Skills and Creativity 20, 83–96.
Tedre, M., Toivonen, T., Kahila, J., Vartiainen, H., Valtonen, T., Jormanainen, I., and Pears, A. 2021. Teaching machine learning in k–12 classroom: Pedagogical and technological trajectories for artificial intelligence education. IEEE Access 9, 110558–110572.
Tsai, Y.-S., Hsieh, Y.-Z., Lin, S.-S., and Chen, N.-C. 2020. The real-time depth estimation for an occluded person based on a single image and openpose method. Mathematics 8, 8, 1333.
Vrzakova, H., Amon, M. J., Stewart, A., Duran, N. D., and D’Mello, S. K. 2020. Focused or stuck together: Multimodal patterns reveal triads’ performance in collaborative problem solving. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge. LAK ’20. Association for Computing Machinery, New York, NY, USA, 295–304.
Yan, L., Martinez-Maldonado, R., Cordoba, B. G., Deppeler, J., Corrigan, D., Nieto, G. F., and Gasevic, D. 2021. Footprints at school: Modelling in-class social dynamics from students’ physical positioning traces. In LAK21: 11th International Learning Analytics and Knowledge Conference. LAK21. Association for Computing Machinery, New York, NY, USA, 43–54.
Zhou, Q., Bhattacharya, A., Suraworachet, W., Nagahara, H., and Cukurova, M. 2023. Automated detection of students’ gaze interactions in collaborative learning videos: A novel approach. In Responsive and Sustainable Educational Futures, O. Viberg, I. Jivet, P. Muñoz-Merino, M. Perifanou, and T. Papathoma, Eds. Springer Nature Switzerland, Cham, 504–517.
Zhou, Q. and Cukurova, M. 2023. Zoom lens: An mmla framework for evaluating collaborative learning at both individual and group levels. In Proceedings of the Third International Workshop on Multimodal Immersive Learning Systems (MILeS 2023). CEUR Workshop Proceedings, Aveiro, Portugal, 28–35.
Issue
Section
Extended Articles from the EDM 2024 Conference
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:
- The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
- Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
- The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons 4.0 License (Attribution-Noncommercial-No Derivatives 4.0 International), or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
- Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
- Noncommercial—other users (including Publisher) may not use this Work for commercial purposes;
- No Derivative Works—other users (including Publisher) may not alter, transform, or build upon this Work,with the understanding that any of the above conditions can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
- The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
- Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
- The Author represents and warrants that:
- the Work is the Author’s original work;
- the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
- the Work is not pending review or under consideration by another publisher;
- the Work has not previously been published;
- the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
- the Work contains no libel, invasion of privacy, or other unlawful matter.
- The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.