Affect, Support, and Personal Factors: Multimodal Causal Models of One-on-one Coaching
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Oct 31, 2021
Lujie Karen Chen
Joseph Ramsey
Artur Dubrawski
https://orcid.org/0000-0002-7185-8405
Abstract
Human one-on-one coaching involves complex multimodal interactions. Successful coaching requires teachers to closely monitor students' cognitive-affective states and provide support of optimal type, timing, and amount. However, most of the existing human tutoring studies focus primarily on verbal interactions and have yet to incorporate the rich aspects of multimodal cognitive-affective experiences. Meanwhile, the research community lacks principled methods to fully exploit complex multimodal data to uncover the causal relationships between coaching supports, students' cognitive-affective experiences, and their stable individual factors. We explore an analytical framework that is explainable and amenable to incorporating domain knowledge. The proposed framework combines statistical approaches in Sparse Multiple Canonical Correlation, causal discovery, and inference methods for observations. We demonstrate this framework using a multimodal one-on-one math problem-solving coaching dataset collected in naturalistic home environments involving parents and young children. The insights derived from our analyses may inform the design of effective technology-inspired interventions that are personalized and adaptive.
How to Cite
Chen, L. K., Ramsey, J., & Dubrawski, A. (2021). Affect, Support, and Personal Factors: Multimodal Causal Models of One-on-one Coaching. Journal of Educational Data Mining, 13(3), 36–68. https://doi.org/10.5281/zenodo.5634222
##plugins.themes.bootstrap3.article.details##
Keywords
multimodal learning analytics, causal discovery, causal inference, parent coaching, affective and cognitive support
References
JOHN, O. P. AND SRIVASTAVA, S. 1999. The big five trait taxonomy: History, measurement, and theoretical perspectives. In Handbook of Personality: Theory and research, O. P. John and R. W. Robins,Eds. Guilford, 102–138.
KAPUR, M. 2014. Productive failure in learning math. Cognitive Science 38, 5, 1008–1022.
KOEDINGER, K. R. AND ALEVEN, V. 2007. Exploring the assistance dilemma in experiments with cognitive tutors. Educational Psychology Review 19, 3, 239–264.
KOEDINGER, K. R., MCLAUGHLIN, E. A., JIA, J. Z., AND BIER, N. L. 2016. Is the doer effect a causal relationship? How can we tell and why it’s important. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, D. Gasevic, G. Lynch, S. Dawson, H. Drachsler, and C. P. Rosé, Eds. 388–397.
LEHMAN, B., D’MELLO, S., CADE, W., AND PERSON, N. 2012. How do they do it? Investigating dialogue moves within dialogue modes in expert human tutoring. In International Conference on Intelligent Tutoring Systems, S. A. Cerri, W. J. Clancey, G. Papadourakis, and K. Panourgia, Eds. Springer, 557–562.
LINNENBRINK-GARCIA, L., DURIK, A. M., CONLEY, A. M., BARRON, K. E., TAUER, J. M.,KARABENICK, S. A., AND HARACKIEWICZ, J. M. 2010. Measuring situational interest in academic domains. Educational and Psychological Measurement 70, 4, 647–671.
MARTINEZ-MALDONADO, R., KAY, J., BUCKINGHAM SHUM, S., AND YACEF, K. 2019. Collocated collaboration analytics: Principles and dilemmas for mining multimodal interaction data. Human–Computer Interaction 34, 1, 1–50.
MILLER, I. W. AND NORMAN, W. H. 1979. Learned helplessness in humans: A review and attribution-theory model. Psychological Bulletin 86, 1, 93.
MU, S., CUI, M., AND HUANG, X. 2020. Multimodal data fusion in learning analytics: A systematic review. Sensors 20, 23, 6856.
OCHOA, X. 2017. Multimodal Learning Analytics. In The Handbook of Learning Analytics, 1 ed., C. Lang, G. Siemens, A. F. Wise, and D. Gasevic, Eds. Society for Learning Analytics Research, Alberta, Canada, 129–141.
PEKRUN, R. AND STEPHENS, E. J. 2010. Achievement emotions: A control-value approach. Social and Personality Psychology Compass 4, 4, 238–255.
PINTRICH, P., SMITH, D., GARCÍA, T., AND MCKEACHIE, W. 1991. A manual for the use of the motivated strategies for learning questionnaire (MSLQ). Tech. rep., University of Michigan, Ann Arbor, MI.
PORAYSKA-POMSTA, K., MAVRIKIS, M., AND PAIN, H. 2008. Diagnosing and acting on student affect: The tutor’s perspective. User Modeling and User-Adapted Interaction 18, 1-2, 125–173.
PRIETO, L. P., SHARMA, K., DILLENBOURG, P., AND JESÚS, M. 2016. Teaching analytics: Towards automatic extraction of orchestration graphs using wearable sensors. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, D. Gasevic, G. Lynch, S. Dawson, H. Drachsler, and C. P. Rosé, Eds. 148–157.
PRIETO, L. P., SHARMA, K., KIDZINSKI, Ł., RODRÍGUEZ-TRIANA, M. J., AND DILLENBOURG, P.2018. Multimodal teaching analytics: Automated extraction of orchestration graphs from wearable sensor data. Journal of Computer Assisted Learning 34, 2, 193–203.
SANTOS, O. C., SANEIRO, M., SALMERON-MAJADAS, S., AND BOTICARIO, J. G. 2014. A methodological approach to eliciting affective educational recommendations. In 2014 IEEE 14th International Conference on Advanced Learning Technologies. IEEE, 529–533.
SCHEINES, R., SILVER, E., AND GOLDIN, I. M. 2014. Discovering prerequisite relationships among knowledge components. In Proceedings of the 7th International Conference on Educational DataMining, J. C. Stamper, Z. A. Pardos, M. Mavrikis, and B. M. McLaren, Eds. 355–356.
SCHOENFELD, A. H. 2016. Learning to think mathematically: Problem solving, metacognition, and sense making in mathematics (reprint). Journal of Education 196, 2, 1–38.
SHARMA, K. AND GIANNAKOS, M. 2020. Multimodal data capabilities for learning: What can multimodal data tell us about learning? British Journal of Educational Technology 51, 5, 1450–1484.
TSUKAYAMA, E., DUCKWORTH, A. L., AND KIM, B. 2013. Domain-specific impulsivity in school-age children. Developmental Science 16, 6, 879–893.
WITTEN, D. M. AND TIBSHIRANI, R. J. 2009. Extensions of sparse canonical correlation analysis with applications to genomic data. Statistical Applications in Genetics and Molecular Biology 8, 1, 1–27.
WORSLEY, M. AND BLIKSTEIN, P. 2018. A multimodal analysis of making. International Journal ofArtificial Intelligence in Education 28, 3, 385–419.
YADEGARIDEHKORDI, E., NOOR, N. F. B. M., AYUB, M. N. B., AFFAL, H. B., AND HUSSIN, N. B.2019. Affective computing in education: A systematic review and future research. Computers &Education 142, 103649.
ZHU, G., XING, W., COSTA, S., SCARDAMALIA, M., AND PEI, B. 2019. Exploring emotional and cognitive dynamics of knowledge building in grades 1 and 2. User Modeling and User-Adapted Interaction 29, 4, 789–820.
ZIMMERMAN, B. J. 2000. Attaining self-regulation: A social cognitive perspective. In Handbook of Self-Regulation, M. Boekaerts, P. R. Pintrich, and M. Zeidner, Eds. Academic Press, San Diego, 13–39
KAPUR, M. 2014. Productive failure in learning math. Cognitive Science 38, 5, 1008–1022.
KOEDINGER, K. R. AND ALEVEN, V. 2007. Exploring the assistance dilemma in experiments with cognitive tutors. Educational Psychology Review 19, 3, 239–264.
KOEDINGER, K. R., MCLAUGHLIN, E. A., JIA, J. Z., AND BIER, N. L. 2016. Is the doer effect a causal relationship? How can we tell and why it’s important. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, D. Gasevic, G. Lynch, S. Dawson, H. Drachsler, and C. P. Rosé, Eds. 388–397.
LEHMAN, B., D’MELLO, S., CADE, W., AND PERSON, N. 2012. How do they do it? Investigating dialogue moves within dialogue modes in expert human tutoring. In International Conference on Intelligent Tutoring Systems, S. A. Cerri, W. J. Clancey, G. Papadourakis, and K. Panourgia, Eds. Springer, 557–562.
LINNENBRINK-GARCIA, L., DURIK, A. M., CONLEY, A. M., BARRON, K. E., TAUER, J. M.,KARABENICK, S. A., AND HARACKIEWICZ, J. M. 2010. Measuring situational interest in academic domains. Educational and Psychological Measurement 70, 4, 647–671.
MARTINEZ-MALDONADO, R., KAY, J., BUCKINGHAM SHUM, S., AND YACEF, K. 2019. Collocated collaboration analytics: Principles and dilemmas for mining multimodal interaction data. Human–Computer Interaction 34, 1, 1–50.
MILLER, I. W. AND NORMAN, W. H. 1979. Learned helplessness in humans: A review and attribution-theory model. Psychological Bulletin 86, 1, 93.
MU, S., CUI, M., AND HUANG, X. 2020. Multimodal data fusion in learning analytics: A systematic review. Sensors 20, 23, 6856.
OCHOA, X. 2017. Multimodal Learning Analytics. In The Handbook of Learning Analytics, 1 ed., C. Lang, G. Siemens, A. F. Wise, and D. Gasevic, Eds. Society for Learning Analytics Research, Alberta, Canada, 129–141.
PEKRUN, R. AND STEPHENS, E. J. 2010. Achievement emotions: A control-value approach. Social and Personality Psychology Compass 4, 4, 238–255.
PINTRICH, P., SMITH, D., GARCÍA, T., AND MCKEACHIE, W. 1991. A manual for the use of the motivated strategies for learning questionnaire (MSLQ). Tech. rep., University of Michigan, Ann Arbor, MI.
PORAYSKA-POMSTA, K., MAVRIKIS, M., AND PAIN, H. 2008. Diagnosing and acting on student affect: The tutor’s perspective. User Modeling and User-Adapted Interaction 18, 1-2, 125–173.
PRIETO, L. P., SHARMA, K., DILLENBOURG, P., AND JESÚS, M. 2016. Teaching analytics: Towards automatic extraction of orchestration graphs using wearable sensors. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, D. Gasevic, G. Lynch, S. Dawson, H. Drachsler, and C. P. Rosé, Eds. 148–157.
PRIETO, L. P., SHARMA, K., KIDZINSKI, Ł., RODRÍGUEZ-TRIANA, M. J., AND DILLENBOURG, P.2018. Multimodal teaching analytics: Automated extraction of orchestration graphs from wearable sensor data. Journal of Computer Assisted Learning 34, 2, 193–203.
SANTOS, O. C., SANEIRO, M., SALMERON-MAJADAS, S., AND BOTICARIO, J. G. 2014. A methodological approach to eliciting affective educational recommendations. In 2014 IEEE 14th International Conference on Advanced Learning Technologies. IEEE, 529–533.
SCHEINES, R., SILVER, E., AND GOLDIN, I. M. 2014. Discovering prerequisite relationships among knowledge components. In Proceedings of the 7th International Conference on Educational DataMining, J. C. Stamper, Z. A. Pardos, M. Mavrikis, and B. M. McLaren, Eds. 355–356.
SCHOENFELD, A. H. 2016. Learning to think mathematically: Problem solving, metacognition, and sense making in mathematics (reprint). Journal of Education 196, 2, 1–38.
SHARMA, K. AND GIANNAKOS, M. 2020. Multimodal data capabilities for learning: What can multimodal data tell us about learning? British Journal of Educational Technology 51, 5, 1450–1484.
TSUKAYAMA, E., DUCKWORTH, A. L., AND KIM, B. 2013. Domain-specific impulsivity in school-age children. Developmental Science 16, 6, 879–893.
WITTEN, D. M. AND TIBSHIRANI, R. J. 2009. Extensions of sparse canonical correlation analysis with applications to genomic data. Statistical Applications in Genetics and Molecular Biology 8, 1, 1–27.
WORSLEY, M. AND BLIKSTEIN, P. 2018. A multimodal analysis of making. International Journal ofArtificial Intelligence in Education 28, 3, 385–419.
YADEGARIDEHKORDI, E., NOOR, N. F. B. M., AYUB, M. N. B., AFFAL, H. B., AND HUSSIN, N. B.2019. Affective computing in education: A systematic review and future research. Computers &Education 142, 103649.
ZHU, G., XING, W., COSTA, S., SCARDAMALIA, M., AND PEI, B. 2019. Exploring emotional and cognitive dynamics of knowledge building in grades 1 and 2. User Modeling and User-Adapted Interaction 29, 4, 789–820.
ZIMMERMAN, B. J. 2000. Attaining self-regulation: A social cognitive perspective. In Handbook of Self-Regulation, M. Boekaerts, P. R. Pintrich, and M. Zeidner, Eds. Academic Press, San Diego, 13–39
Section
EDM 2021 Journal Track
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:
- The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
- Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
- The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons 4.0 License (Attribution-Noncommercial-No Derivatives 4.0 International), or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
- Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
- Noncommercial—other users (including Publisher) may not use this Work for commercial purposes;
- No Derivative Works—other users (including Publisher) may not alter, transform, or build upon this Work,with the understanding that any of the above conditions can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
- The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
- Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
- The Author represents and warrants that:
- the Work is the Author’s original work;
- the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
- the Work is not pending review or under consideration by another publisher;
- the Work has not previously been published;
- the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
- the Work contains no libel, invasion of privacy, or other unlawful matter.
- The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.