Automatically Predicting Peer Satisfaction During Collaborative Learning with Linguistic, Acoustic, and Visual Features
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Jun 21, 2023
Yingbo Ma
Gloria Ashiya Katuka
Mehmet Celepkolu
Kristy Elizabeth Boyer
https://orcid.org/0009-0001-5271-7025
Abstract
Collaborative learning has numerous benefits such as enhancing learners’ critical thinking, developing
social skills, and improving learning gains. While engaging in this interactive process, learners’ satisfaction
toward their partners plays a crucial role in defining the success of the collaboration. However,
detecting learners’ satisfaction during an ongoing collaboration remains challenging, and there are no
automatic techniques to predict learners’ satisfaction. In this paper, we propose a multimodal approach
to automatically predict peer satisfaction for co-located collaboration with features extracted from 44
middle school learners’ collaborative dialogues. We investigated three types of features extracted from
learners’ dialogues: 1) linguistic features indicating semantics and sentiment; 2) acoustic-prosodic features
including energy and pitch; and 3) visual features including eye gaze, head pose, facial action units,
and body pose. We then trained several regression models with each of those features to predict the peer
satisfaction scores that learners received from their partners. The results revealed that head position and
body location were significant indicators of peer satisfaction: lower head and body distances between
partners were associated with more positive peer satisfaction. Next, we investigated the influence of multimodal
feature fusion methods on peer satisfaction prediction accuracy: early fusion versus late fusion.
We report the comparison results between models trained with (1) best-performing unimodal features, (2)
multimodal features combined by early fusion, and (3) multimodal features combined by late fusion. This
line of research reveals how multimodal features from collaborative dialogues are associated with peer
satisfaction, and represents a step toward the development of real-time intelligent systems that support
collaborative learning.
How to Cite
Ma, Y., Ashiya Katuka, G., Celepkolu, M., & Elizabeth Boyer, K. (2023). Automatically Predicting Peer Satisfaction During Collaborative Learning with Linguistic, Acoustic, and Visual Features. Journal of Educational Data Mining, 15(2), 86–122. https://doi.org/10.5281/zenodo.7304816
##plugins.themes.bootstrap3.article.details##
Keywords
collaborative learning, peer satisfaction, pair programming, multimodal modeling, multimodal data fusion
References
AHUJA, K., KIM, D., XHAKAJ, F., VARGA, V., XIE, A., ZHANG, S., TOWNSEND, J. E., HARRISON, C., OGAN, A., AND AGARWAL, Y. 2019. Edusense: Practical classroom sensing at scale. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, S. Santini, Ed. Vol. 3. ACM, New York, NY, 1–26.
ANDRADE, A. 2017. Understanding student learning trajectories using multimodal learning analytics within an embodied-interaction learning environment. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference. ACM, New York, NY, 70–79.
BALTRUSAITIS, T., ZADEH, A., LIM, Y. C., AND MORENCY, L.-P. 2018. Openface 2.0: Facial behavior analysis toolkit. In Proceedings of the 13th IEEE International Conference on Automatic Face & Gesture Recognition. IEEE, New York, NY, 59–66.
BLIKSTEIN, P. 2013. Multimodal learning analytics. In Proceedings of the 3rd International Conference on Learning Analytics and Knowledge, D. Suthers, K. Verbert, and X. O. Erik Duval, Eds. ACM, New York, NY, 102–106.
CAO, Z., SIMON, T., WEI, S.-E., AND SHEIKH, Y. 2017. Realtime multi-person 2d pose estimation using part affinity fields. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, New York, NY, 7291–7299.
CELEPKOLU, M., FUSSELL, D. A., GALDO, A. C., BOYER, K. E., WIEBE, E. N., MOTT, B. W., AND LESTER, J. C. 2020. Exploring middle school students’ reflections on the infusion of CS into science classrooms. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. ACM, New York, NY, 671–677.
CHAE, S. W. 2016. Perceived proximity and trust network on creative performance in virtual collaboration environment. Procedia Computer Science 91, 807–812.
CHAN, C.-L., JIANG, J. J., AND KLEIN, G. 2008. Team task skills as a facilitator for application and development skills. IEEE Transactions on Engineering Management 55, 3, 434–441.
CHAN, L. H. AND CHEN, C.-H. 2010. Conflict from teamwork in project-based collaborative learning. Performance Improvement 49, 2, 23–28.
CHEN, Y. 2018. Perceptions of EFL college students toward collaborative learning. English Language Teaching 11, 2, 1–4.
CHOWDHURY, S. A., STEPANOV, E. A., AND RICCARDI, G. 2016. Predicting user satisfaction from turn-taking in spoken conversations. In INTERSPEECH 2016. International Speech Communication Association, France, 2910–2914.
CHURCH, K. W. 2017. Word2vec. Natural Language Engineering 23, 1, 155–162.
CIMATTI, B. 2016. Definition, development, assessment of soft skills and their role for the quality of organizations and enterprises. International Journal for Quality Research 10, 1, 97–130.
CLARKE, A. D. AND TATLER, B. W. 2014. Deriving an appropriate baseline for describing fixation behaviour. Vision Research 102, 41–51.
CUKUROVA, M., ZHOU, Q., SPIKOL, D., AND LANDOLFI, L. 2020. Modelling collaborative problem-solving competence with transparent learning analytics: is video data enough? In Proceedings of the 10th International Conference on Learning Analytics & Knowledge. ACM, New York, NY, 270–275.
DAOUDI, I., TRANVOUEZ, E., CHEBIL, R., ESPINASSE, B., AND CHAARI, W. 2020. An edm-based multimodal method for assessing learners’ affective states in collaborative crisis management serious games. In Proceedings of the 13th International Conference on Educational Data Mining, A. N. Rafferty, J. Whitehill, C. Romero, and V. Cavalli-Sforza, Eds. International Educational Data Mining Society, 596–600.
DEVLIN, J., CHANG, M.-W., LEE, K., AND TOUTANOVA, K. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ACL, Minneapolis, MN, 4171–4186.
DEWIYANTI, S., BRAND-GRUWEL, S., JOCHEMS, W., AND BROERS, N. J. 2007. Students’ experiences with collaborative learning in asynchronous computer-supported collaborative learning environments. Computers in Human Behavior 23, 1, 496–514.
DI MITRI, D., SCHEFFEL, M., DRACHSLER, H., BÖRNER, D., TERNIER, S., AND SPECHT, M. 2017. Learning pulse: A machine learning approach for predicting performance in selfregulated learning using multimodal data. In Proceedings of the 7th International Learning Analytics & Knowledge Conference (LAK 2017), A. Wise, P. H. Winne, and G. Lynch, Eds. ACM, New York, NY, 188–197.
D’MELLO, S., STEWART, A. E., AMON, M. J., SUN, C., DURAN, N. D., AND SHUTE, V. 2019. Towards dynamic intelligent support for collaborative problem solving. In Proceedings of the 20th Artificial Intelligence in Education Conference, S. Isotani, E. Millán, A. Ogan, P. Hastings, B. McLaren, and R. Luckin, Eds. Springer Cham, 59–65.
ECHEVERRIA, V., MARTINEZ-MALDONADO, R., AND BUCKINGHAM SHUM, S. 2019. Towards collaboration translucence: Giving meaning to multimodal group data. In Proceedings of the 2019 CHI Conference on Human Factors In Computing Systems. ACM, New York, NY, 1–16.
EYBEN, F., SCHERER, K. R., SCHULLER, B. W., SUNDBERG, J., ANDRÉ, E., BUSSO, C., DEVILLERS, L. Y., EPPS, J., LAUKKA, P., NARAYANAN, S. S., ET AL. 2015. The geneva minimalistic acoustic parameter set (gemaps) for voice research and affective computing. IEEE Transactions on Affective Computing 7, 2, 190–202.
EYBEN, F., WÖLLMER, M., AND SCHULLER, B. 2010. Opensmile: the munich versatile and fast open-source audio feature extractor. In Proceedings of the 2010 International Conference on Multimedia, S.-F. C. Alberto del Bimbo, Ed. ACM, New York, NY, 1459–1462.
FORSYTH, C., ANDREWS-TODD, J., AND STEINBERG, J. 2020. Are you really a team player? profiling of collaborative problem solvers in an online environment. In Proceedings of the 13th International Conference on Educational Data Mining, A. N. Rafferty, J. Whitehill, C. Romero, and V. Cavalli-Sforza, Eds. International Educational Data Mining Society, 403– 408.
GADZICKI, K., KHAMSEHASHARI, R., AND ZETZSCHE, C. 2020. Early vs late fusion in multimodal convolutional neural networks. In Proceedings of the 23rd International Conference on Information Fusion. IEEE, New York, NY, 1–6.
GOGATE, M., ADEEL, A., AND HUSSAIN, A. 2017. Deep learning driven multimodal fusion for automated deception detection. In Proccedings of 2017 IEEE Symposium Series on Computational Intelligence. IEEE, New York, NY, 1–6.
GOUD, T. T., SMRITHIREKHA, V., AND SANGEETHA, G. 2017. Factors influencing group member satisfaction in the software industry. In Proceedings of the 2nd Conference on Data Engineering and Communication Technology, A. J. Kulkarni, S. C. Satapathy, T. Kang, and A. H. Kashan, Eds. Springer Singapore, 223–230.
GRAFSGAARD, J. F., WIGGINS, J. B., BOYER, K. E., WIEBE, E. N., AND LESTER, J. C. 2013. Automatically recognizing facial indicators of frustration: a learning-centric analysis. In 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction. IEEE, New York, NY, 159–165.
GRIFFITH, A. E., KATUKA, G. A.,WIGGINS, J. B., BOYER, K. E., FREEMAN, J., MAGERKO, B., AND MCKLIN, T. 2022. Investigating the relationship between dialogue states and partner satisfaction during co-creative learning tasks. International Journal of Artificial Intelligence in Education Aug, 1, 1–40.
HASLER-WATERS, L. AND NAPIER, W. 2002. Building and supporting student team collaboration in the virtual classroom. Quarterly Review of Distance Education 3, 3, 345–52.
HOUSSAMI, N., MACASKILL, P., MARINOVICH, M. L., DIXON, J. M., IRWIG, L., BRENNAN, M. E., AND SOLIN, L. J. 2010. Meta-analysis of the impact of surgical margins on local recurrence in women with early-stage invasive breast cancer treated with breast-conserving therapy. European Journal of Cancer 46, 18, 3219–3232.
HUANG, K., BRYANT, T., AND SCHNEIDER, B. 2017. Identifying collaborative learning states using unsupervised machine learning on eye-tracking, physiological and motion sensor data. In Proceedings of the 12th International Conference on Educational Data Mining, X. Hu, T. Barnes, A. Hershkovitz, and L. Paquette, Eds. International Educational Data Mining Society, 318–323.
KAPP, E. 2009. Improving student teamwork in a collaborative project-based course. College Teaching 57, 3, 139–143.
KATUKA, G. A., BEX, R. T., CELEPKOLU, M., BOYER, K. E., WIEBE, E., MOTT, B., AND LESTER, J. 2021. My partner was a good partner: Investigating the relationship between dialogue acts and satisfaction among middle school computer science learners. In Proceedings of the 14th International Conference on Computer-Supported Collaborative Learning, C. E. Hmelo-Silver, B. D. Wever, and J. Oshima, Eds. International Society of the Learning Sciences, 51–58.
KHALEGHI, B., KHAMIS, A., KARRAY, F. O., AND RAZAVI, S. N. 2013. Multisensor data fusion: A review of the state-of-the-art. Information Fusion 14, 1, 28–44.
KIM, J., KWON, Y., AND CHO, D. 2011. Investigating factors that influence social presence and learning outcomes in distance higher education. Computers & Education 57, 2, 1512–1520.
KU, H.-Y., TSENG, H. W., AND AKARASRIWORN, C. 2013. Collaboration factors, teamwork satisfaction, and student attitudes toward online collaborative learning. Computers in Human Behavior 29, 3, 922–929.
LAHAT, D., ADALI, T., AND JUTTEN, C. 2015. Multimodal data fusion: an overview of methods, challenges, and prospects. Proceedings of the IEEE 103, 9, 1449–1477.
LAN, Z.-Z., BAO, L., YU, S.-I., LIU, W., AND HAUPTMANN, A. G. 2014. Multimedia classification and event detection using double fusion. Multimedia Tools and Applications 71, 1, 333–347.
LAW, Q. P., SO, H. C., AND CHUNG, J. W. 2017. Effect of collaborative learning on enhancement of students’ self-efficacy, social skills and knowledge towards mobile apps development. American Journal of Educational Research 5, 1, 25–29.
LIU, R., DAVENPORT, J., AND STAMPER, J. 2016. Beyond log files: Using multi-modal data streams towards data-driven kc model improvement. In Proceedings of the 9th International Conference on Educational Data Mining, T. Barnes, M. Chi, and M. Feng, Eds. International Educational Data Mining Society, 436–441.
LOES, C. N. AND PASCARELLA, E. T. 2017. Collaborative learning and critical thinking: Testing the link. The Journal of Higher Education 88, 5, 726–753.
MA, Y., CELEPKOLU, M., AND BOYER, K. E. 2022. Detecting impasse during collaborative problem solving with multimodal learning analytics. In Proceedings of the 12th International Learning Analytics & Knowledge Conference. ACM, New York, NY, 45–55.
MA, Y., KATUKA, G. A., CELEPKOLU, M., AND BOYER, K. E. 2022. Investigating multimodal predictors of peer satisfaction for collaborative coding in middle school. In Proceedings of the 15th International Conference on Educational Data Mining, A. Mitrovic and N. Bosch, Eds. International Educational Data Mining Society, 133–144.
MADAIO, M., LASKO, R., OGAN, A., AND CASSELL, J. 2017. Using temporal association rule mining to predict dyadic rapport in peer tutoring. In Proceedings of the 10th International Conference on Educational Data Mining, X. Hu, T. Barnes, A. Hershkovitz, and L. Paquette, Eds. International Educational Data Mining Society, 318–323.
MAGERKO, B., FREEMAN, J., MCKLIN, T., REILLY, M., LIVINGSTON, E., MCCOID, S., AND CREWS-BROWN, A. 2016. Earsketch: A steam-based approach for underrepresented populations in high school computer science education. ACM Transactions on Computing Education 16, 4, 1–25.
MAGNISALIS, I., DEMETRIADIS, S., AND KARAKOSTAS, A. 2011. Adaptive and intelligent systems for collaborative learning support: A review of the field. IEEE Transactions on Learning Technologies 4, 1, 5–20.
MALMBERG, J., JÄRVELÄ, S., HOLAPPA, J., HAATAJA, E., HUANG, X., AND SIIPO, A. 2019. Going beyond what is visible: What multichannel data can reveal about interaction in the context of collaborative learning? Computers in Human Behavior 96, 235–245.
MANGAROSKA, K., SHARMA, K., GAŠEVÍC , D., AND GIANNAKOS, M. 2022. Exploring students’ cognitive and affective states during problem solving through multimodal data: Lessons learned from a programming activity. Journal of Computer Assisted Learning 38, 1, 40–59.
MEE, R. W. AND CHUA, T. C. 1991. Regression toward the mean and the paired sample t test. The American Statistician 45, 1, 39–42.
MOLINILLO, S., AGUILAR-ILLESCAS, R., ANAYA-SÁNCHEZ, R., AND VALLESPÍN-ARÁN, M. 2018. Exploring the impacts of interactions, social presence and emotional engagement on active collaborative learning in a social web-based environment. Computers & Education 123, 41–52.
MURRAY, G. AND OERTEL, C. 2018. Predicting group performance in task-based interaction. In Proceedings of the 20th ACM International Conference on Multimodal Interaction (ICMI 2018), S. K. D’Mello, P. P. Georgiou, and S. Scherer, Eds. ACM, New York, NY, 14–20.
NAKANO, Y. I., NIHONYANAGI, S., TAKASE, Y., HAYASHI, Y., AND OKADA, S. 2015. Predicting participation styles using co-occurrence patterns of nonverbal behaviors in collaborative learning. In Proceedings of the 17th International Conference on Multimodal Interaction (ICMI 2015), Z. Zhang and P. Cohen, Eds. ACM, New York, NY, 91–98.
OCHOA, X., CHILUIZA, K., MÉNDEZ, G., LUZARDO, G., GUAMÁN, B., AND CASTELLS, J. 2013. Expertise estimation based on simple multimodal features. In Proceedings of the 15th ACM on International Conference on Multimodal Interaction, J. Epps, F. Chen, S. Oviatt, and K. Mase, Eds. ACM, New York, NY, 583–590.
OLSEN, J. K., SHARMA, K., RUMMEL, N., AND ALEVEN, V. 2020. Temporal analysis of multimodal data to predict collaborative learning outcomes. British Journal of Educational Technology 51, 5, 1527–1547.
PRAHARAJ, S., SCHEFFEL, M., SCHMITZ, M., SPECHT, M., AND DRACHSLER, H. 2022. Towards collaborative convergence: quantifying collaboration quality with automated colocated collaboration analytics. In LAK22: 12th International Learning Analytics & Knowledge Conference, A. F. Wise, R. Martinez-Maldonado, and I. Hilliger, Eds. ACM, New York, NY, 358–369.
RADU, I., TU, E., AND SCHNEIDER, B. 2020. Relationships between body postures and collaborative learning states in an augmented reality study. In Proceedings of the 21st International Conference on Artificial Intelligence in Education (AIED 2020), I. I. Bittencourt, M. Cukurova, K. Muldner, R. Luckin, and E. Millán, Eds. Springer, New York, NY, 257–262.
RAJENDRAN, R., KUMAR, A., CARTER, K. E., LEVIN, D. T., AND BISWAS, G. 2018. Predicting learning by analyzing eye-gaze data of reading behavior. In Proceedings of the 11th International Conference on Educational Data Mining, K. E. Boyer and M. Yudelson, Eds. International Educational Data Mining Society, 455–461.
RAMACHANDRAM, D. AND TAYLOR, G. W. 2017. Deep multimodal learning: A survey on recent advances and trends. IEEE Signal Processing Magazine 34, 6, 96–108.
REILLY, J. M. AND SCHNEIDER, B. 2019. Predicting the quality of collaborative problem solving through linguistic analysis of discourse. In Proceedings of The 12th International Conference on Educational Data Mining, C. F. Lynch, A. Merceron, M. Desmarais, and R. Nkambou, Eds. International Educational Data Mining Society, 149–157.
SCHERER, S., WEIBEL, N., MORENCY, L.-P., AND OVIATT, S. 2012. Multimodal prediction of expertise and leadership in learning groups. In Proceedings of the 1st International Workshop on Multimodal Learning Analytics, L.-P. Morency, D. Bohus, and H. Aghajan, Eds. ACM, New York, NY, 1–8.
SCHNEIDER, B. 2019. Unpacking collaborative learning processes during hands-on activities using mobile eye-trackers. In Proceedings of the 13th International Conference on Computer- Supported Collaborative Learning, K. Lund, G. P. Niccolai, E. Lavoué, C. E. Hmelo-Silver, G. Gweon, and M. Baker, Eds. International Society of the Learning Sciences, 156–165.
SCHNEIDER, B., SHARMA, K., CUENDET, S., ZUFFEREY, G., DILLENBOURG, P., AND PEA, R. 2018. Leveraging mobile eye-trackers to capture joint visual attention in co-located collaborative learning groups. International Journal of Computer-Supported Collaborative Learning 13, 3, 241–261.
SCHULTZ, J. L., WILSON, J. R., AND HESS, K. C. 2010. Team-based classroom pedagogy reframed: The student perspective. American Journal of Business Education 3, 7, 17–24.
SHARMA, K., PAPAMITSIOU, Z., OLSEN, J. K., AND GIANNAKOS, M. 2020. Predicting learners’ effortful behaviour in adaptive assessment using multimodal data. In Proceedings of the 10th International Conference on Learning Analytics & Knowledge. ACM, New York, NY, 480–489.
SINCLAIR, A. J. AND SCHNEIDER, B. 2021. Linguistic and gestural coordination: Do learners converge in collaborative dialogue?. In Proceedings of The 14th International Conference on Educational Data Mining, I.-H. Hsiao, S. Sahebi, F. Bouchet, and J.-J. Vie, Eds. International Educational Data Mining Society, 431–438.
SNOEK, C. G., WORRING, M., AND SMEULDERS, A. W. 2005. Early versus late fusion in semantic video analysis. In Proceedings of the 13th International Conference on Multimedia. ACM, New York, NY, 399–402.
SO, H.-J. AND BRUSH, T. A. 2008. Student perceptions of collaborative learning, social presence and satisfaction in a blended learning environment: Relationships and critical factors. Computers & Education 51, 1, 318–336.
SPIKOL, D., RUFFALDI, E., LANDOLFI, L., AND CUKUROVA, M. 2017. Estimation of success in collaborative learning based on multimodal learning analytics features. In Proceedings of the 17th International Conference on Advanced Learning Technologies. IEEE, New York, NY, 269–273.
STEWART, A. E., KEIRN, Z., AND D’MELLO, S. K. 2021. Multimodal modeling of collaborative problem-solving facets in triads. User Modeling and User-Adapted Interaction 31, 4, 713–751.
STEWART, A. E., KEIRN, Z. A., AND D’MELLO, S. K. 2018. Multimodal modeling of coordination and coregulation patterns in speech rate during triadic collaborative problem solving. In Proceedings of the 20th ACM International Conference on Multimodal Interaction, S. K. D’Mello, P. Georgiou, and S. Scherer, Eds. ACM, New York, NY, 21–30.
TSAN, J., VANDENBERG, J., ZAKARIA, Z., BOULDEN, D. C., LYNCH, C., WIEBE, E., AND BOYER, K. E. 2021. Collaborative dialogue and types of conflict: An analysis of pair programming interactions between upper elementary students. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. ACM, New York, NY, 1184–1190.
TSENG, H., WANG, C., KU, H., AND SUN, L. 2009. Key factors in online collaboration and their relationship to teamwork satisfaction. Quarterly Review of Distance Education 10, 2, 195–206.
VIVIAN, R., FALKNER, K., FALKNER, N., AND TARMAZDI, H. 2016. A method to analyze computer science students’ teamwork in online collaborative learning environments. ACM Transactions on Computing Education 16, 2, 1–28.
VRZAKOVA, H., AMON, M. J., STEWART, A., DURAN, N. D., AND D’MELLO, S. K. 2020. Focused or stuck together: multimodal patterns reveal triads’ performance in collaborative problem solving. In Proceedings of the 10th International Conference on Learning Analytics & Knowledge. ACM, New York, NY, 295–304.
WALKER, E., RUMMEL, N., AND KOEDINGER, K. R. 2014. Adaptive intelligent support to improve peer tutoring in algebra. International Journal of Artificial Intelligence in Education 24, 1, 33–61.
WEI, W., LI, S., OKADA, S., AND KOMATANI, K. 2021. Multimodal user satisfaction recognition for non-task oriented dialogue systems. In Proceedings of the 23rd International Conference on Multimodal Interaction, Z. Hammal, C. Busso, C. Pelachaud, S. Oviatt, A. A. Salah, and G. Zhao, Eds. ACM, New York, NY, 586–594.
WORSLEY, M. 2018a. (dis)engagement matters: Identifying efficacious learning practices with multimodal learning analytics. In Proceedings of the 8th International Conference on Learning Analytics & Knowledge. ACM, New York, NY, 365–369.
WORSLEY, M. 2018b. Multimodal learning analytics’ past, present, and potential futures. In Companion Proceedings of the 8th International Conference on Learning Analytics & Knowledge, A. Pardo, K. Bartimote-Aufflick, S. Grace Lynch, Buckingham Shum, R. Ferguson, A. Merceron, and X. Ochoa, Eds. Society for Learning Analytics Research (SoLAR).
YE, G., LIU, D., JHUO, I.-H., AND CHANG, S.-F. 2012. Robust late fusion with rank minimization. In Proceedings of the 25th International Conference on Computer Vision and Pattern Recognition. IEEE, New York, NY, 3021–3028.
ZEITUN, R. M., ABDULQADER, K. S., AND ALSHARE, K. A. 2013. Team satisfaction and student group performance: A cross-cultural study. Journal of Education for Business 88, 5, 286–293.
ZHANG, X., MENG, Y., DE PABLOS, P. O., AND SUN, Y. 2019. Learning analytics in collaborative learning supported by slack: From the perspective of engagement. Computers in Human Behavior 92, 625–633.
ZHENG, L. AND HUANG, R. 2016. The effects of sentiments and co-regulation on group performance in computer supported collaborative learning. The Internet and Higher Education 28, 59–67.
ZHU, C. 2012. Student satisfaction, performance, and knowledge construction in online collaborative learning. Journal of Educational Technology & Society 15, 1, 127–136.
ZHU, J., LI, H., LIU, T., ZHOU, Y., ZHANG, J., AND ZONG, C. 2018. Msmo: Multimodal summarization with multimodal output. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), E. Riloff, D. Chiang, J. Hockenmaier, and J. Tsujii, Eds. ACL, Minneapolis, MN, 4154–4164
ANDRADE, A. 2017. Understanding student learning trajectories using multimodal learning analytics within an embodied-interaction learning environment. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference. ACM, New York, NY, 70–79.
BALTRUSAITIS, T., ZADEH, A., LIM, Y. C., AND MORENCY, L.-P. 2018. Openface 2.0: Facial behavior analysis toolkit. In Proceedings of the 13th IEEE International Conference on Automatic Face & Gesture Recognition. IEEE, New York, NY, 59–66.
BLIKSTEIN, P. 2013. Multimodal learning analytics. In Proceedings of the 3rd International Conference on Learning Analytics and Knowledge, D. Suthers, K. Verbert, and X. O. Erik Duval, Eds. ACM, New York, NY, 102–106.
CAO, Z., SIMON, T., WEI, S.-E., AND SHEIKH, Y. 2017. Realtime multi-person 2d pose estimation using part affinity fields. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, New York, NY, 7291–7299.
CELEPKOLU, M., FUSSELL, D. A., GALDO, A. C., BOYER, K. E., WIEBE, E. N., MOTT, B. W., AND LESTER, J. C. 2020. Exploring middle school students’ reflections on the infusion of CS into science classrooms. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. ACM, New York, NY, 671–677.
CHAE, S. W. 2016. Perceived proximity and trust network on creative performance in virtual collaboration environment. Procedia Computer Science 91, 807–812.
CHAN, C.-L., JIANG, J. J., AND KLEIN, G. 2008. Team task skills as a facilitator for application and development skills. IEEE Transactions on Engineering Management 55, 3, 434–441.
CHAN, L. H. AND CHEN, C.-H. 2010. Conflict from teamwork in project-based collaborative learning. Performance Improvement 49, 2, 23–28.
CHEN, Y. 2018. Perceptions of EFL college students toward collaborative learning. English Language Teaching 11, 2, 1–4.
CHOWDHURY, S. A., STEPANOV, E. A., AND RICCARDI, G. 2016. Predicting user satisfaction from turn-taking in spoken conversations. In INTERSPEECH 2016. International Speech Communication Association, France, 2910–2914.
CHURCH, K. W. 2017. Word2vec. Natural Language Engineering 23, 1, 155–162.
CIMATTI, B. 2016. Definition, development, assessment of soft skills and their role for the quality of organizations and enterprises. International Journal for Quality Research 10, 1, 97–130.
CLARKE, A. D. AND TATLER, B. W. 2014. Deriving an appropriate baseline for describing fixation behaviour. Vision Research 102, 41–51.
CUKUROVA, M., ZHOU, Q., SPIKOL, D., AND LANDOLFI, L. 2020. Modelling collaborative problem-solving competence with transparent learning analytics: is video data enough? In Proceedings of the 10th International Conference on Learning Analytics & Knowledge. ACM, New York, NY, 270–275.
DAOUDI, I., TRANVOUEZ, E., CHEBIL, R., ESPINASSE, B., AND CHAARI, W. 2020. An edm-based multimodal method for assessing learners’ affective states in collaborative crisis management serious games. In Proceedings of the 13th International Conference on Educational Data Mining, A. N. Rafferty, J. Whitehill, C. Romero, and V. Cavalli-Sforza, Eds. International Educational Data Mining Society, 596–600.
DEVLIN, J., CHANG, M.-W., LEE, K., AND TOUTANOVA, K. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ACL, Minneapolis, MN, 4171–4186.
DEWIYANTI, S., BRAND-GRUWEL, S., JOCHEMS, W., AND BROERS, N. J. 2007. Students’ experiences with collaborative learning in asynchronous computer-supported collaborative learning environments. Computers in Human Behavior 23, 1, 496–514.
DI MITRI, D., SCHEFFEL, M., DRACHSLER, H., BÖRNER, D., TERNIER, S., AND SPECHT, M. 2017. Learning pulse: A machine learning approach for predicting performance in selfregulated learning using multimodal data. In Proceedings of the 7th International Learning Analytics & Knowledge Conference (LAK 2017), A. Wise, P. H. Winne, and G. Lynch, Eds. ACM, New York, NY, 188–197.
D’MELLO, S., STEWART, A. E., AMON, M. J., SUN, C., DURAN, N. D., AND SHUTE, V. 2019. Towards dynamic intelligent support for collaborative problem solving. In Proceedings of the 20th Artificial Intelligence in Education Conference, S. Isotani, E. Millán, A. Ogan, P. Hastings, B. McLaren, and R. Luckin, Eds. Springer Cham, 59–65.
ECHEVERRIA, V., MARTINEZ-MALDONADO, R., AND BUCKINGHAM SHUM, S. 2019. Towards collaboration translucence: Giving meaning to multimodal group data. In Proceedings of the 2019 CHI Conference on Human Factors In Computing Systems. ACM, New York, NY, 1–16.
EYBEN, F., SCHERER, K. R., SCHULLER, B. W., SUNDBERG, J., ANDRÉ, E., BUSSO, C., DEVILLERS, L. Y., EPPS, J., LAUKKA, P., NARAYANAN, S. S., ET AL. 2015. The geneva minimalistic acoustic parameter set (gemaps) for voice research and affective computing. IEEE Transactions on Affective Computing 7, 2, 190–202.
EYBEN, F., WÖLLMER, M., AND SCHULLER, B. 2010. Opensmile: the munich versatile and fast open-source audio feature extractor. In Proceedings of the 2010 International Conference on Multimedia, S.-F. C. Alberto del Bimbo, Ed. ACM, New York, NY, 1459–1462.
FORSYTH, C., ANDREWS-TODD, J., AND STEINBERG, J. 2020. Are you really a team player? profiling of collaborative problem solvers in an online environment. In Proceedings of the 13th International Conference on Educational Data Mining, A. N. Rafferty, J. Whitehill, C. Romero, and V. Cavalli-Sforza, Eds. International Educational Data Mining Society, 403– 408.
GADZICKI, K., KHAMSEHASHARI, R., AND ZETZSCHE, C. 2020. Early vs late fusion in multimodal convolutional neural networks. In Proceedings of the 23rd International Conference on Information Fusion. IEEE, New York, NY, 1–6.
GOGATE, M., ADEEL, A., AND HUSSAIN, A. 2017. Deep learning driven multimodal fusion for automated deception detection. In Proccedings of 2017 IEEE Symposium Series on Computational Intelligence. IEEE, New York, NY, 1–6.
GOUD, T. T., SMRITHIREKHA, V., AND SANGEETHA, G. 2017. Factors influencing group member satisfaction in the software industry. In Proceedings of the 2nd Conference on Data Engineering and Communication Technology, A. J. Kulkarni, S. C. Satapathy, T. Kang, and A. H. Kashan, Eds. Springer Singapore, 223–230.
GRAFSGAARD, J. F., WIGGINS, J. B., BOYER, K. E., WIEBE, E. N., AND LESTER, J. C. 2013. Automatically recognizing facial indicators of frustration: a learning-centric analysis. In 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction. IEEE, New York, NY, 159–165.
GRIFFITH, A. E., KATUKA, G. A.,WIGGINS, J. B., BOYER, K. E., FREEMAN, J., MAGERKO, B., AND MCKLIN, T. 2022. Investigating the relationship between dialogue states and partner satisfaction during co-creative learning tasks. International Journal of Artificial Intelligence in Education Aug, 1, 1–40.
HASLER-WATERS, L. AND NAPIER, W. 2002. Building and supporting student team collaboration in the virtual classroom. Quarterly Review of Distance Education 3, 3, 345–52.
HOUSSAMI, N., MACASKILL, P., MARINOVICH, M. L., DIXON, J. M., IRWIG, L., BRENNAN, M. E., AND SOLIN, L. J. 2010. Meta-analysis of the impact of surgical margins on local recurrence in women with early-stage invasive breast cancer treated with breast-conserving therapy. European Journal of Cancer 46, 18, 3219–3232.
HUANG, K., BRYANT, T., AND SCHNEIDER, B. 2017. Identifying collaborative learning states using unsupervised machine learning on eye-tracking, physiological and motion sensor data. In Proceedings of the 12th International Conference on Educational Data Mining, X. Hu, T. Barnes, A. Hershkovitz, and L. Paquette, Eds. International Educational Data Mining Society, 318–323.
KAPP, E. 2009. Improving student teamwork in a collaborative project-based course. College Teaching 57, 3, 139–143.
KATUKA, G. A., BEX, R. T., CELEPKOLU, M., BOYER, K. E., WIEBE, E., MOTT, B., AND LESTER, J. 2021. My partner was a good partner: Investigating the relationship between dialogue acts and satisfaction among middle school computer science learners. In Proceedings of the 14th International Conference on Computer-Supported Collaborative Learning, C. E. Hmelo-Silver, B. D. Wever, and J. Oshima, Eds. International Society of the Learning Sciences, 51–58.
KHALEGHI, B., KHAMIS, A., KARRAY, F. O., AND RAZAVI, S. N. 2013. Multisensor data fusion: A review of the state-of-the-art. Information Fusion 14, 1, 28–44.
KIM, J., KWON, Y., AND CHO, D. 2011. Investigating factors that influence social presence and learning outcomes in distance higher education. Computers & Education 57, 2, 1512–1520.
KU, H.-Y., TSENG, H. W., AND AKARASRIWORN, C. 2013. Collaboration factors, teamwork satisfaction, and student attitudes toward online collaborative learning. Computers in Human Behavior 29, 3, 922–929.
LAHAT, D., ADALI, T., AND JUTTEN, C. 2015. Multimodal data fusion: an overview of methods, challenges, and prospects. Proceedings of the IEEE 103, 9, 1449–1477.
LAN, Z.-Z., BAO, L., YU, S.-I., LIU, W., AND HAUPTMANN, A. G. 2014. Multimedia classification and event detection using double fusion. Multimedia Tools and Applications 71, 1, 333–347.
LAW, Q. P., SO, H. C., AND CHUNG, J. W. 2017. Effect of collaborative learning on enhancement of students’ self-efficacy, social skills and knowledge towards mobile apps development. American Journal of Educational Research 5, 1, 25–29.
LIU, R., DAVENPORT, J., AND STAMPER, J. 2016. Beyond log files: Using multi-modal data streams towards data-driven kc model improvement. In Proceedings of the 9th International Conference on Educational Data Mining, T. Barnes, M. Chi, and M. Feng, Eds. International Educational Data Mining Society, 436–441.
LOES, C. N. AND PASCARELLA, E. T. 2017. Collaborative learning and critical thinking: Testing the link. The Journal of Higher Education 88, 5, 726–753.
MA, Y., CELEPKOLU, M., AND BOYER, K. E. 2022. Detecting impasse during collaborative problem solving with multimodal learning analytics. In Proceedings of the 12th International Learning Analytics & Knowledge Conference. ACM, New York, NY, 45–55.
MA, Y., KATUKA, G. A., CELEPKOLU, M., AND BOYER, K. E. 2022. Investigating multimodal predictors of peer satisfaction for collaborative coding in middle school. In Proceedings of the 15th International Conference on Educational Data Mining, A. Mitrovic and N. Bosch, Eds. International Educational Data Mining Society, 133–144.
MADAIO, M., LASKO, R., OGAN, A., AND CASSELL, J. 2017. Using temporal association rule mining to predict dyadic rapport in peer tutoring. In Proceedings of the 10th International Conference on Educational Data Mining, X. Hu, T. Barnes, A. Hershkovitz, and L. Paquette, Eds. International Educational Data Mining Society, 318–323.
MAGERKO, B., FREEMAN, J., MCKLIN, T., REILLY, M., LIVINGSTON, E., MCCOID, S., AND CREWS-BROWN, A. 2016. Earsketch: A steam-based approach for underrepresented populations in high school computer science education. ACM Transactions on Computing Education 16, 4, 1–25.
MAGNISALIS, I., DEMETRIADIS, S., AND KARAKOSTAS, A. 2011. Adaptive and intelligent systems for collaborative learning support: A review of the field. IEEE Transactions on Learning Technologies 4, 1, 5–20.
MALMBERG, J., JÄRVELÄ, S., HOLAPPA, J., HAATAJA, E., HUANG, X., AND SIIPO, A. 2019. Going beyond what is visible: What multichannel data can reveal about interaction in the context of collaborative learning? Computers in Human Behavior 96, 235–245.
MANGAROSKA, K., SHARMA, K., GAŠEVÍC , D., AND GIANNAKOS, M. 2022. Exploring students’ cognitive and affective states during problem solving through multimodal data: Lessons learned from a programming activity. Journal of Computer Assisted Learning 38, 1, 40–59.
MEE, R. W. AND CHUA, T. C. 1991. Regression toward the mean and the paired sample t test. The American Statistician 45, 1, 39–42.
MOLINILLO, S., AGUILAR-ILLESCAS, R., ANAYA-SÁNCHEZ, R., AND VALLESPÍN-ARÁN, M. 2018. Exploring the impacts of interactions, social presence and emotional engagement on active collaborative learning in a social web-based environment. Computers & Education 123, 41–52.
MURRAY, G. AND OERTEL, C. 2018. Predicting group performance in task-based interaction. In Proceedings of the 20th ACM International Conference on Multimodal Interaction (ICMI 2018), S. K. D’Mello, P. P. Georgiou, and S. Scherer, Eds. ACM, New York, NY, 14–20.
NAKANO, Y. I., NIHONYANAGI, S., TAKASE, Y., HAYASHI, Y., AND OKADA, S. 2015. Predicting participation styles using co-occurrence patterns of nonverbal behaviors in collaborative learning. In Proceedings of the 17th International Conference on Multimodal Interaction (ICMI 2015), Z. Zhang and P. Cohen, Eds. ACM, New York, NY, 91–98.
OCHOA, X., CHILUIZA, K., MÉNDEZ, G., LUZARDO, G., GUAMÁN, B., AND CASTELLS, J. 2013. Expertise estimation based on simple multimodal features. In Proceedings of the 15th ACM on International Conference on Multimodal Interaction, J. Epps, F. Chen, S. Oviatt, and K. Mase, Eds. ACM, New York, NY, 583–590.
OLSEN, J. K., SHARMA, K., RUMMEL, N., AND ALEVEN, V. 2020. Temporal analysis of multimodal data to predict collaborative learning outcomes. British Journal of Educational Technology 51, 5, 1527–1547.
PRAHARAJ, S., SCHEFFEL, M., SCHMITZ, M., SPECHT, M., AND DRACHSLER, H. 2022. Towards collaborative convergence: quantifying collaboration quality with automated colocated collaboration analytics. In LAK22: 12th International Learning Analytics & Knowledge Conference, A. F. Wise, R. Martinez-Maldonado, and I. Hilliger, Eds. ACM, New York, NY, 358–369.
RADU, I., TU, E., AND SCHNEIDER, B. 2020. Relationships between body postures and collaborative learning states in an augmented reality study. In Proceedings of the 21st International Conference on Artificial Intelligence in Education (AIED 2020), I. I. Bittencourt, M. Cukurova, K. Muldner, R. Luckin, and E. Millán, Eds. Springer, New York, NY, 257–262.
RAJENDRAN, R., KUMAR, A., CARTER, K. E., LEVIN, D. T., AND BISWAS, G. 2018. Predicting learning by analyzing eye-gaze data of reading behavior. In Proceedings of the 11th International Conference on Educational Data Mining, K. E. Boyer and M. Yudelson, Eds. International Educational Data Mining Society, 455–461.
RAMACHANDRAM, D. AND TAYLOR, G. W. 2017. Deep multimodal learning: A survey on recent advances and trends. IEEE Signal Processing Magazine 34, 6, 96–108.
REILLY, J. M. AND SCHNEIDER, B. 2019. Predicting the quality of collaborative problem solving through linguistic analysis of discourse. In Proceedings of The 12th International Conference on Educational Data Mining, C. F. Lynch, A. Merceron, M. Desmarais, and R. Nkambou, Eds. International Educational Data Mining Society, 149–157.
SCHERER, S., WEIBEL, N., MORENCY, L.-P., AND OVIATT, S. 2012. Multimodal prediction of expertise and leadership in learning groups. In Proceedings of the 1st International Workshop on Multimodal Learning Analytics, L.-P. Morency, D. Bohus, and H. Aghajan, Eds. ACM, New York, NY, 1–8.
SCHNEIDER, B. 2019. Unpacking collaborative learning processes during hands-on activities using mobile eye-trackers. In Proceedings of the 13th International Conference on Computer- Supported Collaborative Learning, K. Lund, G. P. Niccolai, E. Lavoué, C. E. Hmelo-Silver, G. Gweon, and M. Baker, Eds. International Society of the Learning Sciences, 156–165.
SCHNEIDER, B., SHARMA, K., CUENDET, S., ZUFFEREY, G., DILLENBOURG, P., AND PEA, R. 2018. Leveraging mobile eye-trackers to capture joint visual attention in co-located collaborative learning groups. International Journal of Computer-Supported Collaborative Learning 13, 3, 241–261.
SCHULTZ, J. L., WILSON, J. R., AND HESS, K. C. 2010. Team-based classroom pedagogy reframed: The student perspective. American Journal of Business Education 3, 7, 17–24.
SHARMA, K., PAPAMITSIOU, Z., OLSEN, J. K., AND GIANNAKOS, M. 2020. Predicting learners’ effortful behaviour in adaptive assessment using multimodal data. In Proceedings of the 10th International Conference on Learning Analytics & Knowledge. ACM, New York, NY, 480–489.
SINCLAIR, A. J. AND SCHNEIDER, B. 2021. Linguistic and gestural coordination: Do learners converge in collaborative dialogue?. In Proceedings of The 14th International Conference on Educational Data Mining, I.-H. Hsiao, S. Sahebi, F. Bouchet, and J.-J. Vie, Eds. International Educational Data Mining Society, 431–438.
SNOEK, C. G., WORRING, M., AND SMEULDERS, A. W. 2005. Early versus late fusion in semantic video analysis. In Proceedings of the 13th International Conference on Multimedia. ACM, New York, NY, 399–402.
SO, H.-J. AND BRUSH, T. A. 2008. Student perceptions of collaborative learning, social presence and satisfaction in a blended learning environment: Relationships and critical factors. Computers & Education 51, 1, 318–336.
SPIKOL, D., RUFFALDI, E., LANDOLFI, L., AND CUKUROVA, M. 2017. Estimation of success in collaborative learning based on multimodal learning analytics features. In Proceedings of the 17th International Conference on Advanced Learning Technologies. IEEE, New York, NY, 269–273.
STEWART, A. E., KEIRN, Z., AND D’MELLO, S. K. 2021. Multimodal modeling of collaborative problem-solving facets in triads. User Modeling and User-Adapted Interaction 31, 4, 713–751.
STEWART, A. E., KEIRN, Z. A., AND D’MELLO, S. K. 2018. Multimodal modeling of coordination and coregulation patterns in speech rate during triadic collaborative problem solving. In Proceedings of the 20th ACM International Conference on Multimodal Interaction, S. K. D’Mello, P. Georgiou, and S. Scherer, Eds. ACM, New York, NY, 21–30.
TSAN, J., VANDENBERG, J., ZAKARIA, Z., BOULDEN, D. C., LYNCH, C., WIEBE, E., AND BOYER, K. E. 2021. Collaborative dialogue and types of conflict: An analysis of pair programming interactions between upper elementary students. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. ACM, New York, NY, 1184–1190.
TSENG, H., WANG, C., KU, H., AND SUN, L. 2009. Key factors in online collaboration and their relationship to teamwork satisfaction. Quarterly Review of Distance Education 10, 2, 195–206.
VIVIAN, R., FALKNER, K., FALKNER, N., AND TARMAZDI, H. 2016. A method to analyze computer science students’ teamwork in online collaborative learning environments. ACM Transactions on Computing Education 16, 2, 1–28.
VRZAKOVA, H., AMON, M. J., STEWART, A., DURAN, N. D., AND D’MELLO, S. K. 2020. Focused or stuck together: multimodal patterns reveal triads’ performance in collaborative problem solving. In Proceedings of the 10th International Conference on Learning Analytics & Knowledge. ACM, New York, NY, 295–304.
WALKER, E., RUMMEL, N., AND KOEDINGER, K. R. 2014. Adaptive intelligent support to improve peer tutoring in algebra. International Journal of Artificial Intelligence in Education 24, 1, 33–61.
WEI, W., LI, S., OKADA, S., AND KOMATANI, K. 2021. Multimodal user satisfaction recognition for non-task oriented dialogue systems. In Proceedings of the 23rd International Conference on Multimodal Interaction, Z. Hammal, C. Busso, C. Pelachaud, S. Oviatt, A. A. Salah, and G. Zhao, Eds. ACM, New York, NY, 586–594.
WORSLEY, M. 2018a. (dis)engagement matters: Identifying efficacious learning practices with multimodal learning analytics. In Proceedings of the 8th International Conference on Learning Analytics & Knowledge. ACM, New York, NY, 365–369.
WORSLEY, M. 2018b. Multimodal learning analytics’ past, present, and potential futures. In Companion Proceedings of the 8th International Conference on Learning Analytics & Knowledge, A. Pardo, K. Bartimote-Aufflick, S. Grace Lynch, Buckingham Shum, R. Ferguson, A. Merceron, and X. Ochoa, Eds. Society for Learning Analytics Research (SoLAR).
YE, G., LIU, D., JHUO, I.-H., AND CHANG, S.-F. 2012. Robust late fusion with rank minimization. In Proceedings of the 25th International Conference on Computer Vision and Pattern Recognition. IEEE, New York, NY, 3021–3028.
ZEITUN, R. M., ABDULQADER, K. S., AND ALSHARE, K. A. 2013. Team satisfaction and student group performance: A cross-cultural study. Journal of Education for Business 88, 5, 286–293.
ZHANG, X., MENG, Y., DE PABLOS, P. O., AND SUN, Y. 2019. Learning analytics in collaborative learning supported by slack: From the perspective of engagement. Computers in Human Behavior 92, 625–633.
ZHENG, L. AND HUANG, R. 2016. The effects of sentiments and co-regulation on group performance in computer supported collaborative learning. The Internet and Higher Education 28, 59–67.
ZHU, C. 2012. Student satisfaction, performance, and knowledge construction in online collaborative learning. Journal of Educational Technology & Society 15, 1, 127–136.
ZHU, J., LI, H., LIU, T., ZHOU, Y., ZHANG, J., AND ZONG, C. 2018. Msmo: Multimodal summarization with multimodal output. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), E. Riloff, D. Chiang, J. Hockenmaier, and J. Tsujii, Eds. ACL, Minneapolis, MN, 4154–4164
Issue
Section
Extended Articles from the EDM 2022 Conference
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:
- The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
- Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
- The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons 4.0 License (Attribution-Noncommercial-No Derivatives 4.0 International), or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
- Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
- Noncommercial—other users (including Publisher) may not use this Work for commercial purposes;
- No Derivative Works—other users (including Publisher) may not alter, transform, or build upon this Work,with the understanding that any of the above conditions can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
- The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
- Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
- The Author represents and warrants that:
- the Work is the Author’s original work;
- the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
- the Work is not pending review or under consideration by another publisher;
- the Work has not previously been published;
- the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
- the Work contains no libel, invasion of privacy, or other unlawful matter.
- The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.