Developing a Feedback Taxonomy for Math: A Synergy of Perspectives through Data Mining Methods
Main
Sidebar
Published
September 7, 2025
Seiyon Lee
Sami Baral
Hongming (Chip) Li
Li Cheng
Shan Zhang
Carly Siegel Thorp
Jennifer St. John
Tamisha Thompson
Neil Heffernan
Anthony F. Botelho
https://orcid.org/0000-0002-6024-677X
Abstract
Teachers often use open-ended questions to promote students' deeper understanding of the content. These questions are particularly useful in K–12 mathematics education, as they provide richer insights into students' problem-solving processes compared to closed-ended questions. However, they are also challenging to implement in educational technologies as significant time and effort are required to qualitatively evaluate the quality of students' responses and provide timely feedback. In recent years, there has been growing interest in developing algorithms to automatically grade students' open responses and generate feedback. Yet, few studies have focused on augmenting teachers' perceptions and judgments when assessing students' responses and crafting appropriate feedback. Even fewer have aimed to build empirically grounded frameworks and offer a shared language across different stakeholders. In this paper, we propose a taxonomy of feedback using data mining methods to analyze teacher-authored feedback from an online mathematics learning platform. By incorporating qualitative codes from both teachers and researchers, we take a methodological approach that accounts for the varying interpretations across coders. Through a synergy of diverse perspectives and data mining methods, our data-driven taxonomy reflects the complexity of feedback content as it appears in authentic settings. We discuss how this taxonomy can support more generalizable methods for providing pedagogically meaningful feedback at scale.
How to Cite
Developing a Feedback Taxonomy for Math: A Synergy of Perspectives through Data Mining Methods. (2025). Journal of Educational Data Mining, 17(2), 1-23. https://doi.org/10.5281/zenodo.16684563
Details
Keywords
Teacher feedback, Open-ended student response, K-12 mathematics education, Factor analysis, Cluster analysis, Feedback taxonomy, Correlation analysis
References
Abdu Saeed Mohammed, M. and Abdullah Alharbi, M. 2022. Cultivating learners’ technology-mediated dialogue of feedback in writing: Processes, potentials and limitations. Assessment & Evaluation in Higher Education 47, 6, 942–958.
Aleven, V., McLaughlin, E. A., Glenn, R. A., and Koedinger, K. R. 2016. Instruction based on adaptive learning technologies. Handbook of Research on Learning and Instruction 2, 522–560.
Anderson, R. C. and Biddle, W. B. 1975. On asking people questions about what they are reading. In Psychology of Learning and Motivation, G. H. Bower, Ed. Vol. 9. Academic Press, New York, NY, 89–132.
Bahar, A. and Maker, C. J. 2015. Cognitive backgrounds of problem solving: A comparison of open-ended vs. closed mathematics problems. Eurasia Journal of Mathematics, Science and Technology Education 11, 6, 1531–1546.
Baker, R. S., Corbett, A. T., and Koedinger, K. R. 2004. Detecting student misuse of intelligent tutoring systems. In Intelligent Tutoring Systems: 7th International Conference, ITS 2004, Maceió, Alagoas, Brazil, August 30-September 3, 2004. Proceedings 7. Springer, 531–540.
Baker, R. S. d., Corbett, A. T., Gowda, S. M., Wagner, A. Z., MacLaren, B. A., Kauffman, L. R., Mitchell, A. P., and Giguere, S. 2010. Contextual slip and prediction of student performance after use of an intelligent tutor. In User Modeling, Adaptation, and Personalization: 18th International Conference, UMAP 2010, Big Island, HI, USA, June 20-24, 2010. Proceedings 18. Springer, 52–63.
Barabasheva, I. 2021. Feedback as a means of motivation in foreign language teaching. In LATIP 2021: International Conference on Language and Technology in the Interdisciplinary Paradigm. European Proceedings of Social and Behavioural Sciences, vol. 121. European Publisher, Future Academy, London, UK, 221–227.
Baral, S., Botelho, A. F., Erickson, J. A., Benachamardi, P., and Heffernan, N. T. 2021. Improving automated scoring of student open responses in mathematics. International Educational Data Mining Society, 130–138.
Baral, S., Santhanam, A., Botelho, A., Gurung, A., and Heffernan, N. 2023. Automated scoring of image-based responses to open-ended mathematics question. In Proceedings of the 16th International Conference on Educational Data Mining (EDM). International Educational Data Mining Society, Bengaluru, India, 362––369.
Baral, S., Worden, E., Lim, W.-C., Luo, Z., Santorelli, C., and Gurung, A. 2024. Automated assessment in math education: A comparative analysis of LLMs for open-ended responses. In Proceedings of the 17th International Conference on Educational Data Mining, B. Paaßen and C. D. Epp, Eds. International Educational Data Mining Society, Atlanta, Georgia, USA, 732–737.
Basu, S., Jacobs, C., and Vanderwende, L. 2013. Powergrading: A clustering approach to amplify human effort for short answer grading. Transactions of the Association for Computational Linguistics 1, 391–402.
Belur, J., Tompson, L., Thornton, A., and Simon, M. 2021. Interrater reliability in systematic review methodology: Exploring variation in coder decision-making. Sociological Methods & Research 50, 2, 837–865.
Bhutoria, A. 2022. Personalized education and artificial intelligence in the United States, China, and India: A systematic review using a human-in-the-loop model. Computers and Education: Artificial Intelligence 3, 100068.
Bisra, K., Liu, Q., Nesbit, J. C., Salimi, F., and Winne, P. H. 2018. Inducing self-explanation: A meta-analysis. Educational Psychology Review 30, 703–725.
Boaler, J. 1998. Open and closed mathematics: Student experiences and understandings. Journal for Research in Mathematics Education 29, 1, 41–62.
Botelho, A. F., Baral, S., Erickson, J. A., Benachamardi, P., and Heffernan, N. T. 2023. Leveraging natural language processing to support automated assessment and feedback for student open responses in mathematics. Journal of Computer Assisted Learning 39, 3, 823–840.
Boyer, K. E., Phillips, R., Wallis, M., Vouk, M., and Lester, J. 2008. Balancing cognitive and motivational scaffolding in tutorial dialogue. In Intelligent Tutoring Systems: 9th International Conference, ITS 2008, Montreal, Canada, June 23-27, 2008 Proceedings 9. Springer, 239–249.
Brooks, M., Basu, S., Jacobs, C., and Vanderwende, L. 2014. Divide and correct: Using clusters to grade short answers at scale. In Proceedings of the First ACM Conference on Learning@ Scale. 89–98.
Cheng, L., Hampton, J., and Kumar, S. 2022. Engaging students via synchronous peer feedback in a technology-enhanced learning environment. Journal of Research on Technology in Education 56, sup1, 347–371.
Chi, M. T., De Leeuw, N., Chiu, M.-H., and LaVancher, C. 1994. Eliciting self-explanations improves understanding. Cognitive Science 18, 3, 439–477.
Chi, M. T. H. 2000. Self-explaining expository texts: The dual processes of generating inferences and repairing mental models. In Advances in Instructional Psychology, Volume 5, R. Glaser, Ed. Routledge, New York, NY, 161–238.
Coburn, C. E. and Turner, E. O. 2012. The practice of data use: An introduction. American Journal of Education 118, 2, 99–111.
Corbett, A. T. and Anderson, J. R. 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-adapted Interaction 4, 253–278.
Corral, D. and Carpenter, S. K. 2020. Facilitating transfer through incorrect examples and explanatory feedback. Quarterly Journal of Experimental Psychology 73, 9, 1340–1359.
Dawson, P., Henderson, M., Mahoney, P., Phillips, M., Ryan, T., Boud, D., and Molloy, E. 2019. What makes for effective feedback: Staff and student perspectives. Assessment & Evaluation in Higher Education 44, 1, 25–36.
Erickson, J. A., Botelho, A. F., McAteer, S., Varatharaj, A., and Heffernan, N. T. 2020. The automated grading of student open responses in mathematics. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge. 615–624.
Feng, M., Huang, C., and Collins, K. 2023. Promising long term effects of assistments online math homework support. In International Conference on Artificial Intelligence in Education. Springer, 212–217.
Fong, C. J. and Schallert, D. L. 2023. “Feedback to the future”: Advancing motivational and emotional perspectives in feedback research. Educational Psychologist 58, 3, 146–161.
Gaddipati, S. K., Nair, D., and Plöger, P. G. 2020. Comparative evaluation of pretrained transfer learning models on automatic short answer grading. arXiv preprint arXiv:2009.01303.
Gan, Z., An, Z., and Liu, F. 2021. Teacher feedback practices, student feedback motivation, and feedback behavior: How are they associated with learning outcomes? Frontiers in Psychology 12, 697045.
Grawemeyer, B., Mavrikis, M., Holmes, W., Gutiérrez-Santos, S., Wiedmann, M., and Rummel, N. 2017. Affective learning: Improving engagement and enhancing learning with affect-aware feedback. User Modeling and User-Adapted Interaction 27, 119–158.
Gurung, A., Baral, S., Lee, M. P., Sales, A. C., Haim, A., Vanacore, K. P., McReynolds, A. A., Kreisberg, H., Heffernan, C., and Heffernan, N. T. 2023. How common are common wrong answers? crowdsourcing remediation at scale. In Proceedings of the Tenth ACM Conference on Learning@ Scale. 70–80.
Gusukuma, L., Bart, A. C., Kafura, D., and Ernst, J. 2018. Misconception-driven feedback: Results from an experimental study. In Proceedings of the 2018 ACM Conference on International Computing Education Research. 160–168.
Hargreaves, E. 2014. The practice of promoting primary pupils’ autonomy: Examples of teacher feedback. Educational Research 56, 3, 295–309.
Hattie, J. and Timperley, H. 2007. The power of feedback. Review of Educational Research 77, 1, 81–112.
Hauke, J. and Kossowski, T. 2011. Comparison of values of Pearson’s and Spearman’s correlation coefficients on the same sets of data. Quaestiones Geographicae 30, 2, 87–93.
Heffernan, N. T. and Heffernan, C. L. 2014. The ASSISTments ecosystem: Building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. International Journal of Artificial Intelligence in Education 24, 470–497.
Hou, W.-J. and Tsao, J.-H. 2011. Automatic assessment of students’ free-text answers with different levels. International Journal on Artificial Intelligence Tools 20, 02, 327–347.
Klein, R., Kyrilov, A., and Tokman, M. 2011. Automated assessment of short free-text responses in computer science using latent semantic analysis. In Proceedings of the 16th Annual Joint Conference on Innovation and Technology in Computer Science Education. 158–162.
Kluger, A. N. and DeNisi, A. 1996. The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin 119, 2, 254.
Konold, K. E., Miller, S. P., and Konold, K. B. 2004. Using teacher feedback to enhance student learning. Teaching Exceptional Children 36, 6, 64–69.
Kulik, J. A. and Kulik, C.-L. C. 1988. Timing of feedback and verbal learning. Review of Educational Research 58, 1, 79–97.
Kwon, O. N., Park, J. H., and Park, J. S. 2006. Cultivating divergent thinking in mathematics through an open-ended approach. Asia Pacific Education Review 7, 51–61.
Labuhn, A. S., Zimmerman, B. J., and Hasselhorn, M. 2010. Enhancing students’ self-regulation and mathematics performance: The influence of feedback and self-evaluative standards. Metacognition and learning 5, 173–194.
Lee, H.-S., Pallant, A., Pryputniewicz, S., Lord, T., Mulholland, M., and Liu, O. L. 2019. Automated text scoring and real-time adjustable feedback: Supporting revision of scientific arguments involving uncertainty. Science Education 103, 3, 590–622.
Madnani, N., Burstein, J., Sabatini, J., and O’Reilly, T. 2013. Automated scoring of summary-writing tasks designed to measure reading comprehension. Grantee Submission.
McNamara, D. S., Crossley, S. A., Roscoe, R. D., Allen, L. K., and Dai, J. 2015. A hierarchical classification approach to automated essay scoring. Assessing Writing 23, 35–59.
McNichols, H., Lee, J., Fancsali, S., Ritter, S., and Lan, A. 2024. Can large language models replicate ITS feedback on open-ended math questions? In Proceedings of the 17th International Conference on Educational Data Mining, B. Paaßen and C. D. Epp, Eds. International Educational Data Mining Society, Atlanta, Georgia, USA, 769–775.
Meier, S. L., Rich, B. S., and Cady, J. 2006. Teachers’ use of rubrics to score non-traditional tasks: Factors related to discrepancies in scoring. Assessment in Education 13, 01, 69–95.
Monarch, R. M. 2021. Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI. Simon and Schuster.
Moreno, R. 2004. Decreasing cognitive load for novice students: Effects of explanatory versus corrective feedback in discovery-based multimedia. Instructional Science 32, 1, 99–113.
Mory, E. H. 2013. Feedback research revisited. In Handbook of Research on Educational Communications and Technology, J. M. Spector, M. D. Merrill, J. Elen, and M. J. Bishop, Eds. Routledge, New York, NY, 738–776.
Munroe, L. 2015. The open-ended approach framework. European Journal of Educational Research 4, 3, 97–104.
Narciss, S. 2013. Designing and evaluating tutoring feedback strategies for digital learning. Digital Education Review 23, 7–26.
Narciss, S., Sosnovsky, S., Schnaubert, L., Andrès, E., Eichelmann, A., Goguadze, G., and Melis, E. 2014. Exploring feedback and student characteristics relevant for personalizing feedback strategies. Computers & Education 71, 56–76.
Navarro-Gonzalez, D. and Lorenzo-Seva, U. 2021. EFA.MRFA: Dimensionality Assessment Using Minimum Rank Factor Analysis. R package.
Qi, H., Wang, Y., Dai, J., Li, J., and Di, X. 2019. Attention-based hybrid model for automatic short answer scoring. In Simulation Tools and Techniques: 11th International Conference, SIMUtools 2019, Chengdu, China, July 8–10, 2019, Proceedings 11. Springer, 385–394.
Rau, M. A., Aleven, V., and Rummel, N. 2015. Successful learning with multiple graphical representations and self-explanation prompts. Journal of Educational Psychology 107, 1, 30.
Ryan, T., Henderson, M., Ryan, K., and Kennedy, G. 2021. Designing learner-centred text-based feedback: A rapid review and qualitative synthesis. Assessment & Evaluation in Higher Education 46, 6, 894–912.
Sadler, D. R. 1989. Formative assessment and the design of instructional systems. Instructional Science 18, 2, 119–144.
Saldaña, J. 2021. The coding manual for qualitative researchers. SAGE publications Ltd.
Shute, V. J. 2008. Focus on formative feedback. Review of Educational Research 78, 1, 153–189.
Sweller, J. 2011. Cognitive load theory. In Psychology of Learning and Motivation, B. H. Ross, Ed. Vol. 55. Elsevier, San Diego, CA, 37–76.
Taghipour, K. and Ng, H. T. 2016. A neural approach to automated essay scoring. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 1882–1891.
Thompson, D. R. and Senk, S. L. 1998. Implementing the assessment standards for school mathematics: Using rubrics in high school mathematics courses. The Mathematics Teacher 91, 9, 786–793.
Van der Kleij, F. M., Feskens, R. C., and Eggen, T. J. 2015. Effects of feedback in a computer-based learning environment on students’ learning outcomes: A meta-analysis. Review of Educational Research 85, 4, 475–511.
Vanacore, K., Gurung, A., Sales, A., and Heffernan, N. T. 2024. The effect of assistance on gamers: Assessing the impact of on-demand hints & feedback availability on learning for students who game the system. In Proceedings of the 14th Learning Analytics and Knowledge Conference. 462–472.
Wisniewski, B., Zierer, K., and Hattie, J. 2020. The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology 10, 487662.
Wylie, R. and Chi, M. T. H. 2014. The self-explanation principle in multimedia learning. In The Cambridge Handbook of Multimedia Learning, R. E. Mayer, Ed. Cambridge University Press, Cambridge, UK, 413–432.
Yang, M. and Carless, D. 2013. The feedback triangle and the enhancement of dialogic feedback processes. Teaching in Higher Education 18, 3, 285–297.
Zhao, S., Zhang, Y., Xiong, X., Botelho, A., and Heffernan, N. 2017. A memory-augmented neural model for automated grading. In Proceedings of the Fourth (2017) ACM Conference on Learning@ Scale. 189–192.
Aleven, V., McLaughlin, E. A., Glenn, R. A., and Koedinger, K. R. 2016. Instruction based on adaptive learning technologies. Handbook of Research on Learning and Instruction 2, 522–560.
Anderson, R. C. and Biddle, W. B. 1975. On asking people questions about what they are reading. In Psychology of Learning and Motivation, G. H. Bower, Ed. Vol. 9. Academic Press, New York, NY, 89–132.
Bahar, A. and Maker, C. J. 2015. Cognitive backgrounds of problem solving: A comparison of open-ended vs. closed mathematics problems. Eurasia Journal of Mathematics, Science and Technology Education 11, 6, 1531–1546.
Baker, R. S., Corbett, A. T., and Koedinger, K. R. 2004. Detecting student misuse of intelligent tutoring systems. In Intelligent Tutoring Systems: 7th International Conference, ITS 2004, Maceió, Alagoas, Brazil, August 30-September 3, 2004. Proceedings 7. Springer, 531–540.
Baker, R. S. d., Corbett, A. T., Gowda, S. M., Wagner, A. Z., MacLaren, B. A., Kauffman, L. R., Mitchell, A. P., and Giguere, S. 2010. Contextual slip and prediction of student performance after use of an intelligent tutor. In User Modeling, Adaptation, and Personalization: 18th International Conference, UMAP 2010, Big Island, HI, USA, June 20-24, 2010. Proceedings 18. Springer, 52–63.
Barabasheva, I. 2021. Feedback as a means of motivation in foreign language teaching. In LATIP 2021: International Conference on Language and Technology in the Interdisciplinary Paradigm. European Proceedings of Social and Behavioural Sciences, vol. 121. European Publisher, Future Academy, London, UK, 221–227.
Baral, S., Botelho, A. F., Erickson, J. A., Benachamardi, P., and Heffernan, N. T. 2021. Improving automated scoring of student open responses in mathematics. International Educational Data Mining Society, 130–138.
Baral, S., Santhanam, A., Botelho, A., Gurung, A., and Heffernan, N. 2023. Automated scoring of image-based responses to open-ended mathematics question. In Proceedings of the 16th International Conference on Educational Data Mining (EDM). International Educational Data Mining Society, Bengaluru, India, 362––369.
Baral, S., Worden, E., Lim, W.-C., Luo, Z., Santorelli, C., and Gurung, A. 2024. Automated assessment in math education: A comparative analysis of LLMs for open-ended responses. In Proceedings of the 17th International Conference on Educational Data Mining, B. Paaßen and C. D. Epp, Eds. International Educational Data Mining Society, Atlanta, Georgia, USA, 732–737.
Basu, S., Jacobs, C., and Vanderwende, L. 2013. Powergrading: A clustering approach to amplify human effort for short answer grading. Transactions of the Association for Computational Linguistics 1, 391–402.
Belur, J., Tompson, L., Thornton, A., and Simon, M. 2021. Interrater reliability in systematic review methodology: Exploring variation in coder decision-making. Sociological Methods & Research 50, 2, 837–865.
Bhutoria, A. 2022. Personalized education and artificial intelligence in the United States, China, and India: A systematic review using a human-in-the-loop model. Computers and Education: Artificial Intelligence 3, 100068.
Bisra, K., Liu, Q., Nesbit, J. C., Salimi, F., and Winne, P. H. 2018. Inducing self-explanation: A meta-analysis. Educational Psychology Review 30, 703–725.
Boaler, J. 1998. Open and closed mathematics: Student experiences and understandings. Journal for Research in Mathematics Education 29, 1, 41–62.
Botelho, A. F., Baral, S., Erickson, J. A., Benachamardi, P., and Heffernan, N. T. 2023. Leveraging natural language processing to support automated assessment and feedback for student open responses in mathematics. Journal of Computer Assisted Learning 39, 3, 823–840.
Boyer, K. E., Phillips, R., Wallis, M., Vouk, M., and Lester, J. 2008. Balancing cognitive and motivational scaffolding in tutorial dialogue. In Intelligent Tutoring Systems: 9th International Conference, ITS 2008, Montreal, Canada, June 23-27, 2008 Proceedings 9. Springer, 239–249.
Brooks, M., Basu, S., Jacobs, C., and Vanderwende, L. 2014. Divide and correct: Using clusters to grade short answers at scale. In Proceedings of the First ACM Conference on Learning@ Scale. 89–98.
Cheng, L., Hampton, J., and Kumar, S. 2022. Engaging students via synchronous peer feedback in a technology-enhanced learning environment. Journal of Research on Technology in Education 56, sup1, 347–371.
Chi, M. T., De Leeuw, N., Chiu, M.-H., and LaVancher, C. 1994. Eliciting self-explanations improves understanding. Cognitive Science 18, 3, 439–477.
Chi, M. T. H. 2000. Self-explaining expository texts: The dual processes of generating inferences and repairing mental models. In Advances in Instructional Psychology, Volume 5, R. Glaser, Ed. Routledge, New York, NY, 161–238.
Coburn, C. E. and Turner, E. O. 2012. The practice of data use: An introduction. American Journal of Education 118, 2, 99–111.
Corbett, A. T. and Anderson, J. R. 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-adapted Interaction 4, 253–278.
Corral, D. and Carpenter, S. K. 2020. Facilitating transfer through incorrect examples and explanatory feedback. Quarterly Journal of Experimental Psychology 73, 9, 1340–1359.
Dawson, P., Henderson, M., Mahoney, P., Phillips, M., Ryan, T., Boud, D., and Molloy, E. 2019. What makes for effective feedback: Staff and student perspectives. Assessment & Evaluation in Higher Education 44, 1, 25–36.
Erickson, J. A., Botelho, A. F., McAteer, S., Varatharaj, A., and Heffernan, N. T. 2020. The automated grading of student open responses in mathematics. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge. 615–624.
Feng, M., Huang, C., and Collins, K. 2023. Promising long term effects of assistments online math homework support. In International Conference on Artificial Intelligence in Education. Springer, 212–217.
Fong, C. J. and Schallert, D. L. 2023. “Feedback to the future”: Advancing motivational and emotional perspectives in feedback research. Educational Psychologist 58, 3, 146–161.
Gaddipati, S. K., Nair, D., and Plöger, P. G. 2020. Comparative evaluation of pretrained transfer learning models on automatic short answer grading. arXiv preprint arXiv:2009.01303.
Gan, Z., An, Z., and Liu, F. 2021. Teacher feedback practices, student feedback motivation, and feedback behavior: How are they associated with learning outcomes? Frontiers in Psychology 12, 697045.
Grawemeyer, B., Mavrikis, M., Holmes, W., Gutiérrez-Santos, S., Wiedmann, M., and Rummel, N. 2017. Affective learning: Improving engagement and enhancing learning with affect-aware feedback. User Modeling and User-Adapted Interaction 27, 119–158.
Gurung, A., Baral, S., Lee, M. P., Sales, A. C., Haim, A., Vanacore, K. P., McReynolds, A. A., Kreisberg, H., Heffernan, C., and Heffernan, N. T. 2023. How common are common wrong answers? crowdsourcing remediation at scale. In Proceedings of the Tenth ACM Conference on Learning@ Scale. 70–80.
Gusukuma, L., Bart, A. C., Kafura, D., and Ernst, J. 2018. Misconception-driven feedback: Results from an experimental study. In Proceedings of the 2018 ACM Conference on International Computing Education Research. 160–168.
Hargreaves, E. 2014. The practice of promoting primary pupils’ autonomy: Examples of teacher feedback. Educational Research 56, 3, 295–309.
Hattie, J. and Timperley, H. 2007. The power of feedback. Review of Educational Research 77, 1, 81–112.
Hauke, J. and Kossowski, T. 2011. Comparison of values of Pearson’s and Spearman’s correlation coefficients on the same sets of data. Quaestiones Geographicae 30, 2, 87–93.
Heffernan, N. T. and Heffernan, C. L. 2014. The ASSISTments ecosystem: Building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. International Journal of Artificial Intelligence in Education 24, 470–497.
Hou, W.-J. and Tsao, J.-H. 2011. Automatic assessment of students’ free-text answers with different levels. International Journal on Artificial Intelligence Tools 20, 02, 327–347.
Klein, R., Kyrilov, A., and Tokman, M. 2011. Automated assessment of short free-text responses in computer science using latent semantic analysis. In Proceedings of the 16th Annual Joint Conference on Innovation and Technology in Computer Science Education. 158–162.
Kluger, A. N. and DeNisi, A. 1996. The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin 119, 2, 254.
Konold, K. E., Miller, S. P., and Konold, K. B. 2004. Using teacher feedback to enhance student learning. Teaching Exceptional Children 36, 6, 64–69.
Kulik, J. A. and Kulik, C.-L. C. 1988. Timing of feedback and verbal learning. Review of Educational Research 58, 1, 79–97.
Kwon, O. N., Park, J. H., and Park, J. S. 2006. Cultivating divergent thinking in mathematics through an open-ended approach. Asia Pacific Education Review 7, 51–61.
Labuhn, A. S., Zimmerman, B. J., and Hasselhorn, M. 2010. Enhancing students’ self-regulation and mathematics performance: The influence of feedback and self-evaluative standards. Metacognition and learning 5, 173–194.
Lee, H.-S., Pallant, A., Pryputniewicz, S., Lord, T., Mulholland, M., and Liu, O. L. 2019. Automated text scoring and real-time adjustable feedback: Supporting revision of scientific arguments involving uncertainty. Science Education 103, 3, 590–622.
Madnani, N., Burstein, J., Sabatini, J., and O’Reilly, T. 2013. Automated scoring of summary-writing tasks designed to measure reading comprehension. Grantee Submission.
McNamara, D. S., Crossley, S. A., Roscoe, R. D., Allen, L. K., and Dai, J. 2015. A hierarchical classification approach to automated essay scoring. Assessing Writing 23, 35–59.
McNichols, H., Lee, J., Fancsali, S., Ritter, S., and Lan, A. 2024. Can large language models replicate ITS feedback on open-ended math questions? In Proceedings of the 17th International Conference on Educational Data Mining, B. Paaßen and C. D. Epp, Eds. International Educational Data Mining Society, Atlanta, Georgia, USA, 769–775.
Meier, S. L., Rich, B. S., and Cady, J. 2006. Teachers’ use of rubrics to score non-traditional tasks: Factors related to discrepancies in scoring. Assessment in Education 13, 01, 69–95.
Monarch, R. M. 2021. Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI. Simon and Schuster.
Moreno, R. 2004. Decreasing cognitive load for novice students: Effects of explanatory versus corrective feedback in discovery-based multimedia. Instructional Science 32, 1, 99–113.
Mory, E. H. 2013. Feedback research revisited. In Handbook of Research on Educational Communications and Technology, J. M. Spector, M. D. Merrill, J. Elen, and M. J. Bishop, Eds. Routledge, New York, NY, 738–776.
Munroe, L. 2015. The open-ended approach framework. European Journal of Educational Research 4, 3, 97–104.
Narciss, S. 2013. Designing and evaluating tutoring feedback strategies for digital learning. Digital Education Review 23, 7–26.
Narciss, S., Sosnovsky, S., Schnaubert, L., Andrès, E., Eichelmann, A., Goguadze, G., and Melis, E. 2014. Exploring feedback and student characteristics relevant for personalizing feedback strategies. Computers & Education 71, 56–76.
Navarro-Gonzalez, D. and Lorenzo-Seva, U. 2021. EFA.MRFA: Dimensionality Assessment Using Minimum Rank Factor Analysis. R package.
Qi, H., Wang, Y., Dai, J., Li, J., and Di, X. 2019. Attention-based hybrid model for automatic short answer scoring. In Simulation Tools and Techniques: 11th International Conference, SIMUtools 2019, Chengdu, China, July 8–10, 2019, Proceedings 11. Springer, 385–394.
Rau, M. A., Aleven, V., and Rummel, N. 2015. Successful learning with multiple graphical representations and self-explanation prompts. Journal of Educational Psychology 107, 1, 30.
Ryan, T., Henderson, M., Ryan, K., and Kennedy, G. 2021. Designing learner-centred text-based feedback: A rapid review and qualitative synthesis. Assessment & Evaluation in Higher Education 46, 6, 894–912.
Sadler, D. R. 1989. Formative assessment and the design of instructional systems. Instructional Science 18, 2, 119–144.
Saldaña, J. 2021. The coding manual for qualitative researchers. SAGE publications Ltd.
Shute, V. J. 2008. Focus on formative feedback. Review of Educational Research 78, 1, 153–189.
Sweller, J. 2011. Cognitive load theory. In Psychology of Learning and Motivation, B. H. Ross, Ed. Vol. 55. Elsevier, San Diego, CA, 37–76.
Taghipour, K. and Ng, H. T. 2016. A neural approach to automated essay scoring. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 1882–1891.
Thompson, D. R. and Senk, S. L. 1998. Implementing the assessment standards for school mathematics: Using rubrics in high school mathematics courses. The Mathematics Teacher 91, 9, 786–793.
Van der Kleij, F. M., Feskens, R. C., and Eggen, T. J. 2015. Effects of feedback in a computer-based learning environment on students’ learning outcomes: A meta-analysis. Review of Educational Research 85, 4, 475–511.
Vanacore, K., Gurung, A., Sales, A., and Heffernan, N. T. 2024. The effect of assistance on gamers: Assessing the impact of on-demand hints & feedback availability on learning for students who game the system. In Proceedings of the 14th Learning Analytics and Knowledge Conference. 462–472.
Wisniewski, B., Zierer, K., and Hattie, J. 2020. The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology 10, 487662.
Wylie, R. and Chi, M. T. H. 2014. The self-explanation principle in multimedia learning. In The Cambridge Handbook of Multimedia Learning, R. E. Mayer, Ed. Cambridge University Press, Cambridge, UK, 413–432.
Yang, M. and Carless, D. 2013. The feedback triangle and the enhancement of dialogic feedback processes. Teaching in Higher Education 18, 3, 285–297.
Zhao, S., Zhang, Y., Xiong, X., Botelho, A., and Heffernan, N. 2017. A memory-augmented neural model for automated grading. In Proceedings of the Fourth (2017) ACM Conference on Learning@ Scale. 189–192.
Issue
Section
EDM 2025 Journal Track
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:
- The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
- Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
- The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons 4.0 License (Attribution-Noncommercial-No Derivatives 4.0 International), or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
- Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
- Noncommercial—other users (including Publisher) may not use this Work for commercial purposes;
- No Derivative Works—other users (including Publisher) may not alter, transform, or build upon this Work,with the understanding that any of the above conditions can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
- The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
- Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
- The Author represents and warrants that:
- the Work is the Author’s original work;
- the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
- the Work is not pending review or under consideration by another publisher;
- the Work has not previously been published;
- the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
- the Work contains no libel, invasion of privacy, or other unlawful matter.
- The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.