You asked, now what? Modeling Students' Help-Seeking and Coding actions from Request to Resolution
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Dec 18, 2022
Zhikai Gao
Bradley Erickson
Yiqiao Xu
Collin Lynch
Sarah Heckman
Tiffany Barnes
https://orcid.org/0000-0002-9640-4290
Abstract
Demand for education in Computer Science has increased markedly in recent years. With increased demand
has come to an increased need for student support, especially for courses with large programming
projects. Instructors commonly provide online post forums or office hours to address this massive demand
for help requests. Identifying what types of questions students are asking in those interactions and what
triggers their help requests can in turn assist instructors in better managing limited help-providing resources.
In this study, we aim to explore students’ help-seeking actions from the two separate approaches
we mentioned before and investigate their coding actions before help requests to understand better what
motivates students to seek help in programming projects. We collected students’ help request data and
commit logs from two Fall offerings of a CS2 course. In our analysis, we first believe that different types
of questions should be related to different behavioral patterns. Therefore, we first categorized students’
help requests based on their content (e.g., Implementation, General Debugging, or Addressing Teaching
Staff (TS) Test Failures). We found that General Debugging is the most frequently asked question. Then
we analyzed how the popularity of each type of request changed over time. Our results suggest that
implementation is more popular in the early stage of the project cycle, and it changes to General Debugging
and Addressing TS Failures in the later stage. We also calculated the accuracy of students’ commit
frequency one hour before their help requests; the results show that before Implementation requests, the
commit frequency is significantly lower, and before TS failure requests, the frequency is significantly
higher. Moreover, we checked before any help request whether students changed their source code or
test code. The results show implementation requests related to higher chances of source code changes
and coverage questions related to more test code changes. Moreover, we use a Markov Chain model to
show students’ action sequences before, during, and after the requests. And finally, we explored students’
progress after the office hours interaction and found that over half of the students improved the correctness
of their code after 20 minutes of their office hours interaction addressing TS failures ends.
How to Cite
Gao, Z., Erickson, B., Xu, Y., Lynch, C., Heckman, S., & Barnes, T. (2022). You asked, now what? Modeling Students’ Help-Seeking and Coding actions from Request to Resolution. Journal of Educational Data Mining, 14(3), 109–131. https://doi.org/10.5281/zenodo.7304908
##plugins.themes.bootstrap3.article.details##
Keywords
computer science education, help-seeking, self-regulation, blended learning, markov models
References
ALEVEN, V., MCLAREN, B., ROLL, I., AND KOEDINGER, K. 2006. Toward meta-cognitive tutoring: A model of help seeking with a cognitive tutor. International Journal of Artificial Intelligence in Education 16, 2, 101–128.
BEAUBOUEF, T. AND MASON, J. 2005. Why the high attrition rate for computer science students: some thoughts and observations. ACM SIGCSE Bulletin 37, 2, 103–106.
BUMBACHER, E., SANDES, A., DEUTSCH, A., AND BLIKSTEIN, P. 2013. Student coding styles as predictors of help-seeking behavior. In Proceeding of 16th International Conference on Artificial Intelligence in Education, H. C. Lane, K. Yacef, J. Mostow, and P. Pavlik, Eds. Springer, 856–859.
CHEONG, Y. F., PAJARES, F., AND OBERMAN, P. S. 2004. Motivation and academic help-seeking in high school computer science. Computer Science Education 14, 1, 3–19.
GAO, Z., ERICKSON, B., XU, Y., LYNCH, C., HECKMAN, S., AND BARNES, T. 2022. Admitting you have a problem is the first step: Modeling when and why students seek help in programming assignments. In Proceedings of the 15th International Conference on Educational Data Mining, A. Mitrovic and N. Bosch, Eds. International Educational Data Mining Society, Durham, United Kingdom, 508– 514.
GAO, Z., HECKMAN, S., AND LYNCH, C. 2022. Who uses office hours? a comparison of in-person and virtual office hours utilization. In Proceedings of the 53rd ACM Technical Symposium V.1 on Computer Science Education. SIGCSE 2022. Association for Computing Machinery, New York, NY, USA, 300–306.
GAO, Z., LYNCH, C., HECKMAN, S., AND BARNES, T. 2021. Automatically classifying student help requests: A multi-year analysis. In Proceedings of the 14th International Conference on Educational 129 Journal of Educational Data Mining, Volume 14, No 3, 2022 Data Mining, I.-H. Hsiao, S. Sahebi, F. Bouchet, and J.-J. Vie, Eds. International Educational Data Mining Society, 81–92.
GEYER, C. J. 1992. Practical Markov Chain Monte Carlo. Statistical Science 7, 4, 473 – 483.
GRIFFIN, W., COHEN, S. D., BERNDTSON, R., BURSON, K. M., CAMPER, K. M., CHEN, Y., AND SMITH, M. A. 2014. Starting the conversation: An exploratory study of factors that influence student office hour use. College Teaching 62, 3, 94–99.
GUERRERO, M. AND ROD, A. B. 2013. Engaging in office hours: A study of student-faculty interaction and academic performance. Journal of Political Science Education 9, 4, 403–416.
KARABENICK, S. A. 2011. Classroom and technology-supported help seeking: The need for converging research paradigms. Learning and instruction 21, 2, 290–296.
KARLSSON, L., KOIVULA, L., RUOKONEN, I., KAJAANI, P., ANTIKAINEN, L., AND RUISMÄ KI, H. 2012. From novice to expert: Information seeking processes of university students and researchers. Procedia-Social and Behavioral Sciences 45, 577–587.
MARWAN, S., DOMBE, A., AND PRICE, T. W. 2020. Unproductive help-seeking in programming: What it is and how to address it. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education. SIGCSE 2020. Association for Computing Machinery, 54–60.
NEWMAN, R. S. 1990. Children’s help-seeking in the classroom: The role of motivational factors and attitudes. Journal of Educational Psychology 82, 1, 71.
NEWMAN, R. S. 1991. Goals and self-regulated learning: What motivates children to seek academic help. Advances in Motivation and Achievement 7, 151–183.
NEWMAN, R. S. 1994. Adaptive help seeking: A strategy of self-regulated learning. Self-regulation of Learning and Performance: Issues and Educational Applications, 283–301.
NEWMAN, R. S. 2008. The motivational role of adaptive help seeking in self-regulated learning. Motivation and self-regulated learning: Theory, research, and applications, 315–337.
NEWMAN, R. S. AND SCHWAGER, M. T. 1995. Students’ help seeking during problem solving: Effects of grade, goal, and prior achievement. American Educational Research Journal 32, 2, 352–376.
OBERMAN, P. S. 2002. Academic help-seeking in the high school computer science classroom: Relationship to motivation, achievement, gender, and ethnicity. Ph.D. thesis, Emory University.
ROLL, I., ALEVEN, V., MCLAREN, B. M., AND KOEDINGER, K. R. 2011. Improving students’ helpseeking skills using metacognitive feedback in an intelligent tutoring system. Learning and instruction 21, 2, 267–280.
ROLL, I., ALEVEN, V., MCLAREN, B. M., RYU, E., D BAKER, R. S., AND KOEDINGER, K. R.2006. The help tutor: Does metacognitive feedback improve students’ help-seeking actions, skills and learning? In International conference on intelligent tutoring systems, M. Ikeda, K. D. Ashley, and T.-W. Chan, Eds. Springer, 360–369.
RYAN, A. M. AND PINTRICH, P. R. 1997. ”should i ask for help?” the role of motivation and attitudes in adolescents’ help seeking in math class. Journal of Educational Psychology 89, 2, 329.
NELSON-LE GALL, S. 1981. Help-seeking: An understudied problem-solving skill in children. Developmental Review 1, 3, 224–246.
SMITH, A. J., BOYER, K. E., FORBES, J., HECKMAN, S., AND MAYER-PATEL, K. 2017. My digital hand: A tool for scaling up one-to-one peer teaching in support of computer science learning. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education. SIGCSE ’17. Association for Computing Machinery, New York, NY, USA, 549–554. 130 Journal of Educational Data Mining, Volume 14, No 3, 2022 TALLARIDA, R. J. AND MURRAY, R. B. 1987. Chi-square test. In Manual of Pharmacologic Calculations: With Computer Programs, B. Jones, Ed. Springer New York, New York, NY, 140–142.
VAESSEN, B. E., PRINS, F. J., AND JEURING, J. 2014. University students’ achievement goals and help-seeking strategies in an intelligent tutoring system. Computers & Education 72, 196–208.
WON, S., HENSLEY, L. C., AND WOLTERS, C. A. 2021. Brief research report: Sense of belonging and academic help-seeking as self-regulated learning. The Journal of Experimental Education 89, 1, 112–124.
ZAHN, M., BATTESTILLI, L., AND HECKMAN, S. 2022. Academic help seeking patterns in introductory computer science courses. In 2022 ASEE Annual Conference & Exposition. ASEE Conferences, Minneapolis, MN. https://peer.asee.org/41526, Paper ID #37738.
ZIMMERMAN, B. J. 1989. Models of self-regulated learning and academic achievement. In Self-regulated learning and academic achievement, B. Zimmerman and D. Schunk, Eds. Springer, 1–25.
ZIMMERMAN, B. J. 1994. Dimensions of academic self-regulation: A conceptual framework for education. Self-regulation of Learning and Performance: Issues and Educational Applications 1, 33–21.
ZIMMERMAN, B. J. 2000. Attaining self-regulation: A social cognitive perspective. In Handbook of self-regulation, M. Boekaerts, P. R. Pintrich, and M. Zeidner, Eds. Academic Press, 13–39.
ZUSHO, A., KARABENICK, S. A., BONNEY, C. R., AND SIMS, B. C. 2007. Contextual determinants of motivation and help seeking in the college classroom. The scholarship of teaching and learning in higher education: An evidence-based perspective, 611–659.
BEAUBOUEF, T. AND MASON, J. 2005. Why the high attrition rate for computer science students: some thoughts and observations. ACM SIGCSE Bulletin 37, 2, 103–106.
BUMBACHER, E., SANDES, A., DEUTSCH, A., AND BLIKSTEIN, P. 2013. Student coding styles as predictors of help-seeking behavior. In Proceeding of 16th International Conference on Artificial Intelligence in Education, H. C. Lane, K. Yacef, J. Mostow, and P. Pavlik, Eds. Springer, 856–859.
CHEONG, Y. F., PAJARES, F., AND OBERMAN, P. S. 2004. Motivation and academic help-seeking in high school computer science. Computer Science Education 14, 1, 3–19.
GAO, Z., ERICKSON, B., XU, Y., LYNCH, C., HECKMAN, S., AND BARNES, T. 2022. Admitting you have a problem is the first step: Modeling when and why students seek help in programming assignments. In Proceedings of the 15th International Conference on Educational Data Mining, A. Mitrovic and N. Bosch, Eds. International Educational Data Mining Society, Durham, United Kingdom, 508– 514.
GAO, Z., HECKMAN, S., AND LYNCH, C. 2022. Who uses office hours? a comparison of in-person and virtual office hours utilization. In Proceedings of the 53rd ACM Technical Symposium V.1 on Computer Science Education. SIGCSE 2022. Association for Computing Machinery, New York, NY, USA, 300–306.
GAO, Z., LYNCH, C., HECKMAN, S., AND BARNES, T. 2021. Automatically classifying student help requests: A multi-year analysis. In Proceedings of the 14th International Conference on Educational 129 Journal of Educational Data Mining, Volume 14, No 3, 2022 Data Mining, I.-H. Hsiao, S. Sahebi, F. Bouchet, and J.-J. Vie, Eds. International Educational Data Mining Society, 81–92.
GEYER, C. J. 1992. Practical Markov Chain Monte Carlo. Statistical Science 7, 4, 473 – 483.
GRIFFIN, W., COHEN, S. D., BERNDTSON, R., BURSON, K. M., CAMPER, K. M., CHEN, Y., AND SMITH, M. A. 2014. Starting the conversation: An exploratory study of factors that influence student office hour use. College Teaching 62, 3, 94–99.
GUERRERO, M. AND ROD, A. B. 2013. Engaging in office hours: A study of student-faculty interaction and academic performance. Journal of Political Science Education 9, 4, 403–416.
KARABENICK, S. A. 2011. Classroom and technology-supported help seeking: The need for converging research paradigms. Learning and instruction 21, 2, 290–296.
KARLSSON, L., KOIVULA, L., RUOKONEN, I., KAJAANI, P., ANTIKAINEN, L., AND RUISMÄ KI, H. 2012. From novice to expert: Information seeking processes of university students and researchers. Procedia-Social and Behavioral Sciences 45, 577–587.
MARWAN, S., DOMBE, A., AND PRICE, T. W. 2020. Unproductive help-seeking in programming: What it is and how to address it. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education. SIGCSE 2020. Association for Computing Machinery, 54–60.
NEWMAN, R. S. 1990. Children’s help-seeking in the classroom: The role of motivational factors and attitudes. Journal of Educational Psychology 82, 1, 71.
NEWMAN, R. S. 1991. Goals and self-regulated learning: What motivates children to seek academic help. Advances in Motivation and Achievement 7, 151–183.
NEWMAN, R. S. 1994. Adaptive help seeking: A strategy of self-regulated learning. Self-regulation of Learning and Performance: Issues and Educational Applications, 283–301.
NEWMAN, R. S. 2008. The motivational role of adaptive help seeking in self-regulated learning. Motivation and self-regulated learning: Theory, research, and applications, 315–337.
NEWMAN, R. S. AND SCHWAGER, M. T. 1995. Students’ help seeking during problem solving: Effects of grade, goal, and prior achievement. American Educational Research Journal 32, 2, 352–376.
OBERMAN, P. S. 2002. Academic help-seeking in the high school computer science classroom: Relationship to motivation, achievement, gender, and ethnicity. Ph.D. thesis, Emory University.
ROLL, I., ALEVEN, V., MCLAREN, B. M., AND KOEDINGER, K. R. 2011. Improving students’ helpseeking skills using metacognitive feedback in an intelligent tutoring system. Learning and instruction 21, 2, 267–280.
ROLL, I., ALEVEN, V., MCLAREN, B. M., RYU, E., D BAKER, R. S., AND KOEDINGER, K. R.2006. The help tutor: Does metacognitive feedback improve students’ help-seeking actions, skills and learning? In International conference on intelligent tutoring systems, M. Ikeda, K. D. Ashley, and T.-W. Chan, Eds. Springer, 360–369.
RYAN, A. M. AND PINTRICH, P. R. 1997. ”should i ask for help?” the role of motivation and attitudes in adolescents’ help seeking in math class. Journal of Educational Psychology 89, 2, 329.
NELSON-LE GALL, S. 1981. Help-seeking: An understudied problem-solving skill in children. Developmental Review 1, 3, 224–246.
SMITH, A. J., BOYER, K. E., FORBES, J., HECKMAN, S., AND MAYER-PATEL, K. 2017. My digital hand: A tool for scaling up one-to-one peer teaching in support of computer science learning. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education. SIGCSE ’17. Association for Computing Machinery, New York, NY, USA, 549–554. 130 Journal of Educational Data Mining, Volume 14, No 3, 2022 TALLARIDA, R. J. AND MURRAY, R. B. 1987. Chi-square test. In Manual of Pharmacologic Calculations: With Computer Programs, B. Jones, Ed. Springer New York, New York, NY, 140–142.
VAESSEN, B. E., PRINS, F. J., AND JEURING, J. 2014. University students’ achievement goals and help-seeking strategies in an intelligent tutoring system. Computers & Education 72, 196–208.
WON, S., HENSLEY, L. C., AND WOLTERS, C. A. 2021. Brief research report: Sense of belonging and academic help-seeking as self-regulated learning. The Journal of Experimental Education 89, 1, 112–124.
ZAHN, M., BATTESTILLI, L., AND HECKMAN, S. 2022. Academic help seeking patterns in introductory computer science courses. In 2022 ASEE Annual Conference & Exposition. ASEE Conferences, Minneapolis, MN. https://peer.asee.org/41526, Paper ID #37738.
ZIMMERMAN, B. J. 1989. Models of self-regulated learning and academic achievement. In Self-regulated learning and academic achievement, B. Zimmerman and D. Schunk, Eds. Springer, 1–25.
ZIMMERMAN, B. J. 1994. Dimensions of academic self-regulation: A conceptual framework for education. Self-regulation of Learning and Performance: Issues and Educational Applications 1, 33–21.
ZIMMERMAN, B. J. 2000. Attaining self-regulation: A social cognitive perspective. In Handbook of self-regulation, M. Boekaerts, P. R. Pintrich, and M. Zeidner, Eds. Academic Press, 13–39.
ZUSHO, A., KARABENICK, S. A., BONNEY, C. R., AND SIMS, B. C. 2007. Contextual determinants of motivation and help seeking in the college classroom. The scholarship of teaching and learning in higher education: An evidence-based perspective, 611–659.
Issue
Section
Extended Articles from the EDM 2022 Conference
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:
- The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
- Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
- The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons 4.0 License (Attribution-Noncommercial-No Derivatives 4.0 International), or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
- Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
- Noncommercial—other users (including Publisher) may not use this Work for commercial purposes;
- No Derivative Works—other users (including Publisher) may not alter, transform, or build upon this Work,with the understanding that any of the above conditions can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
- The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
- Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
- The Author represents and warrants that:
- the Work is the Author’s original work;
- the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
- the Work is not pending review or under consideration by another publisher;
- the Work has not previously been published;
- the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
- the Work contains no libel, invasion of privacy, or other unlawful matter.
- The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.