Gender differences in Predicting STEM Choice by Affective States and Behaviors in Online Mathematical Problem Solving: Positive-Affect-to-Success Hypothesis



Published Aug 23, 2020
Mei-Shiu Chiu


This study aims to identify effective affective states and behaviors of middle-school students' online mathematics learning in predicting their choices to study science, technology, engineering, and mathematics (STEM) in higher education based on a positive-affect-to-success hypothesis. The dataset (591 students and 316,974 actions) was obtained from the ASSISTments project. In the ASSISTments intelligent tutoring system, students completed mathematical problem-solving tasks, and the data was processed to infer their action-level affective states and behaviors, which were averaged to form student-level measures. The students' future STEM choice was predicted by the student- and action-level affective states and behaviors using logistic regression (LR), ordinary least squares regressions with standardized scores (ORz), and random forest with permutation importance and SHAP values (RFPS). The results revealed that student- and action-level gaming behavior consistently predict STEM choice. In addition to gaming, female students are more likely to study STEM if they are less bored and more off-task, and male students if more concentrated and less frustrated. ORz generates theoretically plausible results and identifies sufficiently distinguishable affective states and behaviors. Suggestions for educational practice and research are provided for adaptive teaching.

How to Cite

Chiu, M.-S. (2020). Gender differences in Predicting STEM Choice by Affective States and Behaviors in Online Mathematical Problem Solving: Positive-Affect-to-Success Hypothesis. Journal of Educational Data Mining, 12(2), 48–77.
Abstract 392 | PDF Downloads 297



affect, gender differences, intelligent tutoring systems, mathematical problem solving, STEM choice

AIKEN, L. S., & WEST, S. G. (1991). Multiple regression: Testing and interpreting interactions. Newbury Park, CA: Sage.

ALLISON, P. D. (2012). Logistic regression using SAS: Theory and application (2nd ed.). Cary, NC: SAS Institute.

BAKER, R. S. J. D., CORBETT, A. T., ROLL, I., KOEDINGER, K. R., ALEVEN, V., COCEA, M., HERSHKOVITZ, A., DE CARAVALHO, A. M. J. B., MITROVIC, A., & MATHEWS, M. (2013). Modeling and studying gaming the system with educational data mining. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 97-115). New York: Springer.

BAKER, R. S., & INVENTADO, P. S. (2014). Educational data mining and learning analytics. In J. A. Larusson & B. White (Eds.), Learning analytics: From research to practice (pp. 61-75). New York, NY: Springer.

BAKER, R. S. J. D., & ROSSI, L. M. (2013). Assessing the disengaged behavior of learners. In R. Sottilare, A. Graesser, X. Hu, & H. Holden (Eds.), Design recommendations for intelligent tutoring systems: Vol. 1. Learner modeling (pp. 155-165). Orlando, FL: U.S. Army Research Lab.

BAKER, R. S., D'MELLO, S. K., RODRIGO, M. M. T., & GRAESSER, A. C. (2010). Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive–affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies, 68, 223-241.

BANERJEE, P. A. (2016). A longitudinal evaluation of the impact of STEM enrichment and enhancement activities in improving educational outcomes: Research protocol. International Journal of Educational Research, 76, 1-11.

BARKATSAS, A. T., KASIMATIS, K., & GIALAMAS, V. (2009). Learning secondary mathematics with technology: Exploring the complex interrelationship between students' attitudes, engagement, gender and achievement. Computers & Education, 52, 562-570.

BECKER, D. (2019). Machine learning explainability course home page. Retrieved from

BERNIER, A., CARLSON, S. M., & WHIPPLE, N. (2010). From external regulation to self‐regulation: Early parenting precursors of young children's executive functioning. Child Development, 81, 326-339.

BOTELHO, A. F., BAKER, R. S., & HEFFERNAN, N. T. (2017). Improving sensor-free affect detection using deep learning. Proceedings of the 18th International Conference on Artificial Intelligence in Education (pp. 40-51). Springer.

BOWLES, M. (2015). Machine learning in Python: Essential techniques for predictive analysis. Indianapolis, IN: John Wiley & Sons.

CARLI, L. L., ALAWA, L., LEE, Y., ZHAO, B., & KIM, E. (2016). Stereotypes about gender and science: Women≠ scientists. Psychology of Women Quarterly, 40, 244-260.

CERINSEK, G., HRIBAR, T., GLODEZ, N., & DOLINSEK, S. (2013). Which are my future career priorities and what influenced my choice of studying science, technology, engineering or mathematics? Some insights on educational choice—case of Slovenia. International Journal of Science Education, 35, 2999-3025.

CHIU, M.-S. (2007). Mathematics as mother/basis of science in affect: Analysis of TIMSS 2003 data. In J. H. Woo, H. C. Lew, K. S. Park, & D. Y. Seo (Eds.), Proceedings of the 31st Conference of the International Group for the Psychology of Mathematics Education, 2, 145-152. Seoul: PME.

CHIU, M.-S. (2012). The internal/external frame of reference model, big-fish-little-pond effect, and combined model for mathematics and science. Journal of Educational Psychology, 104, 87-107.

CHIU, M.-S. (2017). High school student rationales for studying advanced science: Analysis of their psychological and cultural capitals. Journal of Advances in Education Research, 2, 171-182.

CLIFFORD, M. M. (1988). Failure tolerance and academic risk-taking in ten-to twelve-year-old students. British Journal of Educational Psychology, 58, 15-27.

COHEN, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.

DE WITTE, K., HAELERMANS, C., & ROGGE, N. (2015). The effectiveness of a computer‐assisted math learning program. Journal of Computer Assisted Learning, 31, 314-329.

D'MELLO, S., PICARD, R. W., & GRAESSER, A. (2007). Toward an affect-sensitive AutoTutor. IEEE Intelligent Systems, 22(4), 53-61.

EASTWOOD, J. D., FRISCHEN, A., FENSKE, M. J., & SMILEK, D. (2012). The unengaged mind: Defining boredom in terms of attention. Perspectives on Psychological Science, 7, 482-495.

ELSE-QUEST, N. M., HYDE, J. S., & LINN, M. C. (2010). Cross-national patterns of gender differences in mathematics: A meta-analysis. Psychological Bulletin, 136, 103-127.

FAWCETT, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27, 861-874.

GAZLEY, J. L., REMICH, R., NAFFZIGER-HIRSCH, M. E., KELLER, J., CAMPBELL, P. B., & MCGEE, R. (2014). Beyond preparation: Identity, cultural capital, and readiness for graduate school in the biomedical sciences. Journal of Research in Science Teaching, 51, 1021-1048.

GEE, J. P. (2005a). Good video games and good learning. Phi Kappa Phi Forum, 82(2), 34-37.

GEE, J. P. (2005b). Learning by design: Good video games as learning machines. E-learning and Digital Media, 2(1), 5-16.

GLYNN, S. M., TAASOOBSHIRAZI, G., & BRICKMAN, P. (2007). Nonscience majors learning science: A theoretical model of motivation. Journal of Research in Science Teaching, 44, 1088-1107.

GUISO, L., MONTE, F., SAPIENZA, P., & ZINGALES, L. (2008). Culture, gender, and mathematics. Science, 320, 1164-1165.

HAIR, J. F., JR., BLACK, W. C., BABIN, B. J., ANDERSON, R. E., & TATHAM, R. L. (2006). Multivariate data analysis (6th ed.). Upper Saddle River, NJ: Prentice-Hall.

HEFFERNAN, N. T., & HEFFERNAN, C. L. (2014). The ASSISTments ecosystem: building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. International Journal of Artificial Intelligence in Education, 24(4), 470-497.

HELLEVIK, O. (2009). Linear versus logistic regression when the dependent variable is a dichotomy. Quality & Quantity, 43, 59-74.

HO, H., SENTURK, D., LAM, A. G., ZIMMER, J. M., HONG, S., OKAMOTO, Y., CHIU, S., NAKAZAWA, Y., & WANG, C. (2000). The affective and cognitive dimensions of math anxiety: A cross-national study. Journal for Research in Mathematics Education, 31, 362-379.

HOQUE, M. E., MCDUFF, D. J., & PICARD, R. W. (2012). Exploring temporal patterns in classifying frustrated and delighted smiles. IEEE Transactions on Affective Computing, 3(3), 323-334.

HSU, P. L., ROTH, W. M., MARSHALL, A., & GUENETTE, F. (2009). To be or not to be? Discursive resources for (dis-)identifying with science‐related careers. Journal of Research in Science Teaching, 46, 1114-1136.

HUSSEIN, A. (2015). The use of triangulation in social sciences research: Can qualitative and quantitative methods be combined?. Journal of Comparative Social Work, 4(1), 1-12.

HYDE, J. S. (2005). The gender similarities hypothesis. American Psychologist, 60, 581-592.

KAI, S., ALMEDA, M. V., BAKER, R. S., HEFFERNAN, C., & HEFFERNAN, N. (2018). Decision tree modeling of wheel-spinning and productive persistence in Skill Builders. Journal of Educational Data Mining, 10(1), 36-71.

KOEHLER, M. S. (1990). Classrooms, teachers, and gender differences in mathematics. In E. Fennema & G. C. Leder (Eds.), Mathematics and gender (pp. 128-148). New York: Columbia University, Teachers College.

KOLLER, O., BAUMERT, J., & SCHNABEL, K. (2001). Does interest matter? The relationship between academic interest and achievement in mathematics. Journal for Research in Mathematics Education, 32, 448-470.

LEE, D. M. C., RODRIGO, M. M. T., BAKER, R. S., SUGAY, J. O., & CORONEL, A. (2011). Exploring the relationship between novice programmer confusion and achievement. Proceedings of the 4th bi-annual International Conference on Affective Computing and Intelligent Interaction (pp. 175-184). Berlin, Heidelberg: Springer.

LIU, Z., PATARANUTAPORN, V., OCUMPAUGH, J., BAKER, R.S.J.D. (2013) Sequences of frustration and confusion, and Learning. Proceedings of the 6th International Conference on Educational Data Mining, 114-120.

LYUBOMIRSKY, S., KING, L., & DIENER, E. (2005). The benefits of frequent positive affect: Does happiness lead to success? Psychological Bulletin, 131, 803-855.

MALTESE, A. V., & TAI, R. H. (2010). Eyeballs in the fridge: Sources of early interest in science. International Journal of Science Education, 32, 669-685.

MASON, J., BURTON, L., & STACEY, K. (1996). Thinking mathematically. Essex: Addison-Wesley.

MCLEOD, D. B. (1992). Research on affect in mathematics education: A reconceptualisation. In D. A. Grouws (Ed.), Handbook of Research on Mathematics Teaching and Learning: a Project of the National Council of Teachers of Mathematics (pp. 575-596). New York: Macmillan.

MCLEOD, D. B. (1994). Research on affect and mathematics learning in the JRME: 1970 to the present. Journal for Research in Mathematics Education, 25, 637-647.

MEECE, J. L. WIGFIELD, A., & ECCLES, J. S. (1990). Predictors of math anxiety and its influence on young adolescents' course enrollment intentions and performance in mathematics. Journal of Educational Psychology, 82, 60-70.

NAKAMURA, J., & CSIKSZENTMIHALYI, M. (2002). The concept of flow. In C. R. Snyder & S. J. Lopez (Eds.), Handbook of Positive Psychology (pp. 89-105). New York, NY: Oxford University Press.

NUGENT, G., BARKER, B., WELCH, G., GRANDGENETT, N., WU, C., & NELSON, C. (2015). A model of factors contributing to STEM learning and career orientation. International Journal of Science Education, 37, 1067-1088.

OCUMPAUGH, J., BAKER, R. S., & RODRIGO, M. M. T. (2015). Baker Rodrigo Ocumpaugh Monitoring Protocol (BROMP) 2.0 Technical and Training Manual. New York, NY: Teachers College, Columbia University. Manila, Philippines: Ateneo Laboratory for the Learning Sciences.

OCUMPAUGH, J., SAN PEDRO, M. O., LAI, H. Y., BAKER, R. S., & BORGEN, F. (2016). Middle school engagement with mathematics software and later interest and self-efficacy for STEM careers. Journal of Science Education and Technology, 25(6), 877-887.

ORGANIZATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT. (2014). PISA 2012 results: What students know and can do – student performance in mathematics, reading and science (Volume I, Revised edition, February 2014). Paris, France: Author. Retrieved from

ORGANIZATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT. (2015). The ABC of gender equality in education: Aptitude, behavior, confidence. Paris: OECD Publishing.

PAJARES, F., & MILLER, M. D. (1994). Role of self-efficacy and self-concept beliefs in mathematical problem solving: A path analysis. Journal of Educational Psychology, 86, 193-203.

PARDOS, Z. A., BAKER, R. S., SAN PEDRO, M., GOWDA, S. M., & GOWDA, S. M. (2014). Affective states and state tests: Investigating how affect and engagement during the school year predict end-of-year learning outcomes. Journal of Learning Analytics, 1, 107-128.

PETERSON, P. L., & FENNEMA, E. (1985). Effective teaching, student engagement in classroom activities, and sex-related differences in learning mathematics. American Educational Research Journal, 22, 309-335.

PINTRICH, P. R. (2003). A motivational science perspective on the role of student motivation in learning and teaching contexts. Journal of Educational Psychology, 95, 667-686.

POHLMANN, J. T., & LEITNER, D. W. (2003). A comparison of ordinary least squares and logistic regression. Ohio Journal of Science, 103, 118-125.

SAN PEDRO, M. O. Z., BAKER, R. S., BOWERS, A. J., & HEFFERNAN, N. T. (2013a). Predicting college enrollment from student interaction with an intelligent tutoring system in middle school. Proceedings of the 6th International Conference on Educational Data Mining (pp. 177–184).

SAN PEDRO, M. O. Z., BAKER, R., GOWDA, S. M., & HEFFERNAN, N. T. (2013b). Towards an understanding of affect and knowledge from student interaction with an intelligent tutoring system. In Proceedings of the 16th International Conference on Artificial Intelligence and Education, 9–13 July, Exeter, UK, 41–50.

SAN PEDRO, M. O. Z., OCUMPAUGH, J., BAKER, R., & HEFFERNAN, N. T. (2014). Predicting STEM and non-STEM college major enrollment from middle school interaction with mathematics educational software. Proceedings of the 7th International Conference on Educational Data Mining, 276-279. Memphis, TN: International Educational Data Mining Society.

SHUTE, V. J., D'MELLO, S., BAKER, R., CHO, K., BOSCH, N., OCUMPAUGH, J., VENTURA, M., & ALMEDA, V. (2015). Modeling how incoming knowledge, persistence, affective states, and in-game progress influence student learning from an educational game. Computers & Education, 86, 224-235.

STROBL, C., MALLEY, J., & TUTZ, G. (2009). An introduction to recursive partitioning: Rationale, application, and characteristics of classification and regression trees, bagging, and random forests. Psychological Methods, 14(4), 323-348.

TEMPELAAR, D. T., RIENTIES, B., & GIESBERS, B. (2015). In search for the most informative data for feedback generation: Learning Analytics in a data-rich context. Computers in Human Behavior, 47, 157-167.

TZE, V. M., DANIELS, L. M., & KLASSEN, R. M. (2016). Evaluating the relationship between boredom and academic outcomes: A meta-analysis. Educational Psychology Review, 28, 119-144.

YÜKSEL-ŞAHIN, F. (2008). Mathematics anxiety among 4th and 5th grade Turkish elementary school students. International Electronic Journal of Mathematics Education, 3, 179-192.

ZELDIN, A. L., & PAJARES, F. (2000). Against the odds: Self-efficacy beliefs of women in mathematical, scientific, and technological careers. American Educational Research Journal, 37, 215-246.

ZHU, Z. (2007). Gender differences in mathematical problem solving patterns: A review of literature. International Education Journal, 8, 187-203.

ZIMMERMAN, B. J., & MARTINEZ-PONS, M. (1990). Student differences in self-regulated learning: Relating grade, sex, and giftedness to self-efficacy and strategy use. Journal of Educational Psychology, 82, 51-59.
Special Issue on ASSISTments Longitudinal Data