Exploring the Impact of Symbol Spacing and Problem Sequencing on Arithmetic Performance: An Educational Data Mining Approach



Published Jun 27, 2024
Avery Harrison Closser Anthony F. Botelho Jenny Yun-Chen Chan


Experimental research on perception and cognition has shown that inherent and manipulated visual features
of mathematics problems impact individuals’ problem-solving behavior and performance. In a recent study,
we manipulated the spacing between symbols in arithmetic expressions to examine its effect on 174
undergraduate students’ arithmetic performance but found results that were contradictory to most of the
literature (Closser et al., 2023). Here, we applied educational data mining (EDM) methods to that dataset at
the problem level to investigate whether inherent features of the 32 experimental problems (i.e., problem
composition, problem order) may have caused unintended effects on students’ performance. We found that
students were consistently faster to correctly simplify expressions with the higher-order operator on the left,
rather than right, side of the expression. Furthermore, average response times varied based on the symbol
spacing of the current and preceding problem, suggesting that problem sequencing matters. However,
including or excluding problem identifiers in analyses changed the interpretation of results, suggesting that
the effect of sequencing may be impacted by other, undefined problem-level factors. These results advance
cognitive theories on perceptual learning and provide implications for educational researchers: online
experiments designed to investigate students’ performance on mathematics problems should include a
variety of problems, systematically examine the effects of problem order, and consider applying different
data analysis approaches to detect effects of inherent problem features. Moreover, EDM methods can be a
tool to identify nuanced effects on behavior and performance in the context of data from online platforms.

How to Cite

Closser, A. H., Botelho, A., & Chan, J. (2024). Exploring the Impact of Symbol Spacing and Problem Sequencing on Arithmetic Performance: An Educational Data Mining Approach. Journal of Educational Data Mining, 16(1), 84–111. https://doi.org/10.5281/zenodo.11403249
Abstract 71 | HTML Downloads 21 PDF Downloads 26



experimental design, causal inference, bootstrapping, arithmetic, perceptual learning

ABRAHAMSON, D. AND TRNINIC, D. 2015. Bringing forth mathematical concepts: Signifying sensorimotor enactment in fields of promoted action. ZDM Math Education 47, 295–306. doi: 10.1007/s11858-014-0620-0

ALIBALI, M. W., CROOKS, N. M., AND MCNEIL, N. M. 2018. Perceptual support promotes strategy generation: Evidence from equation solving. British Journal of Developmental Psychology 36, 153–168. https://doi.org/10.1111/bjdp.12203

BAKER, R. S. J. D., GOWDA, S. M., CORBETT, A. T., AND OCUMPAUGH, J. 2012. Towards Automatically Detecting Whether Student Learning is Shallow. In Intelligent Tutoring Systems: 11th International Conference. S. A. Cerri, J.C. William, G. Papadourakis, and K. Panourgia, Eds. Springer Berlin Heidelberg, 444-453.

BECK, J. E. AND MOSTOW, J. 2008. Improving the Educational Value of Intelligent Tutoring Systems through Data Mining. Frontiers in Artificial Intelligence and Applications, 158, 421.

BOOTH, J. L., MCGINN, K. M., BARBIERI, C., BEGOLLI, K. N., CHANG, B., MILLER-COTTO, D., YOUNG, L.K. AND DAVENPORT, J. L. 2017. Evidence for cognitive science principles that impact learning in mathematics. In Acquisition of Complex Arithmetic Skills and Higherorder Mathematics Concepts, D. C. Geary, D. B. Berch, R. Ochsendorf, and K. M. Koepke, Eds. Academic Press, 297-325.

BOTELHO, A. F., VARATHARAJ, A., PATIKORN, T., DOHERTY, D., ADJEI, S. A., AND BECK, J. E. 2019. Developing Early Detectors of Student Attrition and Wheel Spinning Using Deep Learning. Journal of IEEE Transactions on Learning Technologies 12, 2, 158-170. doi: https://doi.org/10.1109/TLT.2019.2912162

BOTELHO, A. F., ADJEI, S. A., BAHEL, V., AND BAKER, R. S. 2022. Exploring Relationships Between Temporal Patterns of Affect and Student Learning. In Proceedings of the 30th International Conference on Computers in Education, Iyer, S. et al., Eds., AsiaPacific Society for Computers in Education,139-144.

BRAITHWAITE, D. W., GOLDSTONE, R. L., VAN DER MAAS, H. L. J., AND LANDY, D. H. 2016. Nonformal mechanisms in mathematical cognitive development: The case of arithmetic. Cognition 149, 40–55.

BUTLER, A. C., MARSH, E. J., SLAVINSKY, J. P., AND BARANIUK, R. G. 2014. Integrating cognitive science and technology improves learning in a STEM classroom. Educational Psychology Review 26, 2, 331-340.

BYE, J., LEE, J. E., CHAN, J. Y. C., CLOSSER, A. H., SHAW, S., AND OTTMAR, E. 2022. Perceiving precedence: Order of operations errors are predicted by perception of equivalent expressions [Poster]. Presented at the 2022 Annual Meeting of the American Educational Research Association (AERA).

CARVALHO, P. F. AND GOLDSTONE, R. L. 2014. Putting category learning in order: Category structure and temporal arrangement affect the benefit of interleaved over blocked study. Memory & Cognition 42, 3, 481-495.

CHAN, J. Y.-C., OTTMAR, E., AND LEE, J.-E. 2022a. Slow down to speed up: Longer pause time before solving problems relates to higher strategy efficiency. Learning and Individual Differences 93, 102109. DOI: 10.1016/j.lindif.2021.102109

CHAN J. Y.-C., OTTMAR, E. R., SMITH, H., AND CLOSSER, A. H. 2022b. Variables versus numbers: Effects of symbols and algebraic knowledge on students’ problem-solving strategies. Contemporary Educational Psychology 71, 102114. DOI: 10.1016/j.cedpsych.2022.102114

CHAN, J. Y.-C., SIDNEY, P. G., AND ALIBALI, M. W. 2019. Corresponding color coding facilitates learning of area measurement. In Proceedings of the 41st annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (PME-NA 2019). S. Otten, A. G. Candela, Z. de Araujo, C. Haines, and C. Munter, Eds. North American Chapter of the International Group for the Psychology of Mathematics Education - North American Chapter, 222-223.

CHASE, W. G. AND SIMON, H. A. 1973. Perception in chess. Cognitive Psychology 4, 1, 55-81.

ČECHÁK, J., AND PELÁNEK, R. 2019. Item ordering biases in educational data. In Artificial Intelligence in Education: 20th International Conference, AIED 2019. Isotani, S., Millán, E., Ogan, A., Hastings, R., McLaren, B., and Luckin, R., Eds. Chicago, IL, USA, June 25- 29, Proceedings, Part I 20, Springer International Publishing, 48-58.

CLOSSER, A. H., CHAN, J. Y.-C., AND OTTMAR, E. 2023. Resisting the urge to calculate: The relation between inhibition skills and perceptual cues in arithmetic performance. Quarterly Journal of Experimental Psychology 76, 12, 2690-2703. https://doi.org/10.1177/17470218231156125

CLOSSER, A. H., CHAN, J. Y.-C., SMITH, H., AND OTTMAR, E. R. 2022. Perceptual learning in math: Implications for educational research, practice, and technology. Rapid Community Report Series. Digital Promise and the International Society of the Learning Sciences. https://repository.isls.org//handle/1/7668

CLOSSER, A. H., SALES, A. C., AND BOTELHO, A. F. 2024. Should we account for classrooms? Analyzing online experimental data with student-level randomization. Educational Technology Research and Development. https://doi.org/10.1007/s11423-023-10325-x

EBBINGHAUS, H. 1913. Memory: A contribution to experimental psychology. New York, NY: Teachers College, Columbia University.

EGOROVA, A., NGO, V., LIU, A., MAHONEY, M., MOY, J., AND OTTMAR, E. In press. Math presentation matters: How superfluous brackets and higher order operator position in mathematics can impact problem solving. In press at Mind Brain and Education.

EISENHAUER, J. G. 2003. Regression through the origin. Teaching Statistics 25, 3, 76-80.

FRINGS, C., SCHNEIDER, K. K., AND FOX, E. 2015. The negative priming paradigm: An update and implications for selective attention. Psychonomic Bulletin and Review 22, 6, 1577-1597.

GIBSON, E. J. 1969. Principles of perceptual learning and development. East Norwalk, CT: Appleton-Century-Crofts.

GIBSON, J. J. 1970. The ecological approach to visual perception. Boston: Houghton-Mifflin.

GIVVIN, K. B., MOROZ, V., LOFTUS, W., AND STIGLER, J. W. 2019. Removing opportunities to calculate improves students’ performance on subsequent word problems. Cognitive Research: Principles and Implications 4, 1, 1–13.

GOLDSTONE, R. L., MARGHETIS, T., WEITNAUER, E., OTTMAR, E. R., AND LANDY, D. 2017. Adapting perception, action, and technology for mathematical reasoning. Current Directions in Psychological Science 26, 5, 434-441.

GURUNG, A., BOTELHO, A. F., AND HEFFERNAN, N. T. 2021, April. Examining Student Effort on Help through Response Time Decomposition. In Proceedings of the 11th International Conference on Learning Analytics and Knowledge. ACM. 292-301.

HARRISON, A., SMITH, H., HULSE, T., AND OTTMAR, E. 2020. Spacing out!: Manipulating spatial features in mathematical expressions affects performance. Journal of Numerical Cognition 6, 2, 186-203. https://doi.org/10.5964/jnc.v6i2.243

HIGGINS, E. J., DETTMER, A. M., AND ALBRO, E. R. 2019. Looking back to move forward: a retrospective examination of research at the intersection of cognitive science and education and what it means for the future. Journal of Cognition and Development 20, 2, 278-297.

JIANG, M. J., COOPER, J. L., AND ALIBALI, M. W. 2014. Spatial factors influence arithmetic performance: The case of the minus sign. Quarterly Journal of Experimental Psychology 67, 8, 1626-1642.

KIERAN, C. 1979. Children’s operational thinking within the context of bracketing and the order of operations. In Proceedings of the 3rd Conference of the International Group for the Psychology of Mathematics Education. Tall, D. Ed. Warwick, UK: PME, 128–133.

KOEDINGER, K. R., CARVALHO, P. F., LIU, R., AND MCLAUGHLIN, E. A. 2023. An astonishing regularity in student learning rate. Proceedings of the National Academy of Sciences 120, 13, e2221311120. https://doi.org/10.1073/pnas.2221311120

KOEDINGER, K. R., MCLAUGHLIN, E. A., AND STAMPER, J. C. 2010. Automated Student Model Improvement. In Proceedings of the 5th International Conference on Educational Data Mining. Yacef, K., Zaiane, O., Hershkovitz, A., Yudelson, M., and Stamper, J., Eds. International Educational Data Mining Society, 17–24.

KUMAR, D. 2021. Survey Paper on Information based Learning System using Educational Data Mining. International Journal for Research in Applied Science and Engineering Technology 10, 2, 518-522. https://doi.org/10.22214/ijraset.2021.35431

LANDY, D. AND GOLDSTONE, R. L. 2007. How abstract is symbolic thought?. Journal of Experimental Psychology: Learning, Memory, and Cognition 33, 4, 720-733.

LANDY, D. H. AND GOLDSTONE, R. L. 2010. Proximity and precedence in arithmetic. Quarterly Journal of Experimental Psychology 63, 10, 1953-1968.

LANDY, D. H., JONES, M. N., AND GOLDSTONE, R. L. 2008. How the appearance of an operator affects its formal precedence. In Proceedings of the Thirtieth Annual Conference of the Cognitive Science Society. B. C. Love, K. McRae, and V. M. Sloutsky, Eds. Washington, DC: Cognitive Science Society. 2109–2114

LEE, J. E., CHAN, J. Y. C, BOTELHO, A. F., AND OTTMAR E. R. 2022a. Does slow and steady win the race?: Clustering patterns of students’ behaviors in an interactive online mathematics game. Journal of Educational Technology Research and Development 70, 1575-1599. doi: https://doi.org/10.1007/s11423-022-10138-4

LEE, J. E., HORNBURG, C. B., CHAN, J. Y.-C., AND OTTMAR, E. 2022b. Perceptual and number effects on students’ initial solution strategies in an interactive online mathematics game. Journal of Numerical Cognition 8, 1, 166-182.

LI, N., COHEN, W. W., AND KOEDINGER, K. R. 2013. Problem order implications for learning. International Journal of Artificial Intelligence in Education 23, 71-93.

LINCHEVSKI, L. AND LIVNEH, D. 1999. Structure sense: The relationship between algebraic and numerical contexts. Educational Studies in Mathematics 40, 173-196.

LIU, Q. AND BRAITHWAITE, D. 2023. Affordances of fractions and decimals for arithmetic. Journal of Experimental Psychology: Learning, Memory, and Cognition 49, 9, 1459-1470. Advance online publication. https://doi.org/10.1037/xlm0001161

LOEWENBERG, D. (2003). Mathematical proficiency for all students: Toward a strategic research and development program in mathematics education. Rand Corporation.

LUNNEBORG, C. E. AND TOUSIGNANT, J. P. 1985. Efron's bootstrap with application to the repeated measures design. Multivariate Behavioral Research 20, 2, 161-178.

MARGHETIS, T., LANDY, D., & GOLDSTONE, R. L. (2016). Mastering algebra retrains the visual system to perceive hierarchical structure in equations. Cognitive Research: Principles and Implications, 1, 1-10.

MCNEIL, N. M. 2008. Limitations to teaching children 2+ 2= 4: Typical arithmetic problems can hinder learning of mathematical equivalence. Child Development 79, 5, 1524-1537.

MCNEIL, N. M. AND ALIBALI, M. W. 2005. Why won't you change your mind? Knowledge of operational patterns hinders learning and performance on equations. Child Development 76, 4, 883-899.

MCNEIL, N. M., CHESNEY, D. L., MATTHEWS, P. G., FYFE, E. R., PETERSEN, L. A., DUNWIDDIE, A. E., AND WHEELER, M. C. 2012. It pays to be organized: Organizing arithmetic practice around equivalent values facilitates understanding of math equivalence. Journal of Educational Psychology 104, 4, 1109.

MCNEIL, N. M., GRANDAU, L., KNUTH, E. J., ALIBALI, M. W., STEPHENS, A. C., HATTIKUDUR, S., AND KRILL, D. E. 2006. Middle-school students' understanding of the equal sign: The books they read can't help. Cognition and Instruction 24, 3, 367-385.

NATHAN, M. J. 2023. Disembodied AI and the limits to machine understanding of students’ embodied interactions. Frontiers in Artificial Intelligence 6, 1148227. https://doi.org/10.3389/frai.2023.1148227

NEILL, W. T. 1977. Inhibition and facilitation processes in selective attention. Journal of Experimental Psychology: Human Perception and Performance 3, 444–450.

NEILL, W. T. 1997. Episodic retrieval in negative priming and repetition priming. Journal of Experimental Psychology: Learning, Memory, and Cognition 23, 6, 1291.

NGO, V., PEREZ, L., CLOSSER, A. H., AND OTTMAR, E. 2023. The effects of operand position and superfluous brackets on student performance in math problem-solving. Journal of Numerical Cognition 9, 1, 107-128. https://doi.org/10.5964/jnc.9535

NORTON, S. J. AND COOPER, T. J. 2001. Students’ perceptions of the importance of closure in arithmetic: implications for algebra. In Proceedings of the International Conference of the Mathematics Education into the 21st Century Project, 19-24.

OTTMAR, E. R., LANDY, D., GOLDSTONE, R., AND WEITNAUER, E. 2015. Getting From Here to There!: Testing the effectiveness of an interactive mathematics intervention embedding perceptual learning. In Proceedings of the Thirty-Seventh Annual Conference of the Cognitive Science Society, Pasadena, CA: Cognitive Science Society, 1793–1798.

OSTROW, K. S. AND HEFFERNAN, N. T. 2014. Testing the Effectiveness of the ASSISTments Platform for Learning Mathematics and Mathematics Problem Solving. In Proceedings of the 7th International Conference on Educational Data Mining. Stamper J., Pardos, Z., Mavrikis M., McLaren, B., Eds. London, U.K., International Educational Data Mining Society, 344–347.

PROCTOR, R. W. 1980. The influence of intervening tasks on the spacing effect for frequency judgments. Journal of Experimental Psychology: Human Learning and Memory 6, 3 (1980), 254–266. DOI: 10.1037/0278-7393.6.3.254

RAU, M. A., MASON, B., AND NOWAK, R. 2016. How to Model Implicit Knowledge? Similarity Learning Methods to Assess Perceptions of Visual Representations. In Proceedings of the 9th International Conference on Educational Data Mining. Barnes, T., Chi, M., and Feng, M., Eds. International Educational Data Mining Society, 199-206.

RIVERA, J., AND GARRIGAN, P. 2016. Persistent perceptual grouping effects in the evaluation of simple arithmetic expressions. Memory & Cognition 44, 5, 750-761.

SEN, A., PATEL, P., RAU, M. A., MASON, B., NOWAK, R., ROGERS, T. T., AND ZHU, X. 2018. Machine Beats Human at Sequencing Visuals for Perceptual-Fluency Practice. In Proceedings of the 11th International Conference on Educational Data Mining. Boyer, K., E., and Yudelson, M., Eds. International Educational Data Mining Society, 137-146.

SWAFFORD, J., & KILPATRICK, J. (Eds.). (2002). Helping Children Learn Mathematics. National Academies Press.

SZOKOLSZKY, A., READ, C., PALATINUS, Z., AND PALATINUS, K. 2019. Ecological approaches to perceptual learning: learning to perceive and perceiving as learning. Adaptive Behavior 27, 6, 363-388.

WAGEMANS, J., ELDER, J. H., KUBOVY, M., PALMER, S. E., PETERSON, M. A., SINGH, M., AND VON DER HEYDT, R. 2012. A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization. Psychological Bulletin 138, 2012, 1172–1217.

WERTHEIMER, M. 1938. Gestalt theory. In A source book of Gestalt psychology. Ellis, W. D. Ed. Kegan Paul, Trench, Trubner & Company. 1–11. DOI: 10.1037/11496-001

XINLIN, Z. AND QI, D. 2003. Representation formats for addition and multiplication facts. Acta Psychologica Sinica 35, 3, 345-351.
EDM 2024 Journal Track