Developing Computational Methods to Measure and Track Learners’ Spatial Reasoning in an Open-Ended Simulation
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Abstract
Interactive learning environments can provide learners with opportunities to explore rich, real-world problem spaces, but the nature of these problem spaces can make assessing learner progress difficult. Such assessment can be useful for providing formative and summative feedback to the learners, to educators, and to the designers of the environments. This work adds to a growing body of research that is applying EDM techniques to more open-ended problem spaces. The open-ended problem space under study here was an environmental science simulation. Learners were confronted with the real-world challenge of effectively placing green infrastructure in an urban neighborhood to reduce surface flooding. Learners could try out different 2D spatial arrangements of green infrastructure and use the simulation to test each solution’s impact on flooding. The learners’ solutions and the solutions’ performances were logged during a controlled experiment with different user interface designs for the simulation. As with many open-problem spaces, analyzing this data was difficult due to the large state space, many good solutions, and many alternate paths to those good solutions. This work proposes a procedure for reducing the state space of solutions defined by 2D spatial patterns while maintaining their critical spatial properties. Spatial reasoning problems are a problem class not extensively examined by EDM, so this work sets the stage for further research in this area. This work also details a procedure for discovering effective 2D spatial strategies and solution paths, demonstrates how this information can be used to give formative feedback to the designers of the interactive learning environment, and speculates about how it could be used to provide formative feedback to learners.
How to Cite
##plugins.themes.bootstrap3.article.details##
environmental science simulation, open-ended problem spaces, effective 2D spatial strategies, solution paths, feedback to designers
ANDERSEN, E., YUN-EN LIU, APTER, E., BOUCHER-GENESSE, F., & POPOVIC´, Z. (2010). Gameplay Analysis through State Projection. International Conference on the Foundations of Digital Games. Monterey, California.
ANTLE, A. N., DROUMEVA, M., & HA, D. (2009). Hands on what? Comparing children's mousebased and tangible-based interaction. Proceedings of the 8th International Conference on Interaction Design and Children (pp. 80-88). ACM.
BAKER, R. S., & YACEF, K. (2009). The State of Educational Data Mining in 2009: A Review and Future Visions. Journal of Educational Data Mining , 1 (1), 3-17.
BERLAND, M., BAKER, R. S., & BLIKSTEIN, P. (2014). Educational data mining and learning analytics: Applications to constructionist research. Technology,Knowledge, and Learning , 19 (1-2), 205-220.
BERLAND, M., MARTIN, T., BENTON, T., SMITH, C., & DAVIS, D. (2013). Using Learning Analytics to Understand the Learing Pathwaysof Novice Programmers. Journal of Learing Sciences , 564-599.
BISWAS, G., LORETZ, K. M., & SEGEDY, J. R. (2013). Model-Driven Assessment of Learners in an Open-Ended Learning Environment. Third International Conference on Learning Analytics and Knowledge. New York, NY.
BLIKSTEIN, P. (2011). Using learning analytics to assess students’ behavior in open-ended Programming tasks. Learning Analytics and Knowledge. New York.
BRAVO, M., HERNANDEZ, J., SAORIN, J., & CONTERO, M. (2010). A 3D Educational Mobile Game to Enhance Student's Spatial Skills. IEEE 10th International Conference on Advanced Learning Technologies (ICALT). Sousse,Tunisia: IEEE Computer Society.
CONNELL, M., & STEVENS, D. (2002). A computer-based tutoring system for visual-spatial skills: dynamically adapting to the user's developmental range. Proceedings of the The 2nd International Conference on Development and Learning. Cambridge: IEEE COmputer Society.
DALE, M. (1999). Spatial pattern analysis in plant ecology. Cambridge, UK: Cambridge University Press.
DESMARAIS, M., & LEMIEUX, F. (2013). Clustering and Visualizing Study State Sequences. 6th International Conference on Educational Data Mining. Memphis, Tennessee.
DICERBO, K. E., & KIDWAI, K. (2013). Detecting Player Goals from Game Log Files. 6th International Conference on Educational Data Mining. Memphis, Tennessee.
DIXON, P. M. (1995). Ripley's K function. In Encyclopedia Environmetrics (pp. 1796-1803). NJ: Wiley.
EAGLE, M., & BARNES, T. (2014). Exploring Differences in Problem solving with Data-Driven Approach Maps. Educational Data Mining. Indianapolis.
EKSTROM, R. B., FRENCH, J. W., & HARMON, H. H. (1976). Manual for the Kit of Factor- Referenced Cognitive Tests. ETS.
FALAKMASIR, M. H., PARDOS, Z. A., GORDON, G. J., & BRUSILOVSKY, P. (2013). A Spectral Learning Approach to Knowledge Tracing. 6th International Conference on Educational Data Mining. Memphis, Tennessee.
FALCÃO, T., & PRICE, S. (2009). What have you done! The role of ‘interference’ in tangible environments for supporting collabarative learning. 8th International Conference on Computer Supported Collaborative Learning. Rhodes, Greece.
FORTIN, M. J., DALE, M. R., & HOEF, J. V. (2002). Spatial Analysis in Ecology. In Encyclopedia of Environmetrics (pp. 2051-2058). NJ: Wiley.
FOURNIER-VIGER, P., NKAMBOU, R., NGUIFO, E. M., MAYERS, A., & FAGHIHI, U. (2013). A multiparadigm intelligent tutoring system for robotic arm training. IEEE Transactions on Learning Technologies , 6 (4), 364-377.
FOURNIER-VIGER, P., NKAMBOU, R., NGUIFO, E., MAYERS, A., & FAGHIHI, U. (2013). A Mutilparadigm intelligent tutoring system for robotic arm training. Learning Technologies, IEEE Trasactions , 364-377.
GOBERT, J., SAO PEDRO, M., RAZIUDDIN, J., & BAKER, R. (2013). From Log Files to Assessment Metrics for Science Inquiry Using Educational Data Mining. Journal of the Learning Sciences, 22 (4), 521-563.
HALPERN, D., & COLLEAR, M. (2005). Sex Differences in Visuospatial Abilities. In P. Shah, & A. Miyake, Cambridge Handbook of Visuospatial Thinking (pp. 170-212). New York: Cambridge University Press.
HARPSTEAD, E., MACLELLAN, C. J., KOEDINGER, K. R., ALEVEN, V., DOW, S. P., & MYERS, B. A. (2013). Investigating the Solution Space of an Open-Ended Educational Game Using Conceptual Feature Extraction. 6th International Conference on Educational Data Mining. Memphis, Tennessee.
HEGARTY, M., & WALLER, D. (2005). Individual and age related differences in visuospatial abilities. In P. Shah, & A. Miyake, Cambridge handbook of Visuospatial Thinking (pp. 121- 169). New York: Cambridge University Press.
HUBBARD, C., MENGSHOEL, O., MOON, C., & YONG, S. (1996). Multimedia instructional software for visual reasoning: Visual Reasoning Tutor (VRT). Proceedings of the Third IEEE International Conference on Multimedia Computing and Systems,. Hiroshima, Japan: IEEE Computer Society.
JARUSEK, P., KLUSACEK, M., & PELANEK, R. (2013). Modeling Students’ Learning and Variability of Performance in Problem Solving. 6th International Conference on Educational Data Mining. Memphis, Tennessee.
JOHNSON, M. W., EAGLE, M., & BARNES, T. (2013). InVis: An Interactive Visualization Tool for Exploring Interaction Networks. Educational Data Mining. Memphis.
KARDAN, S., & CONATI, C. (2013). Evaluation of a Data Mining Approach to Providing Adaptive Support in an Open-Ended Learning Environment: A Pilot Study. Artificial Intelligence In Education , 2, 41-48.
KIM, M. J., & MAHER, M. (2008). The impact of tangible user interfaces on spatial cognition during collaborative design. Design Studies , 29 (3), 222-253.
KUTNER, M., NACHTSHEIM, C., & NETER, J. (2004). Applied Linear Regression Models. McGraw Hill.
LAKOFF, G., & JOHNSON, M. (1980). Metaphors We Live By. Chicago: University of Chicago Press.
LEE, S. J., YUN-EN, L., & POPOVIC, Z. (2014). Learning Individual Behavior in an Educational Game: A Data- Driven Approach. Educational data Mining. Indianapolis.
LEVY, S., & WILENSKY, U. (2007). How do I get there...straight, oscillate or inch? High-school students' exploration patterns of Connected Chemistry. 2007 meeting of the American Educational Research Association. Chicago: AERA.
LIU, Y.-E., MANDEL, T., BUTLER, E., ANDERSON, E., O'ROURKE, E., EMMA, B., ZORAN, P. (2013). Predicting Player moves in Educational Game: A Hybrid Approach. Educational Data Mining. Memphis.
LYNCH, C., ASHLEY, K., PINKWART, N., & ALEVEN, V. (2008). Argument graph classification with Genetic programming. Educational Data Mining. Montreal, Quebec.
LYONS, L., DASGUPTA, C., SHELLEY, T., SLATTERY, B., MINOR, E., & ZELLNER, M. (2012). Parsing Patterns: Developing Metrics to Characterize Spatial Problem Solving Strategies within an Environmental Science Simulation. AREA, (p. 19).
MAIMON, O., & ROKACH, L. (2010). Data Mining and Knowledge Discovery Handbook. Springer.
MARSHALL, P. (2007). Do tangible interfaces enhance learning? Proceedings of the 1st international conference on Tangible and embedded interaction (TEI '07) (pp. 163-170). ACM.
MARTINEZ-MALDONADO, R., YACEF, K., & KAY, J. (2013). Data Mining in the Classroom: Discovering Groups Strategies at a Multi-tabletop Environment . 6th International Conference on Educational Data Mining. Memphis,Tennessee.
MASSEY, D., ZELLNER, M. L., COTNER, L., MINOR, E., & GONZALEZ-MELER, M. (2010). Landscape Green Infrastructure Design Model (L-GrID) User’s Manual. Report and software to the Illinois Environmental Protection Agency. Chicago, IL: University of Illinois at Chicago. Chicago.
MENGSHOEL, O., CHAUHAN, S., & YONG, S. (1996). Intelligent critiquing and tutoring of spatial reasoning skills. Artificial Intelligence for Engineering, Design, Analysis and Manufacturing , 235-249.
MINOR, E. S., & URBAN, D. L. (2008). A GraphıTheory Framework for Evaluating Landscape Connectivity and Conservation Planning. Conservation Biology , 22 (2), 297-307.
MULLER, J., KRETZSCHMAR, A., & GREIFF, S. (2013). Exploring Exploration: Inquiries into Exploration Behavior in Complex Problem Solving. 6th International Conference on Educational Data Mining. Memphis, Tennessee.
NESBITT, K., SUTTON, K., WILSON, J., & HOOKHAM, G. (2009). Improving player spatial abilities for 3D challenges. The 6th Australasian Conference on Interactive Entertainment.
Sidney,Australia. PROTÁZIO, J., PEREIRA, W., & ELAYNE JESUS DE CASTRO, F. (1999). Explicit Formulas for an Area Based Edge Effect Correction Method and their Application to Ripley’s K-Function. Journal of Vegetation Science , 10 (3), 433-438.
RAFFERTY, A. N., DAVENPORT, J., & BRUNSKILL, E. (2013). Estimating Student Knowledge from Paired Interaction Data. 6th International Conference on Educational Data Mining, (pp. 260- 263). Memphis, Tennessee.
ROCK, I., & BROSGOLE, L. (1964). Grouping based on phenomenal proximity. Journal of Experimental Psychology , 67 (6), 531-538.
ROMERO, C., & VENTURA, S. (2010). Educational data mining: a review of the state of the art.
Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on , 40 (6), 601-618.
SCHNEIDER, B., JERMANN, P., ZUFFEREY, G., & DILLENBOURG, P. (2011). Benefits of a Tangible Interface for Collaborative Learning and Interaction. IEEE Transactions on Learning Technologies , 4 (3), 222-232.
SCHWEINGRUBER, H., KELLER, T., & QUINN, H. (2012). A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas. National Academies Press.
SHELLEY, T., LYONS, L., MINOR, E., & ZELLNER, M. (2011). Evaluating the embodiment benefits of a paper- based TUI for educational simulations. 29th International Conference on Human Factors in Computing Systems. New York, NY.
SHELLEY, T., LYONS, L., SHI, J., MINOR, E., & ZELLNER, M. (2010). Paper to parameters: designing tangible simulation input. 12th ACM International Conference adjunct papers on Ubiquitous computing. New York, NY.
SHUTE, V. J. (2011). Stealth Assessment in computer based games to support learning. In Computer Games and Instruction (pp. 503-524). Charlotte, NC: Information Age Publishers.
SINGER, M., RADINSKY, J., & GOLDMAN, S. R. (2008). The Role of Gesture in Meaning Construction. Discourse Processes: A Multidisciplinary Journal , 45 (4-5), 365-386.
SISWONO, T. Y. (2008). Promoting creativity in learning mathematics using open-ended problems. The 3rd International Conference on Mathematics and Statistics (ICoMS-3). Indonesia.
SMITH, A., WIEBE, E., MOTT, B., & LESTER, J. (2014). SKETCHMINER: Mining Learner- Generated Science Drawings with Topological Abstraction. Educational Data Mining. Indianapolis.
STIEFF, M., DIXON, B., KUMI, B., & HEGARTY, M. (2014). Strategy Training Eliminates Sex Differences in Spatial Problem Solving in a STEM Domain. Journal of Educational Psychology, 106 (2), 390-402.
UTTAL, D., MEADOW, N., TIPTON, E., HAND , L., ALDEN, A., WARREN, C., NEWCOMBE, N. S (2013). The malleability of spatial skills: A meta-analysis of training studies. Psychological Bulletin , 352-402. VAN LEHN, K. (2011). The relative effectiveness of Human Tutoring, Intelligent Tutoring Systems and other Tutoring System. Educational Psychologist , 197-221.
VYGOTSKY, L. (1978). Mind in Society: The development of higher mental processes. Cambridge, MA: Harvard University Press.
WAI, J., LUBINSKI, D., & BENBOW, C. (2009). Spatial ability for STEMdomains: Aligning over 50 years of cumulative psychological knowledge solidifies its importance. Journal of Educational Psychology , 817-835.
WANG, E., & KIM, Y. (2005). Intelligent Visual Reasoning Tutor. Proceedings of the Fifth IEEE International Conference on Advanced Learning Technologies (pp. 511-515). Washington,D.C: IEEE Computer Society.
WERTHEIMER, M. (1923). Untersuchungen zur Lehre von der Gestalt II. Psycologische Forschung, 4, 301-350.
WIEDERRECHT, M. A., & ULINSKI, A. C. (2012, January). Developmentally appropriate intelligent spatial tutoring for mobile devices. Intelligent Tutoring Systems , 594-596.
WILENSKY, U. J. (1993). Connected Mathematics-Building Concrete Relationship with Mathematical Knowledge. Dissertation of Doctor of Philosophy, Massachusetts Institute of Technology.
ZELLNER, M. L. (2008). Embracing Complexity and Uncertainty: The Potential of Agent-Based Modeling for Environmental Planning and Policy. Planning Theory and Practice , 9 (4), 237-457.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:
- The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
- Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
- The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons 4.0 License (Attribution-Noncommercial-No Derivatives 4.0 International), or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
- Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
- Noncommercial—other users (including Publisher) may not use this Work for commercial purposes;
- No Derivative Works—other users (including Publisher) may not alter, transform, or build upon this Work,with the understanding that any of the above conditions can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
- The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
- Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
- The Author represents and warrants that:
- the Work is the Author’s original work;
- the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
- the Work is not pending review or under consideration by another publisher;
- the Work has not previously been published;
- the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
- the Work contains no libel, invasion of privacy, or other unlawful matter.
- The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.