A First Step in Learning Analytics: Pre-processing Low-Level Alice Logging Data of Middle School Students



Published Jul 25, 2013
Linda Werner Charlie McDowell Jill Denner


Educational data mining can miss or misidentify key findings about student learning without a transparent process of analyzing the data. This paper describes the first steps in the process of using low-level logging data to understand how middle school students used Alice, an initial programming environment. We describe the steps that were required and the decisions that were made in building a tool to translate the low-level logging data into a form that can be used to investigate educational questions about problem solving strategies for a range of different programming tasks. This work contributes to efforts to analyze educational data, and is important for researchers and tool builders involved with the design of logging systems for other programming environments and software tools.

How to Cite

Werner, L., McDowell, C., & Denner, J. (2013). A First Step in Learning Analytics: Pre-processing Low-Level Alice Logging Data of Middle School Students. JEDM | Journal of Educational Data Mining, 5(2), 11-37. https://doi.org/10.5281/zenodo.3554631
Abstract 481 | PDF Downloads 512



Alice, initial programming environment, problem solving strategies, analysing low-level logging data

ALEXANDER, P. 2003. The Development of Expertise: The Journey from Acclimation to Proficiency. Educational Researcher, 32, 10-14.

AMERSHI, S., and CONATI, C. 2009. Combining Unsupervised and Supervised Classification to Build User Models for Exploratory Learning Environments. The Journal of Educational Data Mining, 1(1), 18-71.

BAKER, R.S.J.D. 2007. Modeling and Understanding Students' Off-task Behavior in Intelligent Tutoring Systems. Proceedings of the SIGCHI conference on Human factors in computing systems (CHI '07). New York, NY: ACM, 1059-1068.

BAKER, R.S.J.D., and YACEF, K. 2009. The State of Educational Data Mining in 2009: A Review and Future Visions. Journal of Educational Data Mining, 1(1), 3-17.



SABELLI, N. 2006. Foundations and Opportunities for an Interdisciplinary Science of Learning. In R.K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences. 19- 34. New York: Cambridge University Press.

BRUCE, C., BUCKINGHAM, L., HYND, J., MCMAHON, C., ROGGENKAMP, M., and STOODLEY, I. 2004. Ways of Experiencing the Act of Learning to Program: A Phenomenographic Study of Introductory Programming Students at University. Journal of Information Technology Education, 3, 143-160.

BURNETT, M., COOK, C., PENDSE, O., ROTHERMEL, G., SUMMET, J., and WALLACE, C. 2003. End-user Software Engineering with Assertions in the Spreadsheet Paradigm. In Proceedings of the 25th International Conference on Software Engineering, 93-103. IEEE Computer Society.

CHOQUET, C., and IKSAL, S. 2007. Modeling Tracks for the Model Driven Reengineering of a TEL System. Journal of Interactive Learning Research, 18(2), 161-184.

CONWAY, M.J. 1997. Alice: Easy-to-Learn 3D Scripting for Novices (Doctoral dissertation, University of Virginia).

DANN, W.P., COOPER, S.P., and ERICSON, B. 2009. Exploring Wonderland: Java Programming Using Alice and Media Computation (1st ed.). Upper Saddle River, NJ: Prentice Hall Press.

DIMITRACOPOULOU, A. 2005. Designing Collaborative Learning Systems: Current Trends & Future Research Agenda. In Proceedings of the 2005 Conference on Computer Support for Collaborative Learning: Learning 2005: The next 10 years! 115-124. International Society of the Learning Sciences.

DISESSA, A.A. 2006. A History of Conceptual Change Research: Threads and Fault Lines. In R.K. SAWYER (Ed.), The Cambridge Handbook of the Learning Sciences, 265-281. New York: Cambridge University Press.

FINCHER, S., and UTTING, I. 2010. Machines for Thinking. Transactions on Computing Education, 10(4), Article 13, 1-7.

HATANO, G. 1988. Social and motivational bases for mathematical understanding. In G.B. SAXE and M. GEARHART (Eds.), Children’s Mathematics, 55–70. San Francisco, CA: Jossey-Bass.

JADUD, M.C. 2005. A 1st Look at Novice Compilation Behavior. Computer Science Education, 15(1), 25-40.

JADUD, M.C., and HENRIKSEN, P. 2009. Flexible, Reusable Tools for Studying Novice Programmers. In Proceedings of the Fifth International Computing Education Research Workshop (ICER '09). New York, NY: ACM, 37-42.

JONASSEN, D. H. 1999. Designing Constructivist Learning Environments. Instructional Design Theories and Models: A New Paradigm of Instructional Theory, 2, 215-239.

KELLEHER, C. 2006. Motivating Programming: Using Storytelling to Make Computer Programming Attractive to More Middle School Girls (Doctoral dissertation). Retrieved from reference http://www.dtic.mil/dtic/ number ADA492489.


LEBER, B., and STAMPER, J. 2010. A Data Repository for the EDM Community: The PSLC DataShop. in C. ROMERO, S. VENTURA, M. PECHENIZKIY, R.S.J.D. BAKER (Eds.) Handbook of Educational Data Mining, 46-53. Boca Raton, FL: CRC Press.

MACFADYEN, L.P., and DAWSON, S. 2010. Mining LMS Data to Develop an “Early Warning System” for Educators: A Proof of Concept. Computers & Education, 54, 588- 599.

MCDOWELL, C., WERNER, L., BULLOCK, H.E, and FERNALD, J. 2006. Pair Programming Improves Student Retention, Confidence, and Program Quality. Communications of the ACM, 49(8), 90-95.

MERTEN, C., and CONATI, C. 2006. Eye-tracking to Model and Adapt to User Metacognition in Intelligent Learning Environments. In Proceedings of the 11th International Conference on Intelligent User Interfaces (IUI '06). New York: NY, 39-46.

PIECH, C., COOPER, S., SAHAMI, M., KOLLER, D., and BLIKSTEIN, P. 2012. Modeling How Students Learn to Program. In Proceedings of the 43st ACM Technical Symposium on Computer Science Education (SIGCSE). ACM, New York, NY, USA.

ROBINS, A., ROUNTREE, J., and ROUNTREE, N. 2003. Learning and teaching programming: A review and discussion. Computer Science Education, 13(2), 137-172.

SMITH, B. 2010, June. Effect of Pair Programming on Middle Schoolers Using Storytelling Alice. Poster session presented at the 13th Annual UCSC Undergraduate Research Symposium, Santa Cruz, CA, USA.

SOLOWAY, E., EHRLICH, K., BONAR, J., and GREENSPAN, J. 1983. What do Novices Know about Programming? In B. SHNEIDERMAN and A. BADRE (Eds.), Directions in Human–Computer Interactions, 27-54 . Norwood, NJ: Ablex.

SOLOWAY, E., and SPOHRER, J. C. (Eds.). 1989. Studying the Novice Programmer. Hillsdale N J: Lawrence Erlbaum.


Alice, Greenfoot, and Scratch -- A Discussion. Transactions on Computing Education, 10(4), 1-11.

WERNER, L., DENNER, J., CAMPE, S., and KAWAMOTO, D.C. 2012. The Fairy Performance Assessment: Measuring Computational Thinking in Middle School. Proceedings of the 43st ACM Technical Symposium on Computer Science Education (SIGCSE). ACM, New York, NY, USA.

WING, J.M. 2011, Spring. Computational Thinking: What and Why? In Link Magazine, 6.0. Retrieved from http://link.cs.cmu.edu/article.php?a=600.