Mining the Dynamics of Student Utility and Strategy Use during Vocabulary Learning



Published May 1, 2013
Philip I. Pavlik Jr


This paper describes the development of a dynamical systems model of motivation and metacognition during learning, which explains some of the practically and theoretically important relationships among three studentengagement constructs and performance metrics during learning. In order to better calibrate and understand the model, the model was also fit with additional fixed factor predictors determined from the factor scores from a factor analysis of the pre-survey given to students. This work mined data from computerized adaptive flashcard learning system to create the dynamical systems model. This flashcard practice included pop-up survey questions on the student's experience of recent easiness, strategy use, and usefulness, in addition to the correctness performance data for the practice. The dynamical systems model of this data was then used to simulate various student profiles to predict how they would experience the flashcard system. These simulations show how strategy use in this task is crucial because of the ways it influences performance and perceived usefulness. In the model, this result is shown by a bifurcation for higher and lower strategy use, where the higher strategy use equilibrium is accompanied by performance predictions suggesting learning that is more efficient. In addition, we examined the implications of our data for the flow theory of optimal experience by testing models of this theory and comparing it to a Vygotskian perspective on the results.

How to Cite

Pavlik Jr, P. I. (2013). Mining the Dynamics of Student Utility and Strategy Use during Vocabulary Learning. Journal of Educational Data Mining, 5(1), 39–71.
Abstract 407 | PDF Downloads 334



dynamical system model, language learning, motivation, metacognition, efficacy, utility

ALEXANDER, P.A. AND JUDY, J.E. 1988. The Interaction of Domain-Specific and Strategic Knowledge in Academic Performance. Review of Educational Research 58, 375-404.

ANDERSON, J.R., BOTHELL, D., BYRNE, M.D., DOUGLASS, S., LEBIERE, C. AND QIN, Y. 2004. An Integrated Theory of the Mind. Psychological Review 111, 1036- 1060.

ANDERSON, J.R. AND LEBIERE, C. 1998. The atomic components of thought. Lawrence Erlbaum Associates, Mahwah, NJ.

ATKINSON, R.C. 1975. Mnemotechnics in second-language learning. American Psychologist 30, 821-828.

ATKINSON, R.C. AND RAUGH, M.R. 1975. An application of the mnemonic keyword method to the acquisition of a Russian vocabulary. Journal of Experimental Psychology: Human Learning & Memory 1, 126-133.

BAKER, R., CORBETT, A. AND ALEVEN, V. 2008. More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing. In Proceedings of the 9th International Conference on Intelligent Tutoring Systems, B. WOOLF, E. AIMER AND R. NKAMBOU Eds. Springer-Verlag Berlin, Heidelberg, 406-415.

BAKER, R., RODRIGO, M. AND XOLOCOTZIN, U. 2007. The Dynamics of Affective Transitions in Simulation Problem-Solving Environments. In Proceedings of the 2nd international conference on Affective Computing and Intelligent Interaction Springer-Verlag Berlin, Heidelberg, 666-677.

BAKER, R.S.J.D., D'MELLO, S.K., RODRIGO, M.M.T. AND GRAESSER, A.C. 2010. Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive–affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies 68, 223–241.

BANDURA, A. 1997. Self-efficacy: The exercise of control. Worth Publishers.

BOOTS, B. AND GORDON, G.J. 2011. An Online Spectral Learning Algorithm for Partially Observable Nonlinear Dynamical Systems. In Proceedings of the 25th Association for the Advancement of Artificial Intelligence Conference, 293-300.

BOWER, G.H. 1970. Analysis of a mnemonic device. American Scientist 58, 496-510.

CRUTCHER, R.J. AND ERICSSON, K.A. 2000. The role of mediators in memory retrieval as a function of practice: Controlled mediation to direct access. Journal of Experimental Psychology: Learning, Memory, and Cognition 26, 1297-1317.

CSIKSZENTMIHALYI, M. 1990. Flow: The psychology of optimal experience. Harper and Row, New York.

CSIKSZENTMIHALYI, M. 1997. Finding flow: The psychology of engagement with everyday life. New York, NY, US: Basic Books.

D'MELLO, S. AND GRAESSER, A. 2007. Mind and Body: Dialogue and Posture for Affect Detection in Learning Environments. In Proceedings of the 13th International Conference on Artificial Intelligence in Education, R. LUCKIN AND K.R. KOEDINGER Eds. IOS Press, 1563631, 161-168.

D'MELLO, S., TAYLOR, R. AND GRAESSER, A.C. 2007. Monitoring affective trajectories during complex learning. In Proceedings of the 29th Annual Cognitive Science Society, 203–208.

ECCLES, J.S. 2005. Subjective task value and the Eccles et al. model of achievement- related choices. In Handbook of Competence and Motivation, A.J. ELLIOT AND C.S. DWECK Eds. Guilford, New York, 105-121.

GOTTMAN, J., SWANSON, C. AND SWANSON, K. 2002. A general systems theory of marriage: Nonlinear difference equation modeling of marital interaction. Personality and Social Psychology Review 6, 326.

GUASTELLO, S.J., JOHNSON, E.A. AND RIEKE, M.L. 1999. Nonlinear Dynamics of Motivational Flow. Nonlinear Dynamics, Psychology, and Life Sciences 3, 259-273.

HARTLEY, L.R., SPENCER, J. AND WILLIAMSON, J. 1982. Anxiety, diazepam and retrieval from semantic memory. Psychopharmacology 76, 291-293.

HMELO-SILVER, C.E., DUNCAN, R.G. AND CHINN, C.A. 2007. Scaffolding and Achievement in Problem-Based and Inquiry Learning: A Response to Kirschner, Sweller, and Clark (2006). Educational Psychologist 42, 99 - 107.

JUDD, W.A. AND GLASER, R. 1969. Response Latency as a Function of Training Method, Information Level, Acquisition, and Overlearning. Journal of Educational Psychology 60, 1-30.

KARPICKE, J.D. AND ROEDIGER, H.L., III 2008. The critical importance of retrieval for learning. Science 319, 966–968.

LIEBOVITCH, L.S., NAUDOT, V., VALLACHER, R., NOWAK, A., BUIWRZOSINSKA, L. AND COLEMAN, P. 2008. Dynamics of two-actor cooperation-competition conflict models. Physica A: Statistical Mechanics and its Applications 387, 6360-6378.

LUCKIN, R. AND DU BOULAY, B. 1999. Ecolab: The development and evaluation of a vygotskian design framework. International Journal of Artificial Intelligence in Education 10, 198-220.

MILLER, R.B., DEBACKER, T.K. AND GREENE, B.A. 1999. Perceived instrumentality and academics: The link to task valuing. Journal of Instructional Psychology 26, 250-260.

MOSTOW, J., CHANG, K.-M. AND NELSON, J. 2011. Toward Exploiting EEG Input in a Reading Tutor Artificial Intelligence in Education. In Proceedings of the 15th International Conference on Artificial Intelligence in Education, G. BISWAS, S. BULL, J. KAY AND A. MITROVIC Eds. Springer, 230-237.

PAJARES, F. 1996. Self-Efficacy Beliefs in Academic Settings. Review of Educational Research 66, 543-578.

PAVLIK JR., P.I. 2007. Understanding and applying the dynamics of test practice and study practice. Instructional Science 35, 407–441.

PAVLIK JR., P.I. 2010. Data Reduction Methods Applied to Understanding Complex Learning Hypotheses. In Proceedings of the 3rd International Conference on Educational Data Mining, R.S.J.D. BAKER, A. MERCERON AND P.I. PAVLIK JR. Eds., Pittsburgh, 311-312.

PAVLIK JR., P.I. AND ANDERSON, J.R. 2008. Using a model to compute the optimal schedule of practice. Journal of Experimental Psychology: Applied 14, 101–117.

PAVLIK JR., P.I., BOLSTER, T., WU, S., KOEDINGER, K.R. AND MACWHINNEY, B. 2008. Using optimally selected drill practice to train basic facts. In Proceedings of the 9th International Conference on Intelligent Tutoring Systems, B. WOOLF, E. AIMER AND R. NKAMBOU Eds., Montreal, Canada, 593–602.

PAVLIK JR., P.I., PRESSON, N. AND HORA, D. 2008. Using the FaCT System (Fact and Concept Training System) for Classroom and Laboratory Experiments. In Proceedings of the Inter-Science Of Learning Center Conference, Pittsburgh, PA.

PAVLIK JR., P.I. AND WU, S. 2011. A dynamical system model of microgenetic changes in performance, efficacy, strategy use and value during vocabulary learning. In 4th International Conference on Educational Data Mining M. PECHENIZKIY, T. CALDERS, C. CONATI, S. VENTURA, C. ROMERO AND J. STAMPER Eds., Eindhoven, the Netherlands, 277–282.

PINTRICH, P.R., SMITH, D.A., GARCIA, T. AND MCKEACHIE, W.J. 1993. Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement 53, 801-813.

RICKARD, T.C. 1997. Bending the power law: A CMPL theory of strategy shifts and the automatization of cognitive skills. Journal of Experimental Psychology: General 126, 288-311.

RODRÂIGUEZ, M. AND SADOSKI, M. 2000. Effects of rote, context, keyword, and context/keyword methods on retention of vocabulary in EFL classrooms. Language Learning 50, 385-412.

THOMPSON, C.P., WENGER, S.K. AND BARTLING, C.A. 1978. How recall facilitates subsequent recall: A reappraisal. Journal of Experimental Psychology: Human Learning & Memory 4, 210-221.

VALLACHER, R.R. AND NOWAK, A. 2007. Dynamical social psychology: Finding order in the flow of human experience. In Social Psychology: Handbook of Basic Principles, A.W. KRUGLANSKI AND E.T. HIGGINS Eds. Guildford Publications, New York.

VYGOTSKY, L. 1986. Thought and Language. MIT Press, Cambridge, Mass.

VYGOTSKY, L.S. 1978. Mind in society. Harvard University Press, Cambridge.

WANG, A.Y. AND THOMAS, M.H. 1995. Effects of keyword on long-term retention: Help or hindrance? Journal of Educational Psychology 87, 468-475.

WARD, L.M. 2002. Dynamical cognitive science. MIT Press, Cambridge, Mass.

WIECZYNSKI, D.M. AND BLICK, K.A. 1996. Self-referencing versus the keyword method in learning vocabulary words. Psychological Reports 79, 1391-1394.

WITHERSPOON, A., AZEVEDO, R., GREENE, J., MOOS, D. AND BAKER, S. 2007. The Dynamic Nature of Self-Regulatory Behavior in Self-Regulated Learning and Externally-Regulated Learning Episodes. In Proceedings of the 13th InternationalConference on Artificial Intelligence in Education, R. LUCKIN AND K.R. KOEDINGER Eds. IOS Press, 179-186.

WU, S.-M., YU, Y. AND ZHANG, Y. 2006. Chinese Link: Zhongwen Tiandi, Intermediate Chinese. Pearson Education/ Prentice Hall.