Mining the Dynamics of Student Utility and Strategy Use during Vocabulary Learning
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
May 1, 2013
Philip I. Pavlik Jr
Abstract
This paper describes the development of a dynamical systems model of motivation and metacognition during learning, which explains some of the practically and theoretically important relationships among three studentengagement constructs and performance metrics during learning. In order to better calibrate and understand the model, the model was also fit with additional fixed factor predictors determined from the factor scores from a factor analysis of the pre-survey given to students. This work mined data from computerized adaptive flashcard learning system to create the dynamical systems model. This flashcard practice included pop-up survey questions on the student's experience of recent easiness, strategy use, and usefulness, in addition to the correctness performance data for the practice. The dynamical systems model of this data was then used to simulate various student profiles to predict how they would experience the flashcard system. These simulations show how strategy use in this task is crucial because of the ways it influences performance and perceived usefulness. In the model, this result is shown by a bifurcation for higher and lower strategy use, where the higher strategy use equilibrium is accompanied by performance predictions suggesting learning that is more efficient. In addition, we examined the implications of our data for the flow theory of optimal experience by testing models of this theory and comparing it to a Vygotskian perspective on the results.
How to Cite
Pavlik Jr, P. I. (2013). Mining the Dynamics of Student Utility and Strategy Use during Vocabulary Learning. Journal of Educational Data Mining, 5(1), 39–71. https://doi.org/10.5281/zenodo.3554609
##plugins.themes.bootstrap3.article.details##
Keywords
dynamical system model, language learning, motivation, metacognition, efficacy, utility
References
ALEXANDER, P.A. AND JUDY, J.E. 1988. The Interaction of Domain-Specific and Strategic Knowledge in Academic Performance. Review of Educational Research 58, 375-404.
ANDERSON, J.R., BOTHELL, D., BYRNE, M.D., DOUGLASS, S., LEBIERE, C. AND QIN, Y. 2004. An Integrated Theory of the Mind. Psychological Review 111, 1036- 1060.
ANDERSON, J.R. AND LEBIERE, C. 1998. The atomic components of thought. Lawrence Erlbaum Associates, Mahwah, NJ.
ATKINSON, R.C. 1975. Mnemotechnics in second-language learning. American Psychologist 30, 821-828.
ATKINSON, R.C. AND RAUGH, M.R. 1975. An application of the mnemonic keyword method to the acquisition of a Russian vocabulary. Journal of Experimental Psychology: Human Learning & Memory 1, 126-133.
BAKER, R., CORBETT, A. AND ALEVEN, V. 2008. More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing. In Proceedings of the 9th International Conference on Intelligent Tutoring Systems, B. WOOLF, E. AIMER AND R. NKAMBOU Eds. Springer-Verlag Berlin, Heidelberg, 406-415.
BAKER, R., RODRIGO, M. AND XOLOCOTZIN, U. 2007. The Dynamics of Affective Transitions in Simulation Problem-Solving Environments. In Proceedings of the 2nd international conference on Affective Computing and Intelligent Interaction Springer-Verlag Berlin, Heidelberg, 666-677.
BAKER, R.S.J.D., D'MELLO, S.K., RODRIGO, M.M.T. AND GRAESSER, A.C. 2010. Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive–affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies 68, 223–241.
BANDURA, A. 1997. Self-efficacy: The exercise of control. Worth Publishers.
BOOTS, B. AND GORDON, G.J. 2011. An Online Spectral Learning Algorithm for Partially Observable Nonlinear Dynamical Systems. In Proceedings of the 25th Association for the Advancement of Artificial Intelligence Conference, 293-300.
BOWER, G.H. 1970. Analysis of a mnemonic device. American Scientist 58, 496-510.
CRUTCHER, R.J. AND ERICSSON, K.A. 2000. The role of mediators in memory retrieval as a function of practice: Controlled mediation to direct access. Journal of Experimental Psychology: Learning, Memory, and Cognition 26, 1297-1317.
CSIKSZENTMIHALYI, M. 1990. Flow: The psychology of optimal experience. Harper and Row, New York.
CSIKSZENTMIHALYI, M. 1997. Finding flow: The psychology of engagement with everyday life. New York, NY, US: Basic Books.
D'MELLO, S. AND GRAESSER, A. 2007. Mind and Body: Dialogue and Posture for Affect Detection in Learning Environments. In Proceedings of the 13th International Conference on Artificial Intelligence in Education, R. LUCKIN AND K.R. KOEDINGER Eds. IOS Press, 1563631, 161-168.
D'MELLO, S., TAYLOR, R. AND GRAESSER, A.C. 2007. Monitoring affective trajectories during complex learning. In Proceedings of the 29th Annual Cognitive Science Society, 203–208.
ECCLES, J.S. 2005. Subjective task value and the Eccles et al. model of achievement- related choices. In Handbook of Competence and Motivation, A.J. ELLIOT AND C.S. DWECK Eds. Guilford, New York, 105-121.
GOTTMAN, J., SWANSON, C. AND SWANSON, K. 2002. A general systems theory of marriage: Nonlinear difference equation modeling of marital interaction. Personality and Social Psychology Review 6, 326.
GUASTELLO, S.J., JOHNSON, E.A. AND RIEKE, M.L. 1999. Nonlinear Dynamics of Motivational Flow. Nonlinear Dynamics, Psychology, and Life Sciences 3, 259-273.
HARTLEY, L.R., SPENCER, J. AND WILLIAMSON, J. 1982. Anxiety, diazepam and retrieval from semantic memory. Psychopharmacology 76, 291-293.
HMELO-SILVER, C.E., DUNCAN, R.G. AND CHINN, C.A. 2007. Scaffolding and Achievement in Problem-Based and Inquiry Learning: A Response to Kirschner, Sweller, and Clark (2006). Educational Psychologist 42, 99 - 107.
JUDD, W.A. AND GLASER, R. 1969. Response Latency as a Function of Training Method, Information Level, Acquisition, and Overlearning. Journal of Educational Psychology 60, 1-30.
KARPICKE, J.D. AND ROEDIGER, H.L., III 2008. The critical importance of retrieval for learning. Science 319, 966–968.
LIEBOVITCH, L.S., NAUDOT, V., VALLACHER, R., NOWAK, A., BUIWRZOSINSKA, L. AND COLEMAN, P. 2008. Dynamics of two-actor cooperation-competition conflict models. Physica A: Statistical Mechanics and its Applications 387, 6360-6378.
LUCKIN, R. AND DU BOULAY, B. 1999. Ecolab: The development and evaluation of a vygotskian design framework. International Journal of Artificial Intelligence in Education 10, 198-220.
MILLER, R.B., DEBACKER, T.K. AND GREENE, B.A. 1999. Perceived instrumentality and academics: The link to task valuing. Journal of Instructional Psychology 26, 250-260.
MOSTOW, J., CHANG, K.-M. AND NELSON, J. 2011. Toward Exploiting EEG Input in a Reading Tutor Artificial Intelligence in Education. In Proceedings of the 15th International Conference on Artificial Intelligence in Education, G. BISWAS, S. BULL, J. KAY AND A. MITROVIC Eds. Springer, 230-237.
PAJARES, F. 1996. Self-Efficacy Beliefs in Academic Settings. Review of Educational Research 66, 543-578.
PAVLIK JR., P.I. 2007. Understanding and applying the dynamics of test practice and study practice. Instructional Science 35, 407–441.
PAVLIK JR., P.I. 2010. Data Reduction Methods Applied to Understanding Complex Learning Hypotheses. In Proceedings of the 3rd International Conference on Educational Data Mining, R.S.J.D. BAKER, A. MERCERON AND P.I. PAVLIK JR. Eds., Pittsburgh, 311-312.
PAVLIK JR., P.I. AND ANDERSON, J.R. 2008. Using a model to compute the optimal schedule of practice. Journal of Experimental Psychology: Applied 14, 101–117.
PAVLIK JR., P.I., BOLSTER, T., WU, S., KOEDINGER, K.R. AND MACWHINNEY, B. 2008. Using optimally selected drill practice to train basic facts. In Proceedings of the 9th International Conference on Intelligent Tutoring Systems, B. WOOLF, E. AIMER AND R. NKAMBOU Eds., Montreal, Canada, 593–602.
PAVLIK JR., P.I., PRESSON, N. AND HORA, D. 2008. Using the FaCT System (Fact and Concept Training System) for Classroom and Laboratory Experiments. In Proceedings of the Inter-Science Of Learning Center Conference, Pittsburgh, PA.
PAVLIK JR., P.I. AND WU, S. 2011. A dynamical system model of microgenetic changes in performance, efficacy, strategy use and value during vocabulary learning. In 4th International Conference on Educational Data Mining M. PECHENIZKIY, T. CALDERS, C. CONATI, S. VENTURA, C. ROMERO AND J. STAMPER Eds., Eindhoven, the Netherlands, 277–282.
PINTRICH, P.R., SMITH, D.A., GARCIA, T. AND MCKEACHIE, W.J. 1993. Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement 53, 801-813.
RICKARD, T.C. 1997. Bending the power law: A CMPL theory of strategy shifts and the automatization of cognitive skills. Journal of Experimental Psychology: General 126, 288-311.
RODRÂIGUEZ, M. AND SADOSKI, M. 2000. Effects of rote, context, keyword, and context/keyword methods on retention of vocabulary in EFL classrooms. Language Learning 50, 385-412.
THOMPSON, C.P., WENGER, S.K. AND BARTLING, C.A. 1978. How recall facilitates subsequent recall: A reappraisal. Journal of Experimental Psychology: Human Learning & Memory 4, 210-221.
VALLACHER, R.R. AND NOWAK, A. 2007. Dynamical social psychology: Finding order in the flow of human experience. In Social Psychology: Handbook of Basic Principles, A.W. KRUGLANSKI AND E.T. HIGGINS Eds. Guildford Publications, New York.
VYGOTSKY, L. 1986. Thought and Language. MIT Press, Cambridge, Mass.
VYGOTSKY, L.S. 1978. Mind in society. Harvard University Press, Cambridge.
WANG, A.Y. AND THOMAS, M.H. 1995. Effects of keyword on long-term retention: Help or hindrance? Journal of Educational Psychology 87, 468-475.
WARD, L.M. 2002. Dynamical cognitive science. MIT Press, Cambridge, Mass.
WIECZYNSKI, D.M. AND BLICK, K.A. 1996. Self-referencing versus the keyword method in learning vocabulary words. Psychological Reports 79, 1391-1394.
WITHERSPOON, A., AZEVEDO, R., GREENE, J., MOOS, D. AND BAKER, S. 2007. The Dynamic Nature of Self-Regulatory Behavior in Self-Regulated Learning and Externally-Regulated Learning Episodes. In Proceedings of the 13th InternationalConference on Artificial Intelligence in Education, R. LUCKIN AND K.R. KOEDINGER Eds. IOS Press, 179-186.
WU, S.-M., YU, Y. AND ZHANG, Y. 2006. Chinese Link: Zhongwen Tiandi, Intermediate Chinese. Pearson Education/ Prentice Hall.
ANDERSON, J.R., BOTHELL, D., BYRNE, M.D., DOUGLASS, S., LEBIERE, C. AND QIN, Y. 2004. An Integrated Theory of the Mind. Psychological Review 111, 1036- 1060.
ANDERSON, J.R. AND LEBIERE, C. 1998. The atomic components of thought. Lawrence Erlbaum Associates, Mahwah, NJ.
ATKINSON, R.C. 1975. Mnemotechnics in second-language learning. American Psychologist 30, 821-828.
ATKINSON, R.C. AND RAUGH, M.R. 1975. An application of the mnemonic keyword method to the acquisition of a Russian vocabulary. Journal of Experimental Psychology: Human Learning & Memory 1, 126-133.
BAKER, R., CORBETT, A. AND ALEVEN, V. 2008. More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing. In Proceedings of the 9th International Conference on Intelligent Tutoring Systems, B. WOOLF, E. AIMER AND R. NKAMBOU Eds. Springer-Verlag Berlin, Heidelberg, 406-415.
BAKER, R., RODRIGO, M. AND XOLOCOTZIN, U. 2007. The Dynamics of Affective Transitions in Simulation Problem-Solving Environments. In Proceedings of the 2nd international conference on Affective Computing and Intelligent Interaction Springer-Verlag Berlin, Heidelberg, 666-677.
BAKER, R.S.J.D., D'MELLO, S.K., RODRIGO, M.M.T. AND GRAESSER, A.C. 2010. Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive–affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies 68, 223–241.
BANDURA, A. 1997. Self-efficacy: The exercise of control. Worth Publishers.
BOOTS, B. AND GORDON, G.J. 2011. An Online Spectral Learning Algorithm for Partially Observable Nonlinear Dynamical Systems. In Proceedings of the 25th Association for the Advancement of Artificial Intelligence Conference, 293-300.
BOWER, G.H. 1970. Analysis of a mnemonic device. American Scientist 58, 496-510.
CRUTCHER, R.J. AND ERICSSON, K.A. 2000. The role of mediators in memory retrieval as a function of practice: Controlled mediation to direct access. Journal of Experimental Psychology: Learning, Memory, and Cognition 26, 1297-1317.
CSIKSZENTMIHALYI, M. 1990. Flow: The psychology of optimal experience. Harper and Row, New York.
CSIKSZENTMIHALYI, M. 1997. Finding flow: The psychology of engagement with everyday life. New York, NY, US: Basic Books.
D'MELLO, S. AND GRAESSER, A. 2007. Mind and Body: Dialogue and Posture for Affect Detection in Learning Environments. In Proceedings of the 13th International Conference on Artificial Intelligence in Education, R. LUCKIN AND K.R. KOEDINGER Eds. IOS Press, 1563631, 161-168.
D'MELLO, S., TAYLOR, R. AND GRAESSER, A.C. 2007. Monitoring affective trajectories during complex learning. In Proceedings of the 29th Annual Cognitive Science Society, 203–208.
ECCLES, J.S. 2005. Subjective task value and the Eccles et al. model of achievement- related choices. In Handbook of Competence and Motivation, A.J. ELLIOT AND C.S. DWECK Eds. Guilford, New York, 105-121.
GOTTMAN, J., SWANSON, C. AND SWANSON, K. 2002. A general systems theory of marriage: Nonlinear difference equation modeling of marital interaction. Personality and Social Psychology Review 6, 326.
GUASTELLO, S.J., JOHNSON, E.A. AND RIEKE, M.L. 1999. Nonlinear Dynamics of Motivational Flow. Nonlinear Dynamics, Psychology, and Life Sciences 3, 259-273.
HARTLEY, L.R., SPENCER, J. AND WILLIAMSON, J. 1982. Anxiety, diazepam and retrieval from semantic memory. Psychopharmacology 76, 291-293.
HMELO-SILVER, C.E., DUNCAN, R.G. AND CHINN, C.A. 2007. Scaffolding and Achievement in Problem-Based and Inquiry Learning: A Response to Kirschner, Sweller, and Clark (2006). Educational Psychologist 42, 99 - 107.
JUDD, W.A. AND GLASER, R. 1969. Response Latency as a Function of Training Method, Information Level, Acquisition, and Overlearning. Journal of Educational Psychology 60, 1-30.
KARPICKE, J.D. AND ROEDIGER, H.L., III 2008. The critical importance of retrieval for learning. Science 319, 966–968.
LIEBOVITCH, L.S., NAUDOT, V., VALLACHER, R., NOWAK, A., BUIWRZOSINSKA, L. AND COLEMAN, P. 2008. Dynamics of two-actor cooperation-competition conflict models. Physica A: Statistical Mechanics and its Applications 387, 6360-6378.
LUCKIN, R. AND DU BOULAY, B. 1999. Ecolab: The development and evaluation of a vygotskian design framework. International Journal of Artificial Intelligence in Education 10, 198-220.
MILLER, R.B., DEBACKER, T.K. AND GREENE, B.A. 1999. Perceived instrumentality and academics: The link to task valuing. Journal of Instructional Psychology 26, 250-260.
MOSTOW, J., CHANG, K.-M. AND NELSON, J. 2011. Toward Exploiting EEG Input in a Reading Tutor Artificial Intelligence in Education. In Proceedings of the 15th International Conference on Artificial Intelligence in Education, G. BISWAS, S. BULL, J. KAY AND A. MITROVIC Eds. Springer, 230-237.
PAJARES, F. 1996. Self-Efficacy Beliefs in Academic Settings. Review of Educational Research 66, 543-578.
PAVLIK JR., P.I. 2007. Understanding and applying the dynamics of test practice and study practice. Instructional Science 35, 407–441.
PAVLIK JR., P.I. 2010. Data Reduction Methods Applied to Understanding Complex Learning Hypotheses. In Proceedings of the 3rd International Conference on Educational Data Mining, R.S.J.D. BAKER, A. MERCERON AND P.I. PAVLIK JR. Eds., Pittsburgh, 311-312.
PAVLIK JR., P.I. AND ANDERSON, J.R. 2008. Using a model to compute the optimal schedule of practice. Journal of Experimental Psychology: Applied 14, 101–117.
PAVLIK JR., P.I., BOLSTER, T., WU, S., KOEDINGER, K.R. AND MACWHINNEY, B. 2008. Using optimally selected drill practice to train basic facts. In Proceedings of the 9th International Conference on Intelligent Tutoring Systems, B. WOOLF, E. AIMER AND R. NKAMBOU Eds., Montreal, Canada, 593–602.
PAVLIK JR., P.I., PRESSON, N. AND HORA, D. 2008. Using the FaCT System (Fact and Concept Training System) for Classroom and Laboratory Experiments. In Proceedings of the Inter-Science Of Learning Center Conference, Pittsburgh, PA.
PAVLIK JR., P.I. AND WU, S. 2011. A dynamical system model of microgenetic changes in performance, efficacy, strategy use and value during vocabulary learning. In 4th International Conference on Educational Data Mining M. PECHENIZKIY, T. CALDERS, C. CONATI, S. VENTURA, C. ROMERO AND J. STAMPER Eds., Eindhoven, the Netherlands, 277–282.
PINTRICH, P.R., SMITH, D.A., GARCIA, T. AND MCKEACHIE, W.J. 1993. Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement 53, 801-813.
RICKARD, T.C. 1997. Bending the power law: A CMPL theory of strategy shifts and the automatization of cognitive skills. Journal of Experimental Psychology: General 126, 288-311.
RODRÂIGUEZ, M. AND SADOSKI, M. 2000. Effects of rote, context, keyword, and context/keyword methods on retention of vocabulary in EFL classrooms. Language Learning 50, 385-412.
THOMPSON, C.P., WENGER, S.K. AND BARTLING, C.A. 1978. How recall facilitates subsequent recall: A reappraisal. Journal of Experimental Psychology: Human Learning & Memory 4, 210-221.
VALLACHER, R.R. AND NOWAK, A. 2007. Dynamical social psychology: Finding order in the flow of human experience. In Social Psychology: Handbook of Basic Principles, A.W. KRUGLANSKI AND E.T. HIGGINS Eds. Guildford Publications, New York.
VYGOTSKY, L. 1986. Thought and Language. MIT Press, Cambridge, Mass.
VYGOTSKY, L.S. 1978. Mind in society. Harvard University Press, Cambridge.
WANG, A.Y. AND THOMAS, M.H. 1995. Effects of keyword on long-term retention: Help or hindrance? Journal of Educational Psychology 87, 468-475.
WARD, L.M. 2002. Dynamical cognitive science. MIT Press, Cambridge, Mass.
WIECZYNSKI, D.M. AND BLICK, K.A. 1996. Self-referencing versus the keyword method in learning vocabulary words. Psychological Reports 79, 1391-1394.
WITHERSPOON, A., AZEVEDO, R., GREENE, J., MOOS, D. AND BAKER, S. 2007. The Dynamic Nature of Self-Regulatory Behavior in Self-Regulated Learning and Externally-Regulated Learning Episodes. In Proceedings of the 13th InternationalConference on Artificial Intelligence in Education, R. LUCKIN AND K.R. KOEDINGER Eds. IOS Press, 179-186.
WU, S.-M., YU, Y. AND ZHANG, Y. 2006. Chinese Link: Zhongwen Tiandi, Intermediate Chinese. Pearson Education/ Prentice Hall.
Section
Articles
Authors who publish with this journal agree to the following terms:
- The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
- Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
- The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons 4.0 License (Attribution-Noncommercial-No Derivatives 4.0 International), or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
- Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
- Noncommercial—other users (including Publisher) may not use this Work for commercial purposes;
- No Derivative Works—other users (including Publisher) may not alter, transform, or build upon this Work,with the understanding that any of the above conditions can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
- The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
- Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
- The Author represents and warrants that:
- the Work is the Author’s original work;
- the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
- the Work is not pending review or under consideration by another publisher;
- the Work has not previously been published;
- the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
- the Work contains no libel, invasion of privacy, or other unlawful matter.
- The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.