Evaluating the Impact of Instructional Support Using Data Mining and Process Mining: A Micro-Level Analysis of the Effectiveness of Metacognitive Prompts
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Dec 28, 2016
Christoph Sonnenberg
Maria Bannert
Abstract
In computer-supported learning environments, the deployment of self-regulatory skills represents an essential prerequisite for successful learning. Metacognitive prompts are a promising type of instructional support to activate students' strategic learning activities. However, despite positive effects in previous studies, there are still a large number of students who do not benefit from provided support. Therefore, it may be necessary to consider explicitly the conditions under which a prompt is beneficial for a student, i.e., so-called adaptive scaffolding. The current study aims to (i) classify the effectiveness of prompts on regulatory behavior, (ii) investigate the correspondence of the classification with learning outcome, and (iii) discover the conditions under which prompts induce regulatory activities (i.e., the proper temporal positioning of prompts). The think-aloud data of an experiment in which metacognitive prompts supported the experimental group (n = 35) was used to distinguish between effective and non-effective prompts. Students' activities preceding the prompt presentation were analyzed using data mining and process mining techniques. The results indicate that approximately half of the presented prompts induced metacognitive learning activities as expected. Moreover, the number of induced monitoring activities correlates positively with transfer performance. Finally, the occurrence of orientation and monitoring activities, which are not well-embedded in the course of learning, increases the effectiveness of a presented prompt. In general, our findings demonstrate the benefits of investigating metacognitive support using process data, which can provide implications for the design of effective instructional support.
How to Cite
Sonnenberg, C., & Bannert, M. (2016). Evaluating the Impact of Instructional Support Using Data Mining and Process Mining: A Micro-Level Analysis of the Effectiveness of Metacognitive Prompts. Journal of Educational Data Mining, 8(2), 51–83. https://doi.org/10.5281/zenodo.3554597
##plugins.themes.bootstrap3.article.details##
Keywords
self-regulated learning, instructional support, micro-level analysis, metacognitive prompting, think-aloud data, process mining
References
ALEVEN, V. (2013). Help seeking and intelligent tutoring systems: Theoretical perspectives and a step towards theoretical integration. In R. Azevedo & V. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 311–335). New York: Springer. doi:10.1007/978-1- 4419-5546-3 21
ALEVEN, V., ROLL, I., MCLAREN, B. M., & KOEDINGER, K. R. (2016). Help helps, but only so much: Research on help seeking with intelligent tutoring systems. International Journal of Artificial Intelligence in Education, 26(1), 205–223. doi:10.1007/s40593-015-0089-1
AZEVEDO, R. (2005). Using hypermedia as a metacognitive tool for enhancing student learning? The Role of self-regulated learning. Educational Psychologist, 40(4), 199–209. doi:10.1207/s15326985ep4004 2
AZEVEDO, R. (2009). Theoretical, conceptual, methodological, and instructional issues in research on metacognition and self-regulated learning: A discussion. Metacognition and Learning, 4, 87–95. doi:10.1007/s11409-009-9035-7
AZEVEDO, R. (2014). Issues in dealing with sequential and temporal characteristics of self- and sociallyregulated learning. Metacognition and Learning, 9, 217–228. doi:10.1007/s11409-014-9123-1
AZEVEDO, R., CROMLEY, J. G., MOOS, D. C., GREENE, J. A., & WINTERS, F. I. (2011). Adaptive content and process scaffolding: A key to facilitating students' self-regulated learning with hypermedia. Psychological Testing and Assessment Modeling, 53(1), 106–140.
AZEVEDO, R., CROMLEY, J. G., & SEIBERT, D. (2004). Does adaptive scaffolding facilitate students' ability to regulate their learning with hypermedia? Contemporary Educational Psychology, 29, 344– 370. doi:10.1016/j.cedpsych.2003.09.002
AZEVEDO, R., CROMLEY, J. G., WINTERS, F. I., MOOS, D. C., & GREENE, J. A. (2005). Adaptive human scaffolding facilitates adolescents' self-regulated learning with hypermedia. Instructional Science, 33, 381–412. doi:10.1007/s11251-005-1273-8
AZEVEDO, R., & HADWIN, A. F. (2005). Scaffolding self-regulated learning and metacognition - Implications for the design of computer-based scaffolds. Instructional Science, 33, 367–379. doi:10.1007/s11251-005-1272-9
AZEVEDO, R., LANDIS, R. S., FEYZI-BEHNAGH, R., DUFFY, M., TREVORS, G., HARLEY, J. M., ... HOSSAIN, G. (2012). The effectiveness of pedagogical agents' prompting and feedback in facilitating co-adapted learning with MetaTutor. Lecture Notes in Computer Science, 212–221. doi:10.1007/978- 3-642-30950-2 27
BANNERT, M. (2006). Effects of reflection prompts when learning with hypermedia. Journal of Educational Computing Research, 35(4), 359–375. doi:10.2190/94V6-R58H-3367-G388
BANNERT, M. (2007). Metakognition beim Lernen mit Hypermedia. Erfassung, Beschreibung und Vermittlung wirksamer metakognitiver Lernstrategien und Regulationsaktivitaten [Metacognition and hypermedia learning]. Munster: Waxmann.
BANNERT, M. (2009). Promoting self-regulated learning through prompts: A discussion. Zeitschrift furPadagogische Psychologie, 23(2), 139–145. doi:10.1024/1010-0652.23.2.139
BANNERT, M., & MENGELKAMP, C. (2013). Scaffolding hypermedia learning through metacognitive prompts. In R. Azevedo & V. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 171–186). New York: Springer. doi:10.1007/978-1-4419-5546-3 12
BANNERT, M., & REIMANN, P. (2012). Supporting self-regulated hypermedia learning through prompts. Instructional Science, 40(1), 193–211. doi:10.1007/s11251-011-9167-4
BANNERT, M., REIMANN, P., & SONNENBERG, C. (2014). Process mining techniques for analysing patterns and strategies in students' self-regulated learning. Metacognition and Learning, 9(2), 161– 185. doi:10.1007/s11409-013-9107-6
BANNERT, M., SONNENBERG, C., MENGELKAMP, C., & PIEGER, E. (2015). Short- and long-term effects of students' self-directed metacognitive prompts on navigation behavior and learning performance. Computers in Human Behavior, 52, 293–306. doi:10.1016/j.chb.2015.05.038
BEN-ELIYAHU, A., & BERNACKI, M. L. (2015). Addressing complexities in self-regulated learning: A focus on contextual factors, contingencies, and dynamic relations. Metacognition and Learning, 10, 1–13. doi:10.1007/s11409-015-9134-6
BENJAMINI, Y., & HOCHBERG, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 57(1), 289-300.
BLOOM, B. S. (1956). Taxonomy of educational objectives: The classification of educational goals. New York: D. McKay.
BOUCHET, F., HARLEY, J. M., TREVORS, G. J., & AZEVEDO, R. (2013). Clustering and profiling students according to their interactions with an intelligent tutoring system fostering self-regulated learning. Journal of Educational Data Mining, 5(1), 104–146.
BRUSILOVSKY, P. (2001). Adaptive hypermedia. User Modeling and User-Adapted Interaction, 11, 87– 110. doi:10.1023/A:1011143116306
BRUSILOVSKY, P. (2007). Adaptive navigation support. In P. Brusilovsky, A. Kobsa, & W. Nejdl (Eds.), The adaptive web. Methods and strategies of web personalization (pp. 263–290). Berlin Heidelberg: Springer. doi:10.1007/978-3-540-72079-9
CHI, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. Journal of the Learning Sciences, 6(3), 271–315. doi:10.1207/s15327809jls0603 1
CLAREBOUT, G., & ELEN, J. (2006). Tool use in computer-based learning environments: Towards a research framework. Computers in Human Behavior, 22, 389–411. doi:10.1016/j.chb.2004.09.007
CLAREBOUT, G., ELEN, J., COLLAZO, N. A. J., LUST, G., & JIANG, L. (2013). Metacognition and the use of tools. In R. Azevedo & V. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 187–195). Springer Science. doi: 10.1007/978-1-4419-5546-3 13
COCEA, M., & WEIBELZAHL, S. (2009). Log file analysis for disengagement detection in e-Learning environments. User Modelling and User-Adapted Interaction, 19, 341–385. doi:10.1007/s11257-009- 9065-5
DENT, A. L., & HOYLE, R. H. (2015). A framework for evaluating and enhancing alignment in selfregulated learning research. Metacognition and Learning, 10, 165–179. doi:10.1007/s11409-015- 9136-4.
ERICSSON, K. A., & SIMON, H. A. (1993). Protocol analysis: Verbal reports as data. Cambridge, Mass.: MIT Press.
GE, X. (2013). Designing learning technologies to support self-regulation during III-structured problemsolving processes. In R. Azevedo & V. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 213–228). New York: Springer. doi:10.1007/978-1-4419-5546-3 15
GREENE, J. A., DELLINGER, K. R., TUYS UZOGLU, B. B., & COSTA, L.-J. (2013). A two-tiered approach to analyzing self-regulated learning data to inform the design of hypermedia learning environments. In R. Azevedo & V. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 117–128). New York: Springer. doi:10.1007/978-1-4419-5546-3 8
GUNTHER, C. W., & VAN DER AALST, W. M. P. (2007). Fuzzy mining – Adaptive process simplification based on multi-perspective metrics. Business Process Management - Lecture Notes in Computer Science, 4714, 328–343. doi:10.1007/978-3-540-75183-0
HAMALAINEN, W., & VINNI, M. (2010). Classifiers for educational data mining. In C. Romero, S. Ventura, M. Pechenizkiy, & R. Baker (Eds.), Handbook of Educational Data Mining (pp. 57–74). Boca Raton, FL: Chapman&Hall/CRC.
HATTIE, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York: Routledge.
JEONG, H., GUPTA, A., ROSCOE, R., WAGSTER, J., BISWAS, G., & SCHWARTZ, D. (2008). Using hidden markov models to characterize student behaviors in learning-by-teaching environments. Lecture Notes in Computer Science, 5091, 614–625. doi:10.1007/978-3-540-69132-7 64
JOHNSON, A. M., AZEVEDO, R., & D'MELLO, S. K. (2011). The temporal and dynamic nature of self-regulatory processes during independent and externally assisted hypermedia learning. Cognition and Instruction, 29(4), 471–504. doi:10.1080/07370008.2011.610244
KINNEBREW, J. S., SEGEDY, J. R., & BISWAS, G. (2014). Analyzing the temporal evolution of students' behaviors in open-ended learning environments. Metacognition and Learning, 9, 187–215. doi:10.1007/s11409-014-9112-4
KRAMARSKI, B., & GUTMAN, M. (2006). How can self-regulated learning be supported in mathematical e-learning environments? Journal of Computer Assisted Learning, 22, 24–33. doi:10.1111/j.1365- 2729.2006.00157.x
LAJOIE, S. P., NAISMITH, L., POITRAS, E., HONG, Y.-J., CRUZ-PANESSO, I., RANELLUCCI, J., ... WISEMAN, J. (2013). Technology-rich tools to support self-regulated learning and performance in medicine. In R. Azevedo & V. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 229–242). New York: Springer. doi:10.1007/978-1-4419-5546-3 16
LEHMANN, T., HAHNLEIN, I., & IFENTHALER, D. (2014). Cognitive, metacognitive and motivational perspectives on preflection in self-regulated online learning. Computers in Human Behavior, 32, 313– 323. doi:10.1016/j.chb.2013.07.051
LESTER, J. C., MOTT, B. W., ROBISON, J. L., ROWE, J. P., & SHORES, L. R. (2013). Supporting selfregulated science learning in narrative-centered learning environments. In R. Azevedo & V. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 471–483). New York: Springer. doi:10.1007/978-1-4419-5546-3 30
LIN, X., HMELO, C., KINZER, C. K., & SECULES, T. J. (1999). Designing technology to support reflection. Educational Technology Research and Development, 47(3), 43–62. doi:10.1007/BF02299633
MALMBERG, J., JARVEL A, S., JARVENOJA, H., & PANADERO, E. (2015). Promoting socially shared regulation of learning in CSCL: Progress of socially shared regulation among high- and lowperforming groups. Computers in Human Behavior, 52, 562–572. doi:10.1016/j.chb.2015.03.082
MARTIN, T., & SHERIN, B. (2013). Learning analytics and computational techniques for detecting and evaluating patterns in learning: An introduction to the special issue. Journal of the Learning Sciences, 22(4), 511–520. doi:10.1080/10508406.2013.840466
MOLENAAR, I., & CHIU, M. M. (2014). Dissecting sequences of regulation and cognition: Statistical discourse analysis of primary school children's collaborative learning. Metacognition and Learning, 9(2), 137–160. doi:10.1007/s11409-013-9105-8
MOLENAAR, I., & JARVEL A, S. (2014). Sequential and temporal characteristics of self and socially regulated learning. Metacognition and Learning, 9(2), 75–85. doi:10.1007/s11409-014-9114-2
MOLENAAR, I., & RODA, C. (2008). Attention management for dynamic and adaptive scaffolding. Pragmatics & Cognition, 16(2), 224-271. doi: 10.1075/pc.16.2.04mol
REIMANN, P., MARKAUSKAITE, L., & BANNERT, M. (2014). E-Research and learning theory: What do sequence and process mining methods contribute? British Journal of Educational Technology, 45(3), 528–540. doi:10.1111/bjet.12146
REIMANN, P., & YACEF, K. (2013). Using process mining for understanding learning. In R. Luckin, P. Puntambekar, P. Goodyear, B. Grabowski, J. Underwood, & N. Winters (Eds.), Handbook of Design in Educational Technology (pp. 472–481). New York: Routledge.
ROLL, I., ALEVEN, V., MCLAREN, B. M., & KOEDINGER, K. R. (2011). Improving students' helpseeking skills using metacognitive feedback in an intelligent tutoring system. Learning and Instruction, 21(2), 267–280. doi:10.1016/j.learninstruc.2010.07.004
SCHOOR, C., & BANNERT, M. (2012). Exploring regulatory processes during a computer-supported collaborative learning task using process mining. Computers in Human Behavior, 28, 1321–1331. doi:10.1016/j.chb.2012.02.016
SCHWONKE, R., HAUSER, S., NUCKLES, M., & RENKL, A. (2006). Enhancing computer-supported writing of learning protocols by adaptive prompts. Computers in Human Behavior, 22, 77–92. doi:10.1016/j.chb.2005.01.002
SITZMANN, T., BELL, B. S., KRAIGER, K., & KANAR, A. M. (2009). A multilevel analysis of the effect of prompting self-regulation in technology-delivered instruction. Personnel Psychology, 62, 697–734. doi:10.1111/j.1744-6570.2009.01155.x
SONG, M., & VAN DER AALST, W.M.P. (2007). Supporting process mining by showing events at a glance. In K. Chari & A. Kumar (Eds.), Proceedings of 17th Annual Workshop on Information Technologies and Systems (WITS 2007) (pp. 139–145). Montreal, Canada.
SONNENBERG, C., & BANNERT, M. (2015). Discovering the effects of metacognitive prompts on the sequential structure of SRL-processes using process mining techniques. Journal of Learning Analytics, 2(1), 72–100.
THILLMANN, H., KUNSTING, J., WIRTH, J., & LEUTNER, D. (2009). Is it merely a question of "what" to prompt or also "when" to prompt? Zeitschrift fur P adagogische, 23(2), 105–115. doi:10.1024/1010-0652.23.2.105
TRCKA, N., PECHENIZKIY, M., & VAN DER AALST, W. M. P. (2010). Process mining from educational data. In C. Romero, S. Ventura, M. Pechenizkiy, & R. Baker (Eds.), Handbook of Educational Data Mining (pp. 123–142). Boca Raton, FL: Chapman&Hall/CRC.
VAN DER AALST, W. (2011). Process mining: Discovery, conformance and enhancement of business processes. Berlin, Heidelberg: Springer.
VAN DER AALST, W., WEIJTERS, T., & MARUSTER, L. (2004). Workflow mining: Discovering process models from event logs. IEEE Transactions on Knowledge and Data Engineering, 16(9), 1128–1142. doi:10.1109/TKDE.2004.47
VEENMAN, M. (1993). Metacognitive ability and metacognitive skill: Determinants of discovery learning in computerized learning environments. University of Amsterdam.
VEENMAN, M. V. J. (2007). The assessment and instruction of self-regulation in computer-based environments: A discussion. Metacognition and Learning, 2(2-3), 177–183. doi:10.1007/s11409-007- 9017-6
WALKER, E., RUMMEL, N., & KOEDINGER, K. (2011). Designing automated adaptive support to improve student helping behaviors in a peer tutoring activity. International Journal of Computer Supported Collaborative Learning, 6(2), 279-306. doi: 10.1007/s11412-011-9111-2
WINNE, P. H. (1996). A metacognitive view of individual differences in self-regulated learning. Learning and Individual Differences, 8(4), 327–353.
WINNE, P. H. (2014). Issues in researching self-regulated learning as patterns of events. Metacognition and Learning, 9, 229–237. doi:10.1007/s11409-014-9113-3
WINNE, P. H., & BAKER, R. S. (2013). The potentials of educational data mining for researching metacognition, motivation and self-regulated learning. Journal of Educational Data Mining, 5(1), 1–8.
WINNE, P. H., & HADWIN, A. F. (2008). The weave of motivation and self-regulated learning. In D. Schunk & B. J. Zimmerman (Eds.), Motivation and self-regulated learning: Theory, research, and applications (pp. 297–314). NY: Taylor & Francis.
WIRTH, J. (2009). Promoting self-regulated learning through prompts. Zeitschrift fur P adagogische Psy
chologie, 23(2), 91–94. doi:10.1024/1010-0652.23.2.91
WITTWER, J., & RENKL, A. (2008). Why instructional explanations often do not work: A framework for understanding the effectiveness of instructional explanations. Educational Psychologist, 43, 49–64. doi:10.1080/00461520701756420
YEH, Y. F., CHEN, M. C., HUNG, P. H., & HWANG, G. J. (2010). Optimal self-explanation prompt design in dynamic multi-representational learning environments. Computers and Education, 54, 1089– 1100. doi:10.1016/j.compedu.2009.10.013
ZIMMERMAN, B. J. (2008). Investigating self-regulation and motivation: Historical background, methodological developments, and future prospects. American Educational Research Journal, 45(1), 166–183. doi:10.3102/0002831207312909
ALEVEN, V., ROLL, I., MCLAREN, B. M., & KOEDINGER, K. R. (2016). Help helps, but only so much: Research on help seeking with intelligent tutoring systems. International Journal of Artificial Intelligence in Education, 26(1), 205–223. doi:10.1007/s40593-015-0089-1
AZEVEDO, R. (2005). Using hypermedia as a metacognitive tool for enhancing student learning? The Role of self-regulated learning. Educational Psychologist, 40(4), 199–209. doi:10.1207/s15326985ep4004 2
AZEVEDO, R. (2009). Theoretical, conceptual, methodological, and instructional issues in research on metacognition and self-regulated learning: A discussion. Metacognition and Learning, 4, 87–95. doi:10.1007/s11409-009-9035-7
AZEVEDO, R. (2014). Issues in dealing with sequential and temporal characteristics of self- and sociallyregulated learning. Metacognition and Learning, 9, 217–228. doi:10.1007/s11409-014-9123-1
AZEVEDO, R., CROMLEY, J. G., MOOS, D. C., GREENE, J. A., & WINTERS, F. I. (2011). Adaptive content and process scaffolding: A key to facilitating students' self-regulated learning with hypermedia. Psychological Testing and Assessment Modeling, 53(1), 106–140.
AZEVEDO, R., CROMLEY, J. G., & SEIBERT, D. (2004). Does adaptive scaffolding facilitate students' ability to regulate their learning with hypermedia? Contemporary Educational Psychology, 29, 344– 370. doi:10.1016/j.cedpsych.2003.09.002
AZEVEDO, R., CROMLEY, J. G., WINTERS, F. I., MOOS, D. C., & GREENE, J. A. (2005). Adaptive human scaffolding facilitates adolescents' self-regulated learning with hypermedia. Instructional Science, 33, 381–412. doi:10.1007/s11251-005-1273-8
AZEVEDO, R., & HADWIN, A. F. (2005). Scaffolding self-regulated learning and metacognition - Implications for the design of computer-based scaffolds. Instructional Science, 33, 367–379. doi:10.1007/s11251-005-1272-9
AZEVEDO, R., LANDIS, R. S., FEYZI-BEHNAGH, R., DUFFY, M., TREVORS, G., HARLEY, J. M., ... HOSSAIN, G. (2012). The effectiveness of pedagogical agents' prompting and feedback in facilitating co-adapted learning with MetaTutor. Lecture Notes in Computer Science, 212–221. doi:10.1007/978- 3-642-30950-2 27
BANNERT, M. (2006). Effects of reflection prompts when learning with hypermedia. Journal of Educational Computing Research, 35(4), 359–375. doi:10.2190/94V6-R58H-3367-G388
BANNERT, M. (2007). Metakognition beim Lernen mit Hypermedia. Erfassung, Beschreibung und Vermittlung wirksamer metakognitiver Lernstrategien und Regulationsaktivitaten [Metacognition and hypermedia learning]. Munster: Waxmann.
BANNERT, M. (2009). Promoting self-regulated learning through prompts: A discussion. Zeitschrift furPadagogische Psychologie, 23(2), 139–145. doi:10.1024/1010-0652.23.2.139
BANNERT, M., & MENGELKAMP, C. (2013). Scaffolding hypermedia learning through metacognitive prompts. In R. Azevedo & V. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 171–186). New York: Springer. doi:10.1007/978-1-4419-5546-3 12
BANNERT, M., & REIMANN, P. (2012). Supporting self-regulated hypermedia learning through prompts. Instructional Science, 40(1), 193–211. doi:10.1007/s11251-011-9167-4
BANNERT, M., REIMANN, P., & SONNENBERG, C. (2014). Process mining techniques for analysing patterns and strategies in students' self-regulated learning. Metacognition and Learning, 9(2), 161– 185. doi:10.1007/s11409-013-9107-6
BANNERT, M., SONNENBERG, C., MENGELKAMP, C., & PIEGER, E. (2015). Short- and long-term effects of students' self-directed metacognitive prompts on navigation behavior and learning performance. Computers in Human Behavior, 52, 293–306. doi:10.1016/j.chb.2015.05.038
BEN-ELIYAHU, A., & BERNACKI, M. L. (2015). Addressing complexities in self-regulated learning: A focus on contextual factors, contingencies, and dynamic relations. Metacognition and Learning, 10, 1–13. doi:10.1007/s11409-015-9134-6
BENJAMINI, Y., & HOCHBERG, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 57(1), 289-300.
BLOOM, B. S. (1956). Taxonomy of educational objectives: The classification of educational goals. New York: D. McKay.
BOUCHET, F., HARLEY, J. M., TREVORS, G. J., & AZEVEDO, R. (2013). Clustering and profiling students according to their interactions with an intelligent tutoring system fostering self-regulated learning. Journal of Educational Data Mining, 5(1), 104–146.
BRUSILOVSKY, P. (2001). Adaptive hypermedia. User Modeling and User-Adapted Interaction, 11, 87– 110. doi:10.1023/A:1011143116306
BRUSILOVSKY, P. (2007). Adaptive navigation support. In P. Brusilovsky, A. Kobsa, & W. Nejdl (Eds.), The adaptive web. Methods and strategies of web personalization (pp. 263–290). Berlin Heidelberg: Springer. doi:10.1007/978-3-540-72079-9
CHI, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. Journal of the Learning Sciences, 6(3), 271–315. doi:10.1207/s15327809jls0603 1
CLAREBOUT, G., & ELEN, J. (2006). Tool use in computer-based learning environments: Towards a research framework. Computers in Human Behavior, 22, 389–411. doi:10.1016/j.chb.2004.09.007
CLAREBOUT, G., ELEN, J., COLLAZO, N. A. J., LUST, G., & JIANG, L. (2013). Metacognition and the use of tools. In R. Azevedo & V. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 187–195). Springer Science. doi: 10.1007/978-1-4419-5546-3 13
COCEA, M., & WEIBELZAHL, S. (2009). Log file analysis for disengagement detection in e-Learning environments. User Modelling and User-Adapted Interaction, 19, 341–385. doi:10.1007/s11257-009- 9065-5
DENT, A. L., & HOYLE, R. H. (2015). A framework for evaluating and enhancing alignment in selfregulated learning research. Metacognition and Learning, 10, 165–179. doi:10.1007/s11409-015- 9136-4.
ERICSSON, K. A., & SIMON, H. A. (1993). Protocol analysis: Verbal reports as data. Cambridge, Mass.: MIT Press.
GE, X. (2013). Designing learning technologies to support self-regulation during III-structured problemsolving processes. In R. Azevedo & V. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 213–228). New York: Springer. doi:10.1007/978-1-4419-5546-3 15
GREENE, J. A., DELLINGER, K. R., TUYS UZOGLU, B. B., & COSTA, L.-J. (2013). A two-tiered approach to analyzing self-regulated learning data to inform the design of hypermedia learning environments. In R. Azevedo & V. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 117–128). New York: Springer. doi:10.1007/978-1-4419-5546-3 8
GUNTHER, C. W., & VAN DER AALST, W. M. P. (2007). Fuzzy mining – Adaptive process simplification based on multi-perspective metrics. Business Process Management - Lecture Notes in Computer Science, 4714, 328–343. doi:10.1007/978-3-540-75183-0
HAMALAINEN, W., & VINNI, M. (2010). Classifiers for educational data mining. In C. Romero, S. Ventura, M. Pechenizkiy, & R. Baker (Eds.), Handbook of Educational Data Mining (pp. 57–74). Boca Raton, FL: Chapman&Hall/CRC.
HATTIE, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York: Routledge.
JEONG, H., GUPTA, A., ROSCOE, R., WAGSTER, J., BISWAS, G., & SCHWARTZ, D. (2008). Using hidden markov models to characterize student behaviors in learning-by-teaching environments. Lecture Notes in Computer Science, 5091, 614–625. doi:10.1007/978-3-540-69132-7 64
JOHNSON, A. M., AZEVEDO, R., & D'MELLO, S. K. (2011). The temporal and dynamic nature of self-regulatory processes during independent and externally assisted hypermedia learning. Cognition and Instruction, 29(4), 471–504. doi:10.1080/07370008.2011.610244
KINNEBREW, J. S., SEGEDY, J. R., & BISWAS, G. (2014). Analyzing the temporal evolution of students' behaviors in open-ended learning environments. Metacognition and Learning, 9, 187–215. doi:10.1007/s11409-014-9112-4
KRAMARSKI, B., & GUTMAN, M. (2006). How can self-regulated learning be supported in mathematical e-learning environments? Journal of Computer Assisted Learning, 22, 24–33. doi:10.1111/j.1365- 2729.2006.00157.x
LAJOIE, S. P., NAISMITH, L., POITRAS, E., HONG, Y.-J., CRUZ-PANESSO, I., RANELLUCCI, J., ... WISEMAN, J. (2013). Technology-rich tools to support self-regulated learning and performance in medicine. In R. Azevedo & V. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 229–242). New York: Springer. doi:10.1007/978-1-4419-5546-3 16
LEHMANN, T., HAHNLEIN, I., & IFENTHALER, D. (2014). Cognitive, metacognitive and motivational perspectives on preflection in self-regulated online learning. Computers in Human Behavior, 32, 313– 323. doi:10.1016/j.chb.2013.07.051
LESTER, J. C., MOTT, B. W., ROBISON, J. L., ROWE, J. P., & SHORES, L. R. (2013). Supporting selfregulated science learning in narrative-centered learning environments. In R. Azevedo & V. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 471–483). New York: Springer. doi:10.1007/978-1-4419-5546-3 30
LIN, X., HMELO, C., KINZER, C. K., & SECULES, T. J. (1999). Designing technology to support reflection. Educational Technology Research and Development, 47(3), 43–62. doi:10.1007/BF02299633
MALMBERG, J., JARVEL A, S., JARVENOJA, H., & PANADERO, E. (2015). Promoting socially shared regulation of learning in CSCL: Progress of socially shared regulation among high- and lowperforming groups. Computers in Human Behavior, 52, 562–572. doi:10.1016/j.chb.2015.03.082
MARTIN, T., & SHERIN, B. (2013). Learning analytics and computational techniques for detecting and evaluating patterns in learning: An introduction to the special issue. Journal of the Learning Sciences, 22(4), 511–520. doi:10.1080/10508406.2013.840466
MOLENAAR, I., & CHIU, M. M. (2014). Dissecting sequences of regulation and cognition: Statistical discourse analysis of primary school children's collaborative learning. Metacognition and Learning, 9(2), 137–160. doi:10.1007/s11409-013-9105-8
MOLENAAR, I., & JARVEL A, S. (2014). Sequential and temporal characteristics of self and socially regulated learning. Metacognition and Learning, 9(2), 75–85. doi:10.1007/s11409-014-9114-2
MOLENAAR, I., & RODA, C. (2008). Attention management for dynamic and adaptive scaffolding. Pragmatics & Cognition, 16(2), 224-271. doi: 10.1075/pc.16.2.04mol
REIMANN, P., MARKAUSKAITE, L., & BANNERT, M. (2014). E-Research and learning theory: What do sequence and process mining methods contribute? British Journal of Educational Technology, 45(3), 528–540. doi:10.1111/bjet.12146
REIMANN, P., & YACEF, K. (2013). Using process mining for understanding learning. In R. Luckin, P. Puntambekar, P. Goodyear, B. Grabowski, J. Underwood, & N. Winters (Eds.), Handbook of Design in Educational Technology (pp. 472–481). New York: Routledge.
ROLL, I., ALEVEN, V., MCLAREN, B. M., & KOEDINGER, K. R. (2011). Improving students' helpseeking skills using metacognitive feedback in an intelligent tutoring system. Learning and Instruction, 21(2), 267–280. doi:10.1016/j.learninstruc.2010.07.004
SCHOOR, C., & BANNERT, M. (2012). Exploring regulatory processes during a computer-supported collaborative learning task using process mining. Computers in Human Behavior, 28, 1321–1331. doi:10.1016/j.chb.2012.02.016
SCHWONKE, R., HAUSER, S., NUCKLES, M., & RENKL, A. (2006). Enhancing computer-supported writing of learning protocols by adaptive prompts. Computers in Human Behavior, 22, 77–92. doi:10.1016/j.chb.2005.01.002
SITZMANN, T., BELL, B. S., KRAIGER, K., & KANAR, A. M. (2009). A multilevel analysis of the effect of prompting self-regulation in technology-delivered instruction. Personnel Psychology, 62, 697–734. doi:10.1111/j.1744-6570.2009.01155.x
SONG, M., & VAN DER AALST, W.M.P. (2007). Supporting process mining by showing events at a glance. In K. Chari & A. Kumar (Eds.), Proceedings of 17th Annual Workshop on Information Technologies and Systems (WITS 2007) (pp. 139–145). Montreal, Canada.
SONNENBERG, C., & BANNERT, M. (2015). Discovering the effects of metacognitive prompts on the sequential structure of SRL-processes using process mining techniques. Journal of Learning Analytics, 2(1), 72–100.
THILLMANN, H., KUNSTING, J., WIRTH, J., & LEUTNER, D. (2009). Is it merely a question of "what" to prompt or also "when" to prompt? Zeitschrift fur P adagogische, 23(2), 105–115. doi:10.1024/1010-0652.23.2.105
TRCKA, N., PECHENIZKIY, M., & VAN DER AALST, W. M. P. (2010). Process mining from educational data. In C. Romero, S. Ventura, M. Pechenizkiy, & R. Baker (Eds.), Handbook of Educational Data Mining (pp. 123–142). Boca Raton, FL: Chapman&Hall/CRC.
VAN DER AALST, W. (2011). Process mining: Discovery, conformance and enhancement of business processes. Berlin, Heidelberg: Springer.
VAN DER AALST, W., WEIJTERS, T., & MARUSTER, L. (2004). Workflow mining: Discovering process models from event logs. IEEE Transactions on Knowledge and Data Engineering, 16(9), 1128–1142. doi:10.1109/TKDE.2004.47
VEENMAN, M. (1993). Metacognitive ability and metacognitive skill: Determinants of discovery learning in computerized learning environments. University of Amsterdam.
VEENMAN, M. V. J. (2007). The assessment and instruction of self-regulation in computer-based environments: A discussion. Metacognition and Learning, 2(2-3), 177–183. doi:10.1007/s11409-007- 9017-6
WALKER, E., RUMMEL, N., & KOEDINGER, K. (2011). Designing automated adaptive support to improve student helping behaviors in a peer tutoring activity. International Journal of Computer Supported Collaborative Learning, 6(2), 279-306. doi: 10.1007/s11412-011-9111-2
WINNE, P. H. (1996). A metacognitive view of individual differences in self-regulated learning. Learning and Individual Differences, 8(4), 327–353.
WINNE, P. H. (2014). Issues in researching self-regulated learning as patterns of events. Metacognition and Learning, 9, 229–237. doi:10.1007/s11409-014-9113-3
WINNE, P. H., & BAKER, R. S. (2013). The potentials of educational data mining for researching metacognition, motivation and self-regulated learning. Journal of Educational Data Mining, 5(1), 1–8.
WINNE, P. H., & HADWIN, A. F. (2008). The weave of motivation and self-regulated learning. In D. Schunk & B. J. Zimmerman (Eds.), Motivation and self-regulated learning: Theory, research, and applications (pp. 297–314). NY: Taylor & Francis.
WIRTH, J. (2009). Promoting self-regulated learning through prompts. Zeitschrift fur P adagogische Psy
chologie, 23(2), 91–94. doi:10.1024/1010-0652.23.2.91
WITTWER, J., & RENKL, A. (2008). Why instructional explanations often do not work: A framework for understanding the effectiveness of instructional explanations. Educational Psychologist, 43, 49–64. doi:10.1080/00461520701756420
YEH, Y. F., CHEN, M. C., HUNG, P. H., & HWANG, G. J. (2010). Optimal self-explanation prompt design in dynamic multi-representational learning environments. Computers and Education, 54, 1089– 1100. doi:10.1016/j.compedu.2009.10.013
ZIMMERMAN, B. J. (2008). Investigating self-regulation and motivation: Historical background, methodological developments, and future prospects. American Educational Research Journal, 45(1), 166–183. doi:10.3102/0002831207312909
Issue
Section
Articles
Authors who publish with this journal agree to the following terms:
- The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
- Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
- The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons 4.0 License (Attribution-Noncommercial-No Derivatives 4.0 International), or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
- Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
- Noncommercial—other users (including Publisher) may not use this Work for commercial purposes;
- No Derivative Works—other users (including Publisher) may not alter, transform, or build upon this Work,with the understanding that any of the above conditions can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
- The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
- Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
- The Author represents and warrants that:
- the Work is the Author’s original work;
- the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
- the Work is not pending review or under consideration by another publisher;
- the Work has not previously been published;
- the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
- the Work contains no libel, invasion of privacy, or other unlawful matter.
- The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.