KT-Bi-GRU: Student Performance Prediction with a Bi-Directional Recurrent Knowledge Tracing Neural Network
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Abstract
Student performance is affected by their knowledge which changes dynamically over time. Therefore,
employing recurrent neural networks (RNN), which are known to be very good in dynamic time series
prediction, can be a suitable approach for student performance prediction. We propose such a neural
network architecture containing two modules: (i) a dynamic sub-network including a recurrent Bi-GRU
layer used for knowledge state estimation, (ii) a non-dynamic, feed-forward sub-network for predicting
answer correctness based on the current question and current student knowledge state. The model
modifies our previously proposed architecture and is different from all other existing models because it
estimates the student’s knowledge state considering only their previous responses. Thus the dynamic
sub-network generates more stable knowledge state vector representations since they are independent of
the current question. We studied both single-skill and multi-skill question scenarios and employed embeddings
to represent questions and responses. In the multi-skill case the initialization of the question
embedding matrix with pretrained word-embeddings is found to improve model performance. The experimental
results showed that our current KT-Bi-GRU model and the previous one have similar performance
while both surpassed the performance of previous state-of-the-art knowledge tracing models for five out
of seven datasets where in some cases, the difference is quite noticeable.
How to Cite
##plugins.themes.bootstrap3.article.details##
knowledge tracing, student performance prediction, dynamic neural networks, knowledge state
ASSISTMENTSDATA. 2015. Assistments data. https://sites.google.com/site/assistmentsdata/.
CEN, H., KOEDINGER, K., AND JUNKER, B. 2006. Learning factors analysis – a general method for cognitive model evaluation and improvement. In Intelligent Tutoring Systems, M. Ikeda, K. D. Ashley, and T.-W. Chan, Eds. Springer Berlin Heidelberg, 164–175.
CHO, K., VAN MERRIËNBOER, B., GULCEHRE, C., BAHDANAU, D., BOUGARES, F., SCHWENK, H., AND BENGIO, Y. 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 1724–1734.
CORBETT, A. T. AND ANDERSON, J. R. 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction 4, 4, 253–278.
DELIANIDI, M., DIAMANTARAS, K., CHRYSOGONIDIS, G., AND NIKIFORIDIS, V. 2021. Student performance prediction using dynamic neural models. In Proceedings of the Fourteenth International Conference on Educational Data Mining (EDM 2021), S. Hsiao, S. Sahebi, F. Bouchet, and J.-J. Vie, Eds. Educational Data Mining Society, 46–54.
GHOSH, A., HEFFERNAN, N., AND LAN, A. S. 2020. Context-aware attentive knowledge tracing. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. Association for Computing Machinery, 2330–2339.
HAMBLETON, R. K., SWAMINATHAN, H., AND ROGERS, H. J. 1991. Fundamentals of item response theory. Vol. 2. Sage Publications.
HE, Z., LI, W., AND YAN, Y. 2021. Modeling knowledge proficiency using multi-hierarchical capsule graph neural network. Applied Intelligence 52, 7230–7247. https://doi.org/10.1007/s10489-021- 02765-w.
HOCHREITER, S. AND SCHMIDHUBER, J. 1997. Long short-term memory. Neural computation 9, 8, 1735–1780.
JOULIN, A., GRAVE, E., BOJANOWSKI, P., DOUZE, M., JÉGOU, H., AND MIKOLOV, T. Dec 2016. Fasttext.zip: Compressing text classification models. arXiv, arXiv:1612.03651. https://doi.org/10.48550/arXiv.1612.03651.
KÄSER, T., KLINGLER, S., SCHWING, A. G., AND GROSS, M. H. 2017. Dynamic bayesian networks for student modeling. IEEE Transactions on Learning Technologies 10, 450–462.
KHAYI, N. A. AND RUS, V. 2019. Clustering students based on their prior knowledge. In Proceedings of the 12th International Conference on Educational Data Mining (EDM 2019), C. F. Lynch, A. Merceron, M. Desmarais, and R. Nkambou, Eds. Educational Data Mining Society, 246–251.
KINGMA, D. P. AND BA, J. 2015. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Y. Bengio and Y. LeCun, Eds. https://doi.org/10.48550/arXiv.1412.6980.
KOEDINGER, K. R., BAKER, R. S., CUNNINGHAM, K., SKOGSHOLM, A., LEBER, B., AND STAMPER, J. 2010. A data repository for the edm community: The pslc datashop. In Handbook of educational data mining, C. Romero, S. Ventura, M. Pechenizkiy, and R. S. Baker, Eds. Vol. 43. CRC Press, Boca Raton, FL, 43–56. https://doi.org/10.1201/b10274.
LI, P., LUO, A., LIU, J., WANG, Y., ZHU, J., DENG, Y., AND ZHANG, J. 2020. Bidirectional gated recurrent unit neural network for chinese address element segmentation. ISPRS International Journal of Geo-Information 9, 11. https://doi.org/10.3390/ijgi9110635.
LING, C. X., HUANG, J., AND ZHANG, H. 2003. Auc: a statistically consistent and more discriminating measure than accuracy. In Proceedings of the 18th International Joint Conference on Artificial Intelligence. IJCAI’03, vol. 3. Morgan Kaufmann Publishers Inc., 519–524.
LIU, Q., HUANG, Z., YIN, Y., CHEN, E., XIONG, H., SU, Y., AND HU, G. 2019. Ekt: Exercise-aware knowledge tracing for student performance prediction. IEEE Transactions on Knowledge and Data Engineering 33, 1, 100–115.
LIU, Q., SHEN, S., HUANG, Z., CHEN, E., AND ZHENG, Y. 2021. A survey of knowledge tracing. CoRR abs/2105.15106. https://doi.org/10.48550/arXiv.2105.15106.
LIU, Y., YANG, Y., CHEN, X., SHEN, J., ZHANG, H., AND YU, Y. 2021. Improving knowledge tracing via pre-training question embeddings. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, C. Bessiere, Ed. IJCAI’20. International Joint Conferences on Artificial Intelligence, 1577–1583.
MA, R., ZHANG, L., LI, J., MEI, B., MA, Y., AND ZHANG, H. 2021. Dtkt: An improved deep temporal convolutional network for knowledge tracing. In Proceedings of the 2021 16th International Conference on Computer Science & Education (ICCSE). IEEE, 794–799.
MIKOLOV, T., CHEN, K., CORRADO, G., AND DEAN, J. 2013. Efficient estimation of word representations in vector space. In Proceedings of the 1st International Conference on Learning Representations, Workshop Track Proceedings, Y. Bengio and Y. LeCun, Eds. ICLR 2013. Scottsdale, Arizona, USA. https://doi.org/10.48550/arXiv.1301.3781.
MILLER, A., FISCH, A., DODGE, J., KARIMI, A.-H., BORDES, A., AND WESTON, J. 2016. Keyvalue memory networks for directly reading documents. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, J. Su, K. Duh, and X. Carreras, Eds. Association for Computational Linguistics, 1400–1409.
OMAR, T., ALZAHRANI, A., AND ZOHDY, M. 2020. Clustering approach for analyzing the student’s efficiency and performance based on data. Journal of Data Analysis and Information Processing 8, 03, 171–182. https://doi.org/10.4236/jdaip.2020.83010.
PANDEY, S. AND KARYPIS, G. 2019. A self-attentive model for knowledge tracing. In Proceedings of the 12th International Conference on Educational Data Mining, EDM 2019, C. F. Lynch, A. Merceron, M. Desmarais, and R. Nkambou, Eds. International Educational Data Mining Society, 384–389.
PAVLIK, P. I., CEN, H., AND KOEDINGER, K. R. 2009. Performance factors analysis –a new alternative to knowledge tracing. In Proceedings of the 2009 Conference on Artificial Intelligence in Education: Building Learning Systems That Care: From Knowledge Representation to Affective Modelling, V. Dimitrova, R. Mizoguchi, B. du Boulay, and A. Graesser, Eds. IOS Press, 531–538.
PIECH, C., BASSEN, J., HUANG, J., GANGULI, S., SAHAMI, M., GUIBAS, L., AND SOHL-DICKSTEIN, J. 2015. Deep knowledge tracing. In Advances in Neural Information Processing Systems, C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, Eds. Vol. 28. Curran Associates, Inc., 505–513.
PU, S., CONVERSE, G., AND HUANG, Y. 2021. Deep performance factors analysis for knowledge tracing. In Proceedings of the Artificial Intelligence in Education: 22nd International Conference, AIED 2021. Springer-Verlag, Berlin, Heidelberg, 331–341.
SCHUSTER, M. AND PALIWAL, K. K. 1997. Bidirectional recurrent neural networks. IEEE transactions on Signal Processing 45, 11, 2673–2681.
SHEN, S., LIU, Q., CHEN, E., WU, H., HUANG, Z., ZHAO, W., SU, Y., MA, H., AND WANG, S. Convolutional knowledge tracing: Modeling individualization in student learning process. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. SIGIR ’20. Association for Computing Machinery, 1857–1860. https://doi.org/10.1145/3397271.3401288.
SONG, X., LI, J., TANG, Y., ZHAO, T., CHEN, Y., AND GUAN, Z. 2021. Jkt: A joint graph convolutional network based deep knowledge tracing. Information Sciences 580, 510–523.
SRIVASTAVA, N., HINTON, G., KRIZHEVSKY, A., SUTSKEVER, I., AND SALAKHUTDINOV, R. 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1, 1929–1958.
VASWANI, A., SHAZEER, N., PARMAR, N., USZKOREIT, J., JONES, L., GOMEZ, A. N., KAISER, L., AND POLOSUKHIN, I. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, U. von Luxburg, I. Guyon, S. Bengio, H.Wallach, and R. Fergus, Eds. NIPS’17. Curran Associates Inc., 6000–6010.
WANG, W., LIU, T., CHANG, L., GU, T., AND ZHAO, X. 2020. Convolutional recurrent neural networks for knowledge tracing. In Proceedings of the 2020 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC). IEEE, 287–290.
WANG, Z., LAMB, A., SAVELIEV, E., CAMERON, P., ZAYKOV, Y., HERNÁNDEZ-LOBATO, J. M., TURNER, R. E., BARANIUK, R. G., BARTON, C., JONES, S. P., WOODHEAD, S., AND ZHANG, C. 2020. Diagnostic questions: The neurips 2020 education challenge. arXiv preprint arXiv:2007.12061.
YAMADA, I., ASAI, A., SAKUMA, J., SHINDO, H., TAKEDA, H., TAKEFUJI, Y., AND MATSUMOTO, Y. 2020. Wikipedia2Vec: An efficient toolkit for learning and visualizing the embeddings of words and entities from Wikipedia. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Association for Computational Linguistics, 23–30. https://wikipedia2vec.github.io/wikipedia2vec/.
YANG, S., ZHU, M., HOU, J., AND LU, X. Deep knowledge tracing with convolutions. In 12th International Conference on Educational Data Mining, EDM 2019, C. F. Lynch, A. Merceron, M. Desmarais, and R. Nkambou, Eds. International Educational Data Mining Society, 683–686.
YANG, Y., SHEN, J., QU, Y., LIU, Y., WANG, K., ZHU, Y., ZHANG, W., AND YU, Y. 2021. Gikt: a graph-based interaction model for knowledge tracing. In Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2020, F. Hutter, K. Kersting, J. Lijffijt, and I. Valera, Eds. Springer International Publishing, Cham, 299–315.
YEUNG, C.-K. 2019. Deep-irt: Make deep learning based knowledge tracing explainable using item response theory. In Proceedings of the 12th International Conference on Educational Data Mining, EDM 2019, C. F. Lynch, A. Merceron, M. Desmarais, and R. Nkambou, Eds. International Educational Data Mining Society, 683–686.
ZHANG, J., SHI, X., KING, I., AND YEUNG, D.-Y. 2017. Dynamic key-value memory networks for knowledge tracing. In Proceedings of the 26th International Conference on World Wide Web. WWW ’17. International World Wide Web Conferences Steering Committee, 765–774. https://doi.org/10.1145/3038912.3052580.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:
- The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
- Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
- The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons 4.0 License (Attribution-Noncommercial-No Derivatives 4.0 International), or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
- Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
- Noncommercial—other users (including Publisher) may not use this Work for commercial purposes;
- No Derivative Works—other users (including Publisher) may not alter, transform, or build upon this Work,with the understanding that any of the above conditions can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
- The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
- Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
- The Author represents and warrants that:
- the Work is the Author’s original work;
- the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
- the Work is not pending review or under consideration by another publisher;
- the Work has not previously been published;
- the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
- the Work contains no libel, invasion of privacy, or other unlawful matter.
- The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.