ACE: AI-Assisted Construction of Educational Knowledge Graphs with Prerequisite Relations

##plugins.themes.bootstrap3.article.main##

##plugins.themes.bootstrap3.article.sidebar##

Published Nov 26, 2024
Mehmet Cem Aytekin Yücel Saygın

Abstract

Knowledge graphs are effective tools for organizing information. In this work, we focus on a specialized type of Knowledge Graph called an Educational Knowledge Graph (EKG), with prerequisite relations forming paths that students can follow in their learning process. An EKG provides several features, including a comprehensive visual representation of the learning domain, and offers students alternative learning paths. The manual construction of EKGs is a time-consuming and labor-intensive task, requiring domain experts to evaluate each concept pair to identify prerequisite relations. To address this challenge, we propose a methodology that combines machine learning techniques and expert knowledge. We first introduce a prerequisite scoring mechanism for concept pairs based on semantic references captured through word embeddings. Concept pairs are then ranked with respect to their scores, and pairs with high scores are selected for expert evaluation, reducing the total number of pairs to be evaluated. The expert is iteratively presented with a concept pair, and an EKG is dynamically constructed in the background based on the expert's label. As the graph evolves, some prerequisites can be inferred based on the existing ones, further reducing the expert's task. We implemented our methodology in a web application, allowing experts to interact with the system and create their own graphs. Evaluations on real-life benchmark datasets show that our AI-assisted graph construction methodology forms accurate graphs and significantly reduces expert effort during the process. Further experiments conducted on a dataset from an educational platform demonstrate that students who study concept pairs in a prerequisite order determined by our methodology have a better overall success rate indicating that EKGs can improve learning outcomes in education. Interested readers can access additional material and the dataset at our Github repository*.

How to Cite

Aytekin, M. C., & Saygın, Y. (2024). ACE: AI-Assisted Construction of Educational Knowledge Graphs with Prerequisite Relations. Journal of Educational Data Mining, 16(2), 85–114. https://doi.org/10.5281/zenodo.14250896
Abstract 59 | HTML Downloads 10 PDF Downloads 14

##plugins.themes.bootstrap3.article.details##

Keywords

semantic search, prerequisite relation extraction, knowledge graph construction

References
Adorni, G., Alzetta, C., Koceva, F., Passalacqua, S., and Torre, I. 2019. Towards the identification of propaedeutic relations in textbooks. In Artificial Intelligence in Education, S. Isotani, E. Millán, A. Ogan, P. Hastings, B. McLaren, and R. Luckin, Eds. Springer International Publishing, Cham, 1–13.

Aguiar, C. Z., Cury, D., and Zouaq, A. 2016. Automatic construction of concept maps from texts. In 4th International Conference on Concept Mapping. Springer, Tallinn, Estonia, 1–6.

Aytekin, M. C., Räbiger, S., and Saygın, Y. 2020. Discovering the prerequisite relationships among instructional videos from subtitles. In Proceedings of the 13th International Conference on Educational Data Mining. International Educational Data Mining Society, Fully virtual conference, 569–573.

Chen, P., Lu, Y., Zheng, V. W., Chen, X., and Yang, B. 2018. Knowedu: A system to construct knowledge graph for education. IEEE Access 6, 31553–31563.

Chiou, C.-C., Tien, L.-C., and Lee, L.-T. 2015. Effects on learning of multimedia animation combined with multidimensional concept maps. Computers & Education 80, 211–223.

Devlin, J., Chang, M.-W., Kenton, L., and Toutanova, K. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT. Association for Computational Linguistics, Minneapolis, Minnesota, USA, 4171–4186.

Dong, L., Yang, N., Wang, W., Wei, F., Liu, X., Wang, Y., Gao, J., Zhou, M., and Hon, H.-W. 2019. Unified language model pre-training for natural language understanding and generation. In Proceedings of the 33rd International Conference on Neural Information Processing Systems. Curran Associates Inc., Red Hook, NY, USA.

Furtado, P. G. F., Hirashima, T., and Hayashi, Y. 2019. Reducing cognitive load during closed concept map construction and consequences on reading comprehension and retention. IEEE Transactions on Learning Technologies 12, 3, 402–412.

Gong, W., Smith, D., Wang, Z., Barton, C., Woodhead, S., Pawlowski, N., Jennings, J., and Zhang, C. 2022. Neurips competition instructions and guide: Causal insights for learning paths in education. https://arxiv.org/abs/2208.12610.

Hirashima, T., Yamasaki, K., Fukuda, H., and Funaoi, H. 2015. Framework of kit-build concept map for automatic diagnosis and its preliminary use. Research and Practice in Technology Enhanced Learning 10, 1, 1–21.

Hogan, A., Blomqvist, E., Cochez, M., d’Amato, C., Melo, G. d., Gutierrez, C., Kirrane, S., Gayo, J. E. L., Navigli, R., Neumaier, S., et al. 2021. Knowledge graphs. ACM Computing Surveys (CSUR) 54, 4, 1–37.

Jia, C., Shen, Y., Tang, Y., Sun, L., and Lu, W. 2021. Heterogeneous graph neural networks for concept prerequisite relation learning in educational data. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, K. Toutanova, A. Rumshisky, L. Zettlemoyer, D. Hakkani-Tur, I. Beltagy, S. Bethard, R. Cotterell, T. Chakraborty, and Y. Zhou, Eds. Association for Computational Linguistics, Online, 2036–2047.

Lee, S., Park, Y., and Yoon, W. C. 2015. Burst analysis for automatic concept map creation with a single document. Expert Systems with Applications 42, 22, 8817–8829.

Li, I., Fabbri, A. R., Tung, R. R., and Radev, D. R. 2019. What should I learn first: Introducing lecturebank for NLP education and prerequisite chain learning. Proceedings of the AAAI Conference on Artificial Intelligence 33, 01 (Jul.), 6674–6681.

Liang, C., Wu, Z., Huang, W., and Giles, C. L. 2015. Measuring prerequisite relations among concepts. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Lisbon, Portugal, 1668–1674.

Liang, C., Ye, J., Wu, Z., Pursel, B., and Giles, C. L. 2017. Recovering concept prerequisite relations from university course dependencies. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. AAAI’17. AAAI Press, 4786–4791.

Liang, C., Ye, J., Zhao, H., Pursel, B., and Giles, C. L. 2019. Active learning of strict partial orders: A case study on concept prerequisite relations. In Proceedings of the 12th International Conference on Educational Data Mining (EDM 2019). International Educational Data Mining Society (IEDMS).

Manrique, R., Pereira, B., and Mariño, O. 2019. Exploring knowledge graphs for the identification of concept prerequisites. Smart Learning Environments 6, 1, 1–18.

Miaschi, A., Alzetta, C., Cardillo, F. A., and Dell’Orletta, F. 2019. Linguistically-driven strategy for concept prerequisites learning on Italian. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications. Association for Computational Linguistics, Florence, Italy, 285–295.

Molontay, R., Horváth, N., Bergmann, J., Szekrényes, D., and Szabó, M. 2020. Characterizing curriculum prerequisite networks by a student flow approach. IEEE Transactions on Learning Technologies 13, 3, 491–501.

Novak, J. D. 2010. Learning, creating, and using knowledge: Concept maps as facilitative tools in schools and corporations. Routledge, New York, NY, USA.

Novak, J. D. and Cañas, A. J. 2006. The theory underlying concept maps and how to construct them. Florida Institute for Human and Machine Cognition 1, 2006–2001.

Pan, L., Li, C., Li, J., and Tang, J. 2017. Prerequisite relation learning for concepts in MOOCs. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Vancouver, Canada, 1447–1456.

Pinandito, A., Prasetya, D. D., Hayashi, Y., and Hirashima, T. 2021. Design and development of semi-automatic concept map authoring support tool. Research and Practice in Technology Enhanced Learning 16, 1, 1–19.

Roy, S., Madhyastha, M., Lawrence, S., and Rajan, V. 2019. Inferring concept prerequisite relations from online educational resources. Proceedings of the AAAI Conference on Artificial Intelligence 33, 01 (Jul.), 9589–9594.

Sayyadiharikandeh, M., Gordon, J., Ambite, J.-L., and Lerman, K. 2019. Finding prerequisite relations using the wikipedia clickstream. In Companion Proceedings of The 2019 World Wide Web Conference. WWW ’19. Association for Computing Machinery, New York, NY, USA, 1240–1247.

Shokrzadeh, Z., Feizi-Derakhshi, M.-R., Balafar, M.-A., and Bagherzadeh Mohasefi, J. 2024. Knowledge graph-based recommendation system enhanced by neural collaborative filtering and knowledge graph embedding. Ain Shams Engineering Journal 15, 1, 102263.

Talukdar, P. and Cohen, W. 2012. Crowdsourced comprehension: Predicting prerequisite structure in Wikipedia. In Proceedings of the Seventh Workshop on Building Educational Applications Using NLP. Association for Computational Linguistics, Montréal, Canada, 307–315.

Yang, C., Cui, H., Lu, J., Wang, S., Xu, R., Ma, W., Yu, Y., Yu, S., Kan, X., Ling, C., Fu, T., Zhao, L., Ho, J., and Wang, F. 2024. A review on knowledge graphs for healthcare: Resources, applications, and promises. https://arxiv.org/abs/2306.04802.

Yu, J., Wang, Y., Zhong, Q., Luo, G., Mao, Y., Sun, K., Feng, W., Xu, W., Cao, S., Zeng, K., Yao, Z., Hou, L., Lin, Y., Li, P., Zhou, J., Xu, B., Li, J., Tang, J., and Sun, M. 2021. Mooccubex: A large knowledge-centered repository for adaptive learning in moocs. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. CIKM ’21. Association for Computing Machinery, New York, NY, USA, 4643–4652.

Zhang, J., Lan, H., Yang, X., Zhang, S., Song, W., and Peng, Z. 2022. Weakly supervised setting for learning concept prerequisite relations using multi-head attention variational graph auto-encoders. Knowledge-Based Systems 247, 108689.
Section
Articles