A course hybrid recommender system for limited information scenarios
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Abstract
Recommender systems in educational contexts have proven to be effective in identifying learning
resources that fit the interests and needs of learners. Their usage has been of special interest in online
self-learning scenarios to increase student retention and improve the learning experience. In this article,
we present the design of a hybrid course recommendation system for an online learning platform. The
proposed hybrid system articulates the recommendation carried out by collaborative and content-based
filter strategies. For the collaborative filtering recommender, we address the challenge of recommending
meaningful content with limited information from users by using rating estimation strategies from a log
system (Google Analytics). Our approach posits strategies to mine logs and generates effective ratings
through the counting and temporal analysis of sessions. We evaluate different rating penalty strategies
and compare the use of per-user metrics for rating estimation. For the content-based recommender, we
compare different text embeddings that range from well-known topic models (LSA and LDA) to more
recent multilingual contextual embeddings pre-trained on large-scale unlabelled corpora. The results
show that the best model in terms of P@5 was the Collaborative filtering recommendation model with
a value of 0:4, i.e., two out of five courses recommended could be of the user’s interest. This result is
satisfactory considering that our models were trained from ratings inferred from implicit user data. The
content-based strategies did not yield significant results, however, these strategies help to mitigate the
cold start problem and validate the use of a combined hybrid strategy.
How to Cite
##plugins.themes.bootstrap3.article.details##
recommender systems, collaborative filtering, content-based recommendations, hybrid recommendations, logs mining, Contextual Embeddings
AGGARWAL, C. C. 2016b. Evaluating Recommender Systems, 1st. ed. Springer International Publishing, Cham, Chapter 7, 225–254.
BRANDON, D. 2019. Google Analytics. J. Comput. Sci. Coll. 34, 81–82.
CAMPOS, R., DOS SANTOS, R. P., AND OLIVEIRA, J. 2020. A recommendation system based on knowledge gap identification in moocs ecosystems. In XVI Brazilian Symposium on Information Systems, F. E. Horita, C. A. Kamienski, S. D. Ávila E Silva, A. M. Magdaleno, and D. Viana, Eds. SBSI’20. Association for Computing Machinery, New York, NY, USA, 1–8.
CAO, B., CHEN, J., LIU, J., AND WEN, Y. 2020. A topic attention mechanism and factorization machines based mobile application recommendation method. Mobile Networks and Applications 25, 4 (Aug), 1208–1219.
DO, P., NGUYEN, K., VU, T. N., DUNG, T. N., AND LE, T. D. 2017. Integrating knowledge based reasoning algorithms and collaborative filtering into e-learning material recommendation system. In Future Data and Security Engineering, T. K. Dang, R. Wagner, J. Küng, N. Thoai, M. Takizawa, and E. J. Neuhold, Eds. FDSE ’17, vol. 10646. Springer International Publishing, Cham, 419–432.
GAUR, L., SINGH, G., JEYTA, AND KUMAR, S. 2016. Google Analytics: A tool to make websites more robust. In Proceedings of the Second International Conference on Information and Communication Technology for Competitive Strategies. ICTCS ’16. Association for Computing Machinery, New York, NY, USA, 1–7.
GHEBGHOUB, O., ABEL, M.-H., AND MOULIN, C. 2008. Learning object indexing tool based on a lom ontology. In 2008 Eighth IEEE International Conference on Advanced Learning Technologies, P. Díaz, Kinshuk, I. Aedo, and E. Mora, Eds. IEEE, 576–578.
GRÉVISSE, C., MANRIQUE, R., MARIÑO, O., AND ROTHKUGEL, S. 2018. Knowledge graph-based teacher support for learning material authoring. In Advances in Computing, J. E. Serrano C. and J. C. Martínez-Santos, Eds. Springer International Publishing, Cham, 177–191.
HASSAN, J., LEONG, J., AND SCHNEIDER, B. 2021. Multimodal data collection made easy: The ez-mmla toolkit: A data collection website that provides educators and researchers with easy access to multimodal data streams. In LAK21: 11th International Learning Analytics and Knowledge Conference. LAK21. Association for Computing Machinery, New York, NY, USA, 579–585.
ILYAS, I. F. AND CHU, X. 2019. Data Cleaning. Association for Computing Machinery, New York, NY, USA.
KANG, B. 2021. How the covid-19 pandemic is reshaping the education service. In The Future of Service Post-COVID-19 Pandemic, J. Lee and S. H. Han, Eds. The ICT and Evolution of Work, vol. 1. Springer, Singapore, 15–36.
KEW, S. N. AND TASIR, Z. 2022. Learning analytics in online learning environment: A systematic review on the focuses and the types of student-related analytics data. Technology, Knowledge and Learning 27, 2 (Jun), 405–427.
LIAO, T., FENG, X., SUN, Y., WANG, H., LIAO, C., AND LI, Y. 2020. Online teaching platform based on big data recommendation system. In Proceedings of the 5th International Conference on Information and Education Innovations. ICIEI ’20. Association for Computing Machinery, New York, NY, USA, 35–39.
LIU, Q., KUSNER, M. J., AND BLUNSOM, P. 2020. A survey on contextual embeddings. arXiv preprint arXiv:2003.07278.
LIU, Y., OTT, M., GOYAL, N., DU, J., JOSHI, M., CHEN, D., LEVY, O., LEWIS, M., ZETTLEMOYER, L., AND STOYANOV, V. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
MA, B., LU, M., TANIGUCHI, Y., AND KONOMI, S. 2021. Courseq: the impact of visual and interactive course recommendation in university environments. Research and Practice in Technology Enhanced Learning 16, 1 (Jun), 18.
MADANI, Y., EZZIKOURI, H., ERRITALI, M., AND HSSINA, B. 2020. Finding optimal pedagogical content in an adaptive e-learning platform using a new recommendation approach and reinforcement learning. Journal of Ambient Intelligence and Humanized Computing 11, 10 (Oct), 3921–3936.
MANRIQUE, R., CUETO-RAMIREZ, F., AND MARIÑO, O. 2018. Comparing graph similarity measures for semantic representations of documents. In Advances in Computing, J. E. Serrano C. and J. C. Martínez-Santos, Eds. Springer International Publishing, Cham, 162–176.
MANRIQUE, R., GRÉVISSE, C., MARIÑO, O., AND ROTHKUGEL, S. 2018. Knowledge graph-based core concept identification in learning resources. In Semantic Technology, R. Ichise, F. Lecue, T. Kawamura, D. Zhao, S. Muggleton, and K. Kozaki, Eds. Springer International Publishing, Cham, 36–51.
MANRIQUE, R. AND MARINO, O. 2018. Knowledge graph-based weighting strategies for a scholarly paper recommendation scenario. In Workshop of Knowledge-aware and Conversational Recommender Systems RecSys, V.W. Anelli, T. D. Noia, P. Lops, C. Musto, M. Zanker, P. Basile, D. Bridge, and F. Narducci, Eds. KaRS’18. RecSys, Vancouver, Canada.
MANRIQUE PIRAMANRIQUE, R. F. 2019. Towards automatic learning resources organization via knowledge graphs. Ph.D. thesis, Universidad de los Andes.
MARINO, O. AND PAQUETTE, G. 2010. A competency—driven advisor system for multi-actor learning environments. Procedia Computer Science 1, 2, 2871–2876. Proceedings of the 1st Workshop on Recommender Systems for Technology Enhanced Learning (RecSysTEL 2010).
MAWANE, J., NAJI, A., AND RAMDANI, M. 2020. Unsupervised deep collaborative filtering recommender system for e-learning platforms. In Smart Applications and Data Analysis, M. Hamlich, L. Bellatreche, A. Mondal, and C. Ordonez, Eds. Communications in Computer and Information Science 1207, 146–161.
PAN, Z., ZHAO, L., ZHONG, X., AND XIA, Z. 2021. Application of collaborative filtering recommendation algorithm in internet online courses. In Proceedings of the 6th International Conference on Big Data and Computing. ICBDC ’21. Association for Computing Machinery, New York, NY, USA, 142–147.
PAQUETTE, G., MARINO, O., AND BEJAOUI, R. 2021. A new competency ontology for learning environments personalization. Smart Learning Environments 8, 1 (Aug), 16.
PAQUETTE, G., MARIÑO, O., ROGOZAN, D., AND LÉONARD, M. 2015. Competency-based personalization for massive online learning. Smart Learning Environments 2, 1 (Feb), 4.
ŘEHÜŘEK, R. AND SOJKA, P. 2010. Software Framework for Topic Modelling with Large Corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. ELRA, Valletta, Malta, 45–50.
REIMERS, N. 2022. Pretrained models — sentence-transformers documentation. https: //www.sbert.net/docs/pretrained_models.html. Accessed: 2022-06-02.
REIMERS, N. AND GUREVYCH, I. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China, 3982–3992.
REIMERS, N. AND GUREVYCH, I. 2020. Making monolingual sentence embeddings multilingual using knowledge distillation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online, 4512–4525.
SANGUINO, J., MANRIQUE, R., MARIÑO, O., LINARES, M., AND CARDOZO, N. 2022. Log mining for course recommendation in limited information scenarios. In Proceedings of the International Conference on Educational Data Mining, A. Mitrovic and N. Bosch, Eds. EDM’22. International Educational Data Mining Society, 430–437.
SARI, A. R., BONK, C. J., AND ZHU, M. 2020. Mooc instructor designs and challenges: what can be learned from existing moocs in Indonesia and Malaysia? Asia Pacific Education Review 21, 1 (Mar), 143–166.
SHANSHAN, S., MINGJIN, G., AND LIJUAN, L. 2021. An improved hybrid ontology-based approach for online learning resource recommendations. Educational Technology Research and Development 69, 5 (Oct), 2637–2661.
SHMUELI, E. 2017. Merlot — a reliable framework for oer. In 2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC). Vol. 2. IEEE, Turin, Italy, 697– 699.
SONG, K., TAN, X., QIN, T., LU, J., AND LIU, T.-Y. 2020. Mpnet: Masked and permuted pre-training for language understanding. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds. Vol. 33. Curran Associates, Inc., 16857–16867.
SOUSA, M. J. AND ROCHA, Á. 2020. Learning analytics measuring impacts on organisational performance. Journal of Grid Computing 18, 3 (Sep), 563–571.
TARUS, J. K., NIU, Z., AND MUSTAFA, G. 2018. Knowledge-based recommendation: a review of ontology-based recommender systems for e-learning. Artificial Intelligence Review 50, 1 (Jun), 21–48.
UDDIN, I., IMRAN, A. S., MUHAMMAD, K., FAYYAZ, N., AND SAJJAD, M. 2021. A systematic mapping review on mooc recommender systems. IEEE Access 9, 118379–118405.
WANG, C., ZHU, H., ZHU, C., ZHANG, X., CHEN, E., AND XIONG, H. 2020. Personalized employee training course recommendation with career development awareness. In Proceedings of The Web Conference 2020. WWW ’20. Association for Computing Machinery, New York, NY, USA, 1648–1659.
WANG, W., WEI, F., DONG, L., BAO, H., YANG, N., AND ZHOU, M. 2020. Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds. Vol. 33. Curran Associates, Inc., 5776–5788.
WU, L., HE, X., WANG, X., ZHANG, K., AND WANG, M. 2022. A survey on accuracyoriented neural recommendation: From collaborative filtering to information-rich recommendation. IEEE Transactions on Knowledge and Data Engineering, 1–1.
XIAO, J., WANG, M., JIANG, B., AND LI, J. 2018. A personalized recommendation system with combinational algorithm for online learning. Journal of Ambient Intelligence and Humanized Computing 9, 3 (Jun), 667–677.
YIN, S., YANG, K., AND WANG, H. 2020. A mooc courses recommendation system based on learning behaviours. In Proceedings of the ACM Turing Celebration Conference - China. ACM TURC’20. Association for Computing Machinery, New York, NY, USA, 133–137.
ZAKARIA, I., JAMALUDIN, M., ISMAIL, W. S. A. W., AND ARIFIN, N. 2016. Measuring user’s usage intentions on e-learning portal. In Envisioning the Future of Online Learning, J. E. Luaran, J. Sardi, A. Aziz, and N. A. Alias, Eds. Springer Singapore, Singapore, 347–357.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:
- The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
- Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
- The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons 4.0 License (Attribution-Noncommercial-No Derivatives 4.0 International), or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
- Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
- Noncommercial—other users (including Publisher) may not use this Work for commercial purposes;
- No Derivative Works—other users (including Publisher) may not alter, transform, or build upon this Work,with the understanding that any of the above conditions can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
- The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
- Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
- The Author represents and warrants that:
- the Work is the Author’s original work;
- the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
- the Work is not pending review or under consideration by another publisher;
- the Work has not previously been published;
- the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
- the Work contains no libel, invasion of privacy, or other unlawful matter.
- The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.