Toward Data-Driven Design of Educational Courses: A Feasibility Study
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Sep 25, 2016
Rakesh Agrawal
Behzad Golshan
Evangelos Papalexakis
Abstract
A study plan is the choice of concepts and the organization and sequencing of the concepts to be covered in an educational course. While a good study plan is essential for the success of any course offering, the design of study plans currently remains largely a manual task. We present a novel data-driven method, which given a list of concepts can automatically propose candidate plans to cover all the concepts. Our method uses Wikipedia as an external source of knowledge to both identify which concepts should be studied together and how students should move from one group of concepts to another. For our experimental validation, we synthesize study plan for a course defined by a list of concept names from high school physics. Our user study with domain experts finds that our method is able to produce a study plan of high quality.
How to Cite
Agrawal, R., Golshan, B., & Papalexakis, E. (2016). Toward Data-Driven Design of Educational Courses: A Feasibility Study. Journal of Educational Data Mining, 8(1), 1–21. https://doi.org/10.5281/zenodo.3554601
##plugins.themes.bootstrap3.article.details##
Keywords
prerequisites, Wikipedia, instructional planning, physics
References
AGRAWAL, R., JHAVERI, M. H., AND KENTHAPADI, K. 2014. Evaluating educational interventions at scale. In 1st ACM conference on Learning@scale. ACM, 207–208.
AHO, A. V., GAREY, M. R., AND ULLMAN, J. D. 1972. The transitive reduction of a directed graph. SIAM Journal on Computing 1, 2, 131–137.
BANSAL, N., BLUM, A., AND CHAWLA, S. 2002. Correlation clustering. In Machine Learning. 238– 247.
BRUNSKILL, E. 2011. Estimating prerequisite structure from noisy data. In Proceedings of the 4th International Conference on Educational Data Mining. 217–222.
CHANG, H.-S., HSU, H.-J., AND CHEN, K.-T. 2014. Modeling exercise relationships in e-learning: A unified approach. Tech. rep., Institute of Information Science, Taipei,Taiwan.
COATES, M. 2010. Shaping a New Educational Landscape: Exploring possibilities for education in the 21st century. Bloomsbury Publishing.
DAVIS, B. G. 2009. Tools for teaching. John Wiley & Sons.
DESMARAIS, M. C., MALUF, A., AND LIU, J. 1995. User-expertise modeling with empirically derived probabilistic implication networks. User modeling and user-adapted interaction 5, 3-4, 283–315.
DOIGNON, J.-P. 2014. Learning spaces, and how to build them. In Formal Concept Analysis. Springer, 1–14.
FALMAGNE, J.-C. AND DOIGNON, J.-P. 2011. Learning Spaces. Springer.
FINK, L. D. 2013. Creating significant learning experiences: An integrated approach to designing college courses. John Wiley & Sons.
FORTUNATO, S. 2010. Community detection in graphs. Physics Reports 486, 3, 75–174.
GAGNE, R. M. AND BRIGGS, L. J. 1974. Principles of instructional design. Holt, Rinehart & Winston.
GANTER, B., STUMME, G., AND WILLE, R. 2005. Formal concept analysis: Foundations and applications. Springer.
GLENNERSTER, R. AND TAKAVARASHA, K. 2013. Running randomized evaluations: A practical guide. Princeton University Press.
GRAVES, K. 1996. Teachers as course developers. Cambridge University Press.
GRAY, W. AND LEARY, B. 1935. What makes a book readable. University of Chicago Press.
HANUSHEK, E. A. AND WOESSMANN, L. 2007. The role of education quality for economic growth. Policy Research Department Working Paper 4122, World Bank.
HO, T. K. 1998. The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 8, 832–844.
HSIEH, T.-C. AND WANG, T.-I. 2010. A mining-based approach on discovering courses pattern for constructing suitable learning path. Expert systems with applications 37, 6, 4156–4167.
KOPONEN, I. AND PEHKONEN, M. 2010. Coherent knowledge structures of physics represented as concept networks in teacher education. Science & Education 19, 3, 259–282.
KRIPPENDORFF, K. 1970. Estimating the reliability, systematic error and random error of interval data. Educational and Psychological Measurement 30, 1, 61–70.
MCNEIL, J. D. 2014. Contemporary curriculum: In thought and action. John Wiley & Sons.
MEDELYAN, O., MILNE, D., LEGG, C., AND WITTEN, I. H. 2009. Mining meaning from wikipedia. International Journal of Human-Computer Studies 67, 9, 716–754.
NATHAN, M. J. AND KOEDINGER, K. R. 2000. An investigation of teachers' beliefs of students' algebra development. Cognition and Instruction 18, 2, 209–237.
NOUSIAINEN, M. AND KOPONEN, I. 2010. Concept maps representing knowledge of physics: Connecting structure and content in the context of electricity and magnetism. Nordic Studies in Science Education 6, 2, 155–172.
NOVAK, J. D. AND CANAS, A. J. 2008. The theory underlying concept maps and how to construct and use them. Tech. Rep. IHMC CmapTools 2006-01 Rev 01-2008, Florida Institute for Human and Machine Cognition.
OHLAND, M. W., YUHASZ, A. G., AND SILL, B. L. 2004. Identifying and removing a calculus prerequisite as a bottleneck in clemson's general engineering curriculum. Journal of Engineering Education 93, 3, 253–257.
PAAS, F., RENKL, A., AND SWELLER, J. 2003. Cognitive load theory and instructional design: Recent developments. Educational Psychologist 38, 1.
PAVLIK, P. I., CEN, H., WU, L., AND KOEDINGER, K. R. 2008. Using item-type performance covariance to improve the skill model of an existing tutor. In Proceedings of the 1st International Conference on Educational Data Mining. 77–86.
POLLOCK, E., CHANDLER, P., AND SWELLER, J. 2002. Assimilating complex information. Learning and Instruction 12, 1.
PONZETTO, S. P. AND STRUBE, M. 2007. Knowledge derived from wikipedia for computing semantic relatedness. J. Artif. Intell. Res.(JAIR) 30, 181–212.
REICHARDT, J. AND BORNHOLDT, S. 2006. Statistical mechanics of community detection. Physical Review E 74, 1, 016110.
RUIZ-PRIMO, M. A. AND SHAVELSON, R. J. 1996. Problems and issues in the use of concept maps in science assessment. Journal of research in science teaching 33, 6, 569–600.
SCHEINES, R., SILVER, E., AND GOLDIN, I. 2014. Discovering prerequisite relationships among knowledge components. In Proceedings of the 7th International Conference on Educational Data Mining. 355–356.
SILVESTRE, J. 1998. Approximation heuristics and benchmarkings for the MinLA problem. In Algorithms and Experiments (ALEX '98): Building Bridges Between Theory and Applications. 112–128.
SUCHANEK, F. M., KASNECI, G., AND WEIKUM, G. 2008. Yago: A large ontology from wikipedia and wordnet. Web Semantics: Science, Services and Agents on the World Wide Web 6, 3, 203–217.
TALUKDAR, P. P. AND COHEN, W. W. 2012. Crowdsourced comprehension: predicting prerequisite structure in wikipedia. In 7th Workshop on Building Educational Applications Using NLP. Association for Computational Linguistics, 307–315.
VUONG, A., NIXON, T., AND TOWLE, B. 2011. A method for finding prerequisites within a curriculum. In Proceedings of the 4th International Conference on Educational Data Mining. 211–216.
WAUTERS, K., DESMET, P., AND VAN DEN NOORTGATE, W. 2011. Acquiring item difficulty estimates: a collaborative effort of data and judgment. In Proceedings of the 4th International Conference on Educational Data Mining. 121–128.
WITTE, S. AND FAIGLEY, L. 1981. Coherence, cohesion, and writing quality. College Composition and Communication 32, 2.
WORLD-BANK. 1999. Knowledge for Development: World Development Report: 1998/99. Oxford University Press.
AHO, A. V., GAREY, M. R., AND ULLMAN, J. D. 1972. The transitive reduction of a directed graph. SIAM Journal on Computing 1, 2, 131–137.
BANSAL, N., BLUM, A., AND CHAWLA, S. 2002. Correlation clustering. In Machine Learning. 238– 247.
BRUNSKILL, E. 2011. Estimating prerequisite structure from noisy data. In Proceedings of the 4th International Conference on Educational Data Mining. 217–222.
CHANG, H.-S., HSU, H.-J., AND CHEN, K.-T. 2014. Modeling exercise relationships in e-learning: A unified approach. Tech. rep., Institute of Information Science, Taipei,Taiwan.
COATES, M. 2010. Shaping a New Educational Landscape: Exploring possibilities for education in the 21st century. Bloomsbury Publishing.
DAVIS, B. G. 2009. Tools for teaching. John Wiley & Sons.
DESMARAIS, M. C., MALUF, A., AND LIU, J. 1995. User-expertise modeling with empirically derived probabilistic implication networks. User modeling and user-adapted interaction 5, 3-4, 283–315.
DOIGNON, J.-P. 2014. Learning spaces, and how to build them. In Formal Concept Analysis. Springer, 1–14.
FALMAGNE, J.-C. AND DOIGNON, J.-P. 2011. Learning Spaces. Springer.
FINK, L. D. 2013. Creating significant learning experiences: An integrated approach to designing college courses. John Wiley & Sons.
FORTUNATO, S. 2010. Community detection in graphs. Physics Reports 486, 3, 75–174.
GAGNE, R. M. AND BRIGGS, L. J. 1974. Principles of instructional design. Holt, Rinehart & Winston.
GANTER, B., STUMME, G., AND WILLE, R. 2005. Formal concept analysis: Foundations and applications. Springer.
GLENNERSTER, R. AND TAKAVARASHA, K. 2013. Running randomized evaluations: A practical guide. Princeton University Press.
GRAVES, K. 1996. Teachers as course developers. Cambridge University Press.
GRAY, W. AND LEARY, B. 1935. What makes a book readable. University of Chicago Press.
HANUSHEK, E. A. AND WOESSMANN, L. 2007. The role of education quality for economic growth. Policy Research Department Working Paper 4122, World Bank.
HO, T. K. 1998. The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 8, 832–844.
HSIEH, T.-C. AND WANG, T.-I. 2010. A mining-based approach on discovering courses pattern for constructing suitable learning path. Expert systems with applications 37, 6, 4156–4167.
KOPONEN, I. AND PEHKONEN, M. 2010. Coherent knowledge structures of physics represented as concept networks in teacher education. Science & Education 19, 3, 259–282.
KRIPPENDORFF, K. 1970. Estimating the reliability, systematic error and random error of interval data. Educational and Psychological Measurement 30, 1, 61–70.
MCNEIL, J. D. 2014. Contemporary curriculum: In thought and action. John Wiley & Sons.
MEDELYAN, O., MILNE, D., LEGG, C., AND WITTEN, I. H. 2009. Mining meaning from wikipedia. International Journal of Human-Computer Studies 67, 9, 716–754.
NATHAN, M. J. AND KOEDINGER, K. R. 2000. An investigation of teachers' beliefs of students' algebra development. Cognition and Instruction 18, 2, 209–237.
NOUSIAINEN, M. AND KOPONEN, I. 2010. Concept maps representing knowledge of physics: Connecting structure and content in the context of electricity and magnetism. Nordic Studies in Science Education 6, 2, 155–172.
NOVAK, J. D. AND CANAS, A. J. 2008. The theory underlying concept maps and how to construct and use them. Tech. Rep. IHMC CmapTools 2006-01 Rev 01-2008, Florida Institute for Human and Machine Cognition.
OHLAND, M. W., YUHASZ, A. G., AND SILL, B. L. 2004. Identifying and removing a calculus prerequisite as a bottleneck in clemson's general engineering curriculum. Journal of Engineering Education 93, 3, 253–257.
PAAS, F., RENKL, A., AND SWELLER, J. 2003. Cognitive load theory and instructional design: Recent developments. Educational Psychologist 38, 1.
PAVLIK, P. I., CEN, H., WU, L., AND KOEDINGER, K. R. 2008. Using item-type performance covariance to improve the skill model of an existing tutor. In Proceedings of the 1st International Conference on Educational Data Mining. 77–86.
POLLOCK, E., CHANDLER, P., AND SWELLER, J. 2002. Assimilating complex information. Learning and Instruction 12, 1.
PONZETTO, S. P. AND STRUBE, M. 2007. Knowledge derived from wikipedia for computing semantic relatedness. J. Artif. Intell. Res.(JAIR) 30, 181–212.
REICHARDT, J. AND BORNHOLDT, S. 2006. Statistical mechanics of community detection. Physical Review E 74, 1, 016110.
RUIZ-PRIMO, M. A. AND SHAVELSON, R. J. 1996. Problems and issues in the use of concept maps in science assessment. Journal of research in science teaching 33, 6, 569–600.
SCHEINES, R., SILVER, E., AND GOLDIN, I. 2014. Discovering prerequisite relationships among knowledge components. In Proceedings of the 7th International Conference on Educational Data Mining. 355–356.
SILVESTRE, J. 1998. Approximation heuristics and benchmarkings for the MinLA problem. In Algorithms and Experiments (ALEX '98): Building Bridges Between Theory and Applications. 112–128.
SUCHANEK, F. M., KASNECI, G., AND WEIKUM, G. 2008. Yago: A large ontology from wikipedia and wordnet. Web Semantics: Science, Services and Agents on the World Wide Web 6, 3, 203–217.
TALUKDAR, P. P. AND COHEN, W. W. 2012. Crowdsourced comprehension: predicting prerequisite structure in wikipedia. In 7th Workshop on Building Educational Applications Using NLP. Association for Computational Linguistics, 307–315.
VUONG, A., NIXON, T., AND TOWLE, B. 2011. A method for finding prerequisites within a curriculum. In Proceedings of the 4th International Conference on Educational Data Mining. 211–216.
WAUTERS, K., DESMET, P., AND VAN DEN NOORTGATE, W. 2011. Acquiring item difficulty estimates: a collaborative effort of data and judgment. In Proceedings of the 4th International Conference on Educational Data Mining. 121–128.
WITTE, S. AND FAIGLEY, L. 1981. Coherence, cohesion, and writing quality. College Composition and Communication 32, 2.
WORLD-BANK. 1999. Knowledge for Development: World Development Report: 1998/99. Oxford University Press.
Section
EDM 2016 Journal Track
Authors who publish with this journal agree to the following terms:
- The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
- Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
- The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons 4.0 License (Attribution-Noncommercial-No Derivatives 4.0 International), or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
- Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
- Noncommercial—other users (including Publisher) may not use this Work for commercial purposes;
- No Derivative Works—other users (including Publisher) may not alter, transform, or build upon this Work,with the understanding that any of the above conditions can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
- The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
- Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
- The Author represents and warrants that:
- the Work is the Author’s original work;
- the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
- the Work is not pending review or under consideration by another publisher;
- the Work has not previously been published;
- the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
- the Work contains no libel, invasion of privacy, or other unlawful matter.
- The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.