Published Dec 23, 2017
Andreas Muehling


This article presents concept landscapes - a novel way of investigating the state and development of knowledge structures in groups of persons using concept maps. Instead of focusing on the assessment and evaluation of single maps, the data of many persons is aggregated and data mining approaches are used in analysis. New insights into the ``shared'' knowledge of groups of learners are possible in this way. Electronic collection of concept maps makes it feasible to aggregate the data of a large group of persons, which in turn favors a data mining approach to the analysis. The educational theories underlying the approach, the definition of concept landscapes, and the accompanying analysis methods are presented. Cluster analysis and Pathfinder networks are used on the aggregated data, allowing new insights into the structural configuration of the knowledge of learners. Two real-world research projects serve as case studies for experimental results. The data structures and analysis methods necessary for working with concept landscapes have been implemented as a GNU-R package that is freely available.


AKAIKE, H. 1974. A new look at the statistical model identification. IEEE Transactions on Automatic Control 19, 6, 716–723.

ALBERT, D. AND STEINER, C. 2005. Empirical validation of concept maps: Preliminary methodological considerations. In Proceedings of the Fifth International Conference on Advanced Learning Technologies,
Kaohsiung, Taiwan, 5-8 July 2005, P. Goodyear, D. G. Sampson, D. J.-T. Yang, Kinshuk, T. Okamoto, R. Hartley, and N.-S. Chen, Eds. IEEE Computer Society Press, Los Alamitos, California, 952–953.

AUSUBEL, D. P. 2000. The Acquisition and Retention of Knowledge: A Cognitive View. Kluwer Academic Publishers, Dordrect, Boston.

BAKER, R. S. J. D. 2010. Data mining. In International Encyclopedia of Education, P. Peterson, Ed. Elsevier, Oxford, 112–118.

BALAKRISHNAN, R. AND RANGANATHAN, K. 2012. A Textbook of Graph Theory, 2 ed. Universitext. Springer, New York.

BARTHOLOMEW, D. J., STEELE, F., MOUSTAKI, I., AND GALBRAITH, J. I. 2008. Analysis of Multivariate Social Science Data, 2nd ed ed. Chapman & Hall/CRC and CRC Press, Boca Raton.

BERGES, M., M¨UHLING, A. M., AND HUBWIESER, P. 2012. The gap between knowledge and ability. In Proceedings of the 12th Koli Calling International Conference on Computing Education Research,
Koli, Finnland, 17-15 November 2012, M.-J. Laakso, Ed. ACM, New York, 126–134.

BISWAS, G. AND SULCER, B. 2010. Visual exploratory data analysis methods to characterize student progress in intelligent learning environments. In Technology for Education (T4E), 2010 International
Conference on. 114–121.

CANAS, A. J., CARFF, R., HILL, G., CARVALHO, M., ARGUEDAS, M., ESKRIDGE, T., LOTT, J., AND CARVAJAL, R. 2005. Concept maps: Integrating knowledge and information visualization. In
Knowledge and Information Visualization, S.-O. Tergan and T. Keller, Eds. Lecture notes in computer science, vol. 3426. Springer, Berlin and Heidelberg, 205–219.

CANAS, A. J. AND NOVAK, J. D. 2006. Re-examining the foundations for effective use of concept maps. In Concept Maps: Theory, Methodology, Technology, A. J. Canas and J. D. Novak, Eds. Vol. 1. Universidad de Costa Rica, San Jos´e and Costa Rica, 494–502.

CANAS, A. J. AND NOVAK, J. D. 2012. Freedom vs. restriction of content and structure during concept mapping - possibilities and limitations for construction and assessment. In Concept Maps: Theory, Methodology, Technology, A. J. Canas, J. D. Novak, and J. Vanhear, Eds. Vol. 2. 247–257.

CLAUSET, A., NEWMAN, M. E. J., AND MOORE, C. 2004. Finding community structure in very large networks. Physical Review E 70, 6, 066111.

COOKE, N. J. 1994. Varieties of knowledge elicitation techniques. International Journal of Human-Computer Studies 41, 6, 801–849.

DALEY, B. J., CONCEICAO, S. C. O., MINA, L., ALTMAN, B. A., BALDOR, M., AND BROWN, J. 2010. Integrative literature review: Concept mapping: A strategy to support the development of practice, research, and theory within human resource development. Human Resource Development
Review 9, 4, 357–384.

DE JONG, T. AND FERGUSON-HESSLER, M. G. 1996. Types and qualities of knowledge. Educational Psychologist 31, 2, 105–113.

DEARHOLT, D. W. AND SCHVANEVELDT, R. W. 1990. Properties of pathfinder netowrks. In Pathfinder Associative Networks, R. W. Schvaneveldt, Ed. Ablex Pub. Corp., Norwood and N.J, 1–30.

DEMPSTER, A. P., LAIRD, N. M., AND RUBIN, D. B. 1977. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society: Series B (Statistical Methodology)
39, 1, 1–38.

DERBENTSEVA, N., SAFAYENI, F., AND CA˜NAS, A. J. 2007. Concept maps: Experiments on dynamic thinking. Journal of Research in Science Teaching 44, 3, 448–465.

FREEMAN, L. C. 1978. Centrality in social networks conceptual clarification. Social Networks 1, 1, 215–239.

GLASER, R. AND BASSOK, M. 1989. Learning theory and the study of instruction. Annual Review of Psychology 40, 631–666.

GOLDSMITH, T. E. AND JOHNSON, P. J. 1990. A structural assessment of classroom learning. In Pathfinder Associative Networks, R.W. Schvaneveldt, Ed. Ablex Pub. Corp., Norwood and N.J, 240–

GOLDSTONE, R. L. AND KERSTEN, A. 2003. Concepts and categorization. In Handbook of Psychology, I. B. Weiner, Ed. Vol. 4. John Wiley, Hoboken, 599–621.

GORDON, A. D. 1999. Classification, 2 ed. Monographs on statistics and applied probability, vol. 82. Chapman & Hall/CRC, Boca Raton.

GOULI, E. 2007. Concept mapping in didactics of informatics. assessment as a tool for learning in webbased and adaptive educational environments. Ph.D. thesis, National and Kapodistrian University of Athens, Athen.

GRUNDSPENKIS, J. AND STRAUTMANE, M. 2010. Usage of graph patterns for knowledge assessment based on concept maps. Scientific Journal of Riga Technical University. Computer Sciences 38, 39,

GURLITT, J. AND RENKL, A. 2010. Prior knowledge activation: How different concept mapping tasks lead to substantial differences in cognitive processes, learning outcomes, and perceived self-efficacy.
Instructional Science 38, 4, 417–433.

HAN, J. AND KAMBER, M. 2010. Data Mining: Concepts and Techniques, 2 ed. The Morgan Kaufmann series in data management systems. Elsevier/Morgan Kaufmann, Amsterdam.

HAY, D. B. AND KINCHIN, I. M. 2006. Using concept maps to reveal conceptual typologies. Education + Training 48, 2/3, 127–142.

HOFFMAN, R. R., SHADBOLT, N. R., BURTON, A. M., AND KLEIN, G. 1995. Eliciting knowledge from experts: A methodological analysis. Organizational Behavior and Human Decision Processes 62, 2,

KARAGIORGI, Y. AND SYMEOU, L. 2005. Translating constructivism into instructional design: Potential and limitations. Educational Technology & Society 8, 1, 17–27.

KAUFMAN, L. AND ROUSSEEUW, P. J. 2005. Finding Groups in Data: An Introduction to Cluster Analysis. Wiley-Interscience paperback series. Wiley, Hoboken and N.J.

KEPPENS, J. AND HAY, D. B. 2008. Concept map assessment for teaching computer programming. Computer Science Education 18, 1, 31–42.

KINCHIN, I. M. 2000. Concept mapping in biology. Journal of Biological Education 34, 2, 61–68.

KINCHIN, I. M. AND CABOT, L. B. 2009. An introduction to concept mapping in dental education: the case of partial denture design. European Journal of Dental Education 13, 1, 20–27.

KINNEBREW, J. S., SEGEDY, J. R., AND BISWAS, G. 2014. Analyzing the temporal evolution of students’ behaviors in open-ended learning environments. Metacognition and Learning 9, 2, 187–215.

KOPONEN, I. T. AND PEHKONEN, M. 2010. Entropy and energy in characterizing the organization of concept maps in learning science. Entropy 12, 7, 1653–1672.

KWON, S. Y. AND CIFUENTES, L. 2009. The comparative effect of individually-constructed vs. collaboratively-constructed computer-based concept maps. Computers & Education 52, 2, 365–375.

LARRAZA-MENDILUZE, E. AND GARAY-VITORIA, N. 2013. Use of concept maps to analyze students’ understanding of the i/o subsystem. In Proceedings of the 13th Koli Calling International Conference
on Computing Education Research, Koli, Finnland, 14-17 November 2013, M.-J. Laakso and Simon, Eds. Koli Calling ’13. ACM, New York, 67–76.

LEAKE, D. B., MAGUITMAN, A., AND REICHHERZER, T. 2005. Understanding knowledge models: Modeling assessment of concept importance in concept maps. In Proceedings of the Twenty-Sixth
Annual Conference of the Cognitive Science Society, Chicago, USA, 4-7 August 2004, K. Forbus, D. Gentner, and T. Regier, Eds. Lawrence Erlbaum Associates, Mahwah and N.J, 785–800.

LEAKE, D. B., REICHHERZER, T., CAN AS, A. J., CARVALHO, M., AND ESKRIDGE, T. 2004. “googling” from a concept map: towards automatic concept-map based query formation. In Concept Maps: Theory, Methodology, Technology, A. J. Canas, J. D. Novak, and F. M. Gonzlez Garca, Eds. Vol. 1. 409–416.

MADHYASTHA, T. AND HUNT, E. 2009. Mining diagnostic assessment data for concept similarity. Journal of Educational Data Mining 1, 1, 72–91.

MCCLURE, J. R., SONAK, B., AND SUEN, H. K. 1999. Concept map assessment of classroom learning: Reliability, validity, and logistical practicality. Journal of Research in Science Teaching 36, 4, 475–

NOVAK, J. D. 2010. Learning, Creating, and Using Knowledge: Concept Maps as Facilitative Tools in Schools and Corporations, 2 ed. Routledge, London.

NOVAK, J. D. AND CANAS, A. J. 2008. The theory underlying concept maps and how to construct and use them: Technical report ihmc cmaptools 2006-01 rev 01-2008,.

NOVAK, J. D. AND CANAS, A. J. 2010. The universality and ubiquitousness of concept maps. In Concept Maps: Making Learning Meaningful, J. Sanchez, A. J. Canas, and J. D. Novak, Eds. Vol. 1. Universidad de Chile, Chile, 1–13.

NOVAK, J. D. AND MUSONDA, D. 1991. A twelve-year longitudinal study of science concept learning. American Educational Research Journal 28, 1, 117–153.

OECD. 2012. PISA 2009 Technical Report. PISA. OECD Publishing, Paris.

OZDEMIR, A. 2005. Analyzing concept maps as an assessment (evaluation) tool in teaching mathematics. Journal of Social Sciences 1, 3, 141–149.

R CORE TEAM. 2013. R: A language and environment for statistical computing.

ROSAS, S. R. AND KANE, M. 2012. Quality and rigor of the concept mapping methodology: A pooled study analysis. Evaluation and Program Planning 35, 2, 236–245.

RUIZ-PRIMO, M. A. 2004. Examining concept maps as an assessment tool. In Concept Maps: Theory, Methodology, Technology, A. J. Canas, J. D. Novak, and F. M. Gonzlez Garca, Eds. Vol. 1. 555–562.

RUIZ-PRIMO, M. A. AND SHAVELSON, R. J. 1996. Problems and issues in the use of concept maps in science assessment. Journal of Research in Science Teaching 33, 6, 569–600.

SABITZER, B. 2011. Neurodidactics: Brain-based ideas for ict and computer science education. The International Journal of Learning 18, 2, 167–177.

SANDERS, K., BOUSTEDT, J., ECKERDAL, A., MCCARTNEY, R., MOSTROM, J. E., THOMAS, L., AND ZANDER, C. 2008. Student understanding of object-oriented programming as expressed in concept maps. ACM Inroads 40, 1, 332–336.

SCHVANEVELDT, R. W., DURSO, F. T., AND DEARHOLT, D. W. 1989. Network structures in proximity data. The Psychology of Learning and Motivation 24, 249–284.

SCHWARZ, G. 1978. Estimating the dimension of a model. Annals of Statistics 6, 2, 461–464.

SHAW, M. L. G. AND WOODWARD, J. B. 1990. Modeling expert knowledge. Knowledge Acquisition 2, 3, 179–206.

SOLOMON, K. O., MEDIN, D. L., AND LYNCH, E. 1999. Concepts do more than categorize. Trends in cognitive sciences 3, 3, 99–105.

SOUSA, D. A. 2009. How the Brain Learns: A Multimedia Kit for Professional Development, 3 ed. Corwin Press, Thousand Oaks and Calif.

SQUIRE, L. R. 1987. Memory and Brain. Oxford University Press, New York.

STIBOR, T. 2008. Discriminating self from non-self with finite mixtures of multivariate bernoulli distributions. In Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation,
Atlanta, USA, July 12-16, 2008. GECCO ’08. ACM, New York.

TARICANI, E. M. AND CLARIANA, R. B. 2006. A technique for automatically scoring open-ended concept maps. Educational Technology Research and Development 54, 1, 65–82.

TRUMPOWER, D. L. AND GOLDSMITH, T. E. 2004. Structural enhancement of learning. Contemporary Educational Psychology 29, 4, 426–446.

TRUMPOWER, D. L., SHARARA, H., AND GOLDSMITH, T. E. 2010. Specificity of structural assessment of knowledge. The Journal of Technology, Learning, and Assessment 8, 5.

VALERIO, A., LEAKE, D. B., AND CANAS, A. J. 2008. Automatic classification of concept maps based on a topological taxonomy and its application to studying features of human-built maps. In Concept
Mapping: Connecting Educators, A. J. Canas, P. Reiska, M. Ahlberg, and J. D. Novak, Eds. Vol. 1. Tallinn University, Estonia, 122–129.

VILLALON, J. J. AND CALVO, R. A. 2008. Concept map mining: A definition and a framework for its evaluation. In Proceedings of the 2008 IEEE/WIC/ACM International Conference onWeb Intelligence
and Intelligent Agent Technology, Sydney, Australia, 9-12 December 2008. Vol. 3. IEEE, 357–360.

VON GLASERSFELD, E. 1989. Constructivism in education. In The International Encyclopedia of Education, T. Husen and T. N. Postlethwaite, Eds. Pergamon, Oxford, 162–163.

WITTROCK, M. C. 1992. Generative learning processes of the brain. Educational Psychologist 27, 4, 531–541.

WOLFE, J. H. 1970. Pattern clustering by multivariate mixture analysis. Multivariate Behavioral Research 5, 329–350.

YIN, Y., VANIDES, J., RUIZ-PRIMO, M. A., AYALA, C. C., AND SHAVELSON, R. J. 2005. Comparisonof two concept-mapping techniques: Implications for scoring, interpretation, and use. Journal ofResearch in Science Teaching 42, 2, 166–184.

YOO, J. S. AND CHO, M.-H. 2012. Mining concept maps to understand university students’ learning. In Proceedings of the 5th International Conference on Educational Data Mining, Chania, Greece,
19-21 June 2012, K. Yacef, O. Zayane, H. Hershkovitz, M. Yudelson, and J. Stamper, Eds. 184–187.