Investigating Concept Definition and Skill Modeling for Cognitive Diagnosis in Language Learning

Abstract

Language proficiency diagnosis is essential to extract fine-grained information about the linguistic knowledge
states and skill mastery levels of test takers based on their performance on language tests. Different
from comprehensive standardized tests, many language learning apps often revolve around word-level
questions. Therefore, knowledge concepts and linguistic skills are hard to define, and diagnosis must be
well-designed. Traditional approaches are widely applied for modeling knowledge in science or mathematics,
where skills or knowledge concepts are easy to associate with each item. However, only a
few works focus on defining knowledge concepts and skills using linguistic characteristics for language
knowledge proficiency diagnosis. In addressing this, we propose a framework for language proficiency
diagnosis based on neural networks. Specifically, we propose a series of methods based on our framework
that uses different linguistic features to define skills and knowledge concepts in the context of the
language learning task. Experimental results on a real-world second-language learning dataset demonstrate
the effectiveness and interpretability of our framework. We also provide empirical evidence with
comprehensive experiments and analysis to prove that our knowledge concept and skill definitions are
reasonable and critical to the performance of our model.