A Joint Probabilistic Classification Model of Relevant and Irrelevant Sentences in Mathematical Word Problems



Published Dec 1, 2010
Suleyman Cetintas Luo Si Yan Ping Xin Dake Zhang Joo Young Park Ron Tzur


Estimating the difficulty level of math word problems is an important task for many educational applications. Identification of relevant and irrelevant sentences in math word problems is an important step for calculating the difficulty levels of such problems. This paper addresses a novel application of text categorization to identify two types of sentences in mathematical word problems, namely relevant and irrelevant sentences. A novel joint probabilistic classification model is proposed to estimate the joint probability of classification decisions for all sentences of a math word problem by utilizing the correlation among all sentences along with the correlation between the question sentence and other sentences, and sentence text. The proposed model is compared with i) a SVM classifier which makes independent classification decisions for individual sentences by only using the sentence text and ii) a novel SVM classifier that considers the correlation between the question sentence and other sentences along with the sentence text. An extensive set of experiments demonstrates the effectiveness of the joint probabilistic classification model for identifying relevant and irrelevant sentences as well as the novel SVM classifier that utilizes the correlation between the question sentence and other sentences. Furthermore, empirical results and analysis show that i) it is highly beneficial not to remove stopwords and ii) utilizing part of speech tagging does not make a significant improvement although it has been shown to be effective for the related task of math word problem type classification.

How to Cite

Cetintas, S., Si, L., Xin, Y. P., Zhang, D., Park, J. Y., & Tzur, R. (2010). A Joint Probabilistic Classification Model of Relevant and Irrelevant Sentences in Mathematical Word Problems. Journal of Educational Data Mining, 2(1), 83–101. https://doi.org/10.5281/zenodo.3554741
Abstract 467 | PDF Downloads 286



text categorization, math word problems, relevant and irrelevant sentences, probabilistic graphical model, support vector machine, stopword removal, part of speech tagging, correlation between sentences

ARROYO, I. & WOOLF, B. P. 2003. Students in AWE: changing their role from consumers to producers of ITS content. In Proceedings of the 11 th International AIED Conference, Workshop on Advanced Technologies for Math Education.

ARROYO, I., SCHAPIRA, A. & WOOLF, B. P. 2001. Authoring and sharing word problems with AWE. In Proceedings of the 10 th International AIED Conference.

BAEZA-YATES, R. AND RIBEIRO-NETO, B. 1999. Modern information retrieval. ACM Press Series/Addison Wesley, 75-82.

BEAL, C. R. 2007. On-line tutoring for math achievement testing: A Controlled Evaluation. In Journal of Interactive Online Learning, 6(1):43-55.

BIRCH, M., & BEAL, C. R. 2008. Problem posing in AnimalWatch: an interactive system for student-authored content. In Proceedings of the 21 st International FLAIRS Conference.

BOBROW, D. 1964. Natural Language Input for a Computer Problem Solving System. PhD Thesis, MIT.

BROWN, S. I. & WALTER, M. I. 1990. The art of problem posing. Hillsdale NJ: Lawrence Erlbaum.

CETINTAS, S., SI, L., CHAKRAVARTY, S., AAGARD, H. P., BOWEN, K. 2010. Learning to identify students’ relevant and irrelevant questions in a micro- blogging supported classroom. In Proceedings of ITS-10, the 10 th International Conference on Intelligent Tutoring Systems, 281-284.

CETINTAS, S., SI, L., AAGARD, H. P., BOWEN, K., & CORDOVA-SANCHEZ, M. (To Appear). Micro-blogging in classroom: Classifying students’ relevant and irrelevant questions in a micro-blogging supported classroom. In IEEE Transactions on Learning Technologies (Accepted October 2010).

CETINTAS, S., SI, L., XIN, Y. P., ZHANG, D., PARK, J. Y. 2009. Automatic text categorization of mathematical word problems. In Proceedings of the 22 nd International FLAIRS Conference, 27-32.

COWELL, R. G., DAWID, A. P., LAURITZEN, S. L. & SPIEGELHALTER, D. J. 1999. In Probabilistic networks and expert systems. Springer.

FRAKES, W. and BAEZA-YATES, R. 1992. Information retrieval: data structures and algorithms. Prentice Hall, Englewood Cliffs, NJ.

GOLLUP, J.P., BERTENTHAL, M., LABOV, J., CURTIS, P. C. 2002. Understanding: improving advanced study of mathematics and science in US high schools. National Academy Press, Washington, DC.

HINTON, G. and SEJONWSKI, T. 1986. Learning and relearning in Boltzmann machines. In Rumelhart, editor, Parallel Distributed Processing, 282–317. MIT Press.

HIRASHIMA, T., YOKOYAMA, T, OKAMOTO, M., & TAKEUCHI, A. 2007. Learning by problem-posing as sentence-integration and experimental use. In Proceedings of the 13 th International AIED Conference.

JOACHIMS, T. 1998. Text categorization with support vector machines: learning with many relevant features. In Proceedings of ECML-98, 10 th European Conference on Machine Learning.

JOACHIMS, T. 1999. Making large-scale support vector machine learning practical. MIT Press.

KO, J. and NYBERG, E. and SI, L. 2007. A probabilistic graphical model for joint answer ranking in question answering. In Proceedings of ACM SIGIR-07, 30th ACM SIGIR Conference on Research and Development in Information Retrieval, 343-350.

KOEDINGER, K. R., ANDERSON, J.R., HADLEY, W.H., & MARK, M. A. 1997. Intelligent tutoring goes to school in the big city. In International Journal of Artificial Intelligence in Education, 8, 30-43.

MALETSKY, E. M., ANDREWS, A. G., BURTON, G. M., JOHNSON, H. C., LUCKIE, L. A. 2004. Harcourt Math (Indiana Edition). Chicago: Harcourt.

MARZOCCHI, G. M., LUCANGELI, D., MEO, T. D., FINI, F., CORNOLDI, C. 2002. The disturbing effect of irrelevant information on arithmetic problem solving in inattentive children. In Developmental Psychology, 21(1): 73-92.

MASTROPIERI, M. A. & SCRUGGS, T. E. 2006. The inclusive classroom: Strategies for effective instruction. New Jersey: Prentice Hall.

MINKA, T. 2003. A comparison of numerical optimizers for logistic regression. Unpublished draft.

PASSOLUNGHI, M. C. AND SIEGEL, L. S. 2001. Short-term memory, working memory, and inhibitory control in children with difficulties in arithmetic problem solving. In Journal of Experimental Child Psychology, 80(1): 47-57.

PORTER, M. F. 1980. An algorithm for suffix stripping. In Program: Electronic Library and Information Systems, 14(3):130–137.

RIJSBERGEN, C. J. v. 1979. Information retrieval, 2nd edition. University of Glasgow.

RITTER, S., ANDERSON, J., CYTRYNOWICZ, M., & MEDVEDEVA, O.1998. Authoring content in the PAT algebra tutor. In Journal of Interactive Media in Education, 98(9).

SCOTT, S. and MATWIN, S. 1999. Feature engineering for text classification. In Proceedings of ICML-99, 16th International Conference on Machine Learning.

SEBASTIANI, F. 2002. Machine learning in automated text categorization. In ACM Computing Surveys, 34(1):1–47.

SHIAH, R., MASTROPIERI, M. A., SCRUGGS, T.E., & FULK, B..M. 1995. The effects of computer-assisted instruction on the mathematical problem solving of students with learning disabilities. In Exceptionality. U.S. DEPT. OF EDUCATION. 2006. No Child Left Behind is working. Retrieved November, 2008, from http://www.ed.gov/nclb/overview/importance/nclbworking.html

VERSCHAFFEL, L., GREER, B., CORTE, D. E. 2000. Making sense of word problems. Taylor & Francis.

YANG, Y. & LIU, X. 1999. A re-examination of text categorization methods. In Proceedings of SIGIR-99, 22nd ACM SIGIR Conference on Research and Development in Information Retrieval, 42–49.