A Joint Probabilistic Classification Model of Relevant and Irrelevant Sentences in Mathematical Word Problems
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Dec 1, 2010
Suleyman Cetintas
Luo Si
Yan Ping Xin
Dake Zhang
Joo Young Park
Ron Tzur
Abstract
Estimating the difficulty level of math word problems is an important task for many educational applications. Identification of relevant and irrelevant sentences in math word problems is an important step for calculating the difficulty levels of such problems. This paper addresses a novel application of text categorization to identify two types of sentences in mathematical word problems, namely relevant and irrelevant sentences. A novel joint probabilistic classification model is proposed to estimate the joint probability of classification decisions for all sentences of a math word problem by utilizing the correlation among all sentences along with the correlation between the question sentence and other sentences, and sentence text. The proposed model is compared with i) a SVM classifier which makes independent classification decisions for individual sentences by only using the sentence text and ii) a novel SVM classifier that considers the correlation between the question sentence and other sentences along with the sentence text. An extensive set of experiments demonstrates the effectiveness of the joint probabilistic classification model for identifying relevant and irrelevant sentences as well as the novel SVM classifier that utilizes the correlation between the question sentence and other sentences. Furthermore, empirical results and analysis show that i) it is highly beneficial not to remove stopwords and ii) utilizing part of speech tagging does not make a significant improvement although it has been shown to be effective for the related task of math word problem type classification.
How to Cite
Cetintas, S., Si, L., Xin, Y. P., Zhang, D., Park, J. Y., & Tzur, R. (2010). A Joint Probabilistic Classification Model of Relevant and Irrelevant Sentences in Mathematical Word Problems. Journal of Educational Data Mining, 2(1), 83–101. https://doi.org/10.5281/zenodo.3554741
##plugins.themes.bootstrap3.article.details##
Keywords
text categorization, math word problems, relevant and irrelevant sentences, probabilistic graphical model, support vector machine, stopword removal, part of speech tagging, correlation between sentences
References
ARROYO, I. & WOOLF, B. P. 2003. Students in AWE: changing their role from consumers to producers of ITS content. In Proceedings of the 11 th International AIED Conference, Workshop on Advanced Technologies for Math Education.
ARROYO, I., SCHAPIRA, A. & WOOLF, B. P. 2001. Authoring and sharing word problems with AWE. In Proceedings of the 10 th International AIED Conference.
BAEZA-YATES, R. AND RIBEIRO-NETO, B. 1999. Modern information retrieval. ACM Press Series/Addison Wesley, 75-82.
BEAL, C. R. 2007. On-line tutoring for math achievement testing: A Controlled Evaluation. In Journal of Interactive Online Learning, 6(1):43-55.
BIRCH, M., & BEAL, C. R. 2008. Problem posing in AnimalWatch: an interactive system for student-authored content. In Proceedings of the 21 st International FLAIRS Conference.
BOBROW, D. 1964. Natural Language Input for a Computer Problem Solving System. PhD Thesis, MIT.
BROWN, S. I. & WALTER, M. I. 1990. The art of problem posing. Hillsdale NJ: Lawrence Erlbaum.
CETINTAS, S., SI, L., CHAKRAVARTY, S., AAGARD, H. P., BOWEN, K. 2010. Learning to identify students’ relevant and irrelevant questions in a micro- blogging supported classroom. In Proceedings of ITS-10, the 10 th International Conference on Intelligent Tutoring Systems, 281-284.
CETINTAS, S., SI, L., AAGARD, H. P., BOWEN, K., & CORDOVA-SANCHEZ, M. (To Appear). Micro-blogging in classroom: Classifying students’ relevant and irrelevant questions in a micro-blogging supported classroom. In IEEE Transactions on Learning Technologies (Accepted October 2010).
CETINTAS, S., SI, L., XIN, Y. P., ZHANG, D., PARK, J. Y. 2009. Automatic text categorization of mathematical word problems. In Proceedings of the 22 nd International FLAIRS Conference, 27-32.
COWELL, R. G., DAWID, A. P., LAURITZEN, S. L. & SPIEGELHALTER, D. J. 1999. In Probabilistic networks and expert systems. Springer.
FRAKES, W. and BAEZA-YATES, R. 1992. Information retrieval: data structures and algorithms. Prentice Hall, Englewood Cliffs, NJ.
GOLLUP, J.P., BERTENTHAL, M., LABOV, J., CURTIS, P. C. 2002. Understanding: improving advanced study of mathematics and science in US high schools. National Academy Press, Washington, DC.
HINTON, G. and SEJONWSKI, T. 1986. Learning and relearning in Boltzmann machines. In Rumelhart, editor, Parallel Distributed Processing, 282–317. MIT Press.
HIRASHIMA, T., YOKOYAMA, T, OKAMOTO, M., & TAKEUCHI, A. 2007. Learning by problem-posing as sentence-integration and experimental use. In Proceedings of the 13 th International AIED Conference.
JOACHIMS, T. 1998. Text categorization with support vector machines: learning with many relevant features. In Proceedings of ECML-98, 10 th European Conference on Machine Learning.
JOACHIMS, T. 1999. Making large-scale support vector machine learning practical. MIT Press.
KO, J. and NYBERG, E. and SI, L. 2007. A probabilistic graphical model for joint answer ranking in question answering. In Proceedings of ACM SIGIR-07, 30th ACM SIGIR Conference on Research and Development in Information Retrieval, 343-350.
KOEDINGER, K. R., ANDERSON, J.R., HADLEY, W.H., & MARK, M. A. 1997. Intelligent tutoring goes to school in the big city. In International Journal of Artificial Intelligence in Education, 8, 30-43.
MALETSKY, E. M., ANDREWS, A. G., BURTON, G. M., JOHNSON, H. C., LUCKIE, L. A. 2004. Harcourt Math (Indiana Edition). Chicago: Harcourt.
MARZOCCHI, G. M., LUCANGELI, D., MEO, T. D., FINI, F., CORNOLDI, C. 2002. The disturbing effect of irrelevant information on arithmetic problem solving in inattentive children. In Developmental Psychology, 21(1): 73-92.
MASTROPIERI, M. A. & SCRUGGS, T. E. 2006. The inclusive classroom: Strategies for effective instruction. New Jersey: Prentice Hall.
MINKA, T. 2003. A comparison of numerical optimizers for logistic regression. Unpublished draft.
PASSOLUNGHI, M. C. AND SIEGEL, L. S. 2001. Short-term memory, working memory, and inhibitory control in children with difficulties in arithmetic problem solving. In Journal of Experimental Child Psychology, 80(1): 47-57.
PORTER, M. F. 1980. An algorithm for suffix stripping. In Program: Electronic Library and Information Systems, 14(3):130–137.
RIJSBERGEN, C. J. v. 1979. Information retrieval, 2nd edition. University of Glasgow.
RITTER, S., ANDERSON, J., CYTRYNOWICZ, M., & MEDVEDEVA, O.1998. Authoring content in the PAT algebra tutor. In Journal of Interactive Media in Education, 98(9).
SCOTT, S. and MATWIN, S. 1999. Feature engineering for text classification. In Proceedings of ICML-99, 16th International Conference on Machine Learning.
SEBASTIANI, F. 2002. Machine learning in automated text categorization. In ACM Computing Surveys, 34(1):1–47.
SHIAH, R., MASTROPIERI, M. A., SCRUGGS, T.E., & FULK, B..M. 1995. The effects of computer-assisted instruction on the mathematical problem solving of students with learning disabilities. In Exceptionality. U.S. DEPT. OF EDUCATION. 2006. No Child Left Behind is working. Retrieved November, 2008, from http://www.ed.gov/nclb/overview/importance/nclbworking.html
VERSCHAFFEL, L., GREER, B., CORTE, D. E. 2000. Making sense of word problems. Taylor & Francis.
YANG, Y. & LIU, X. 1999. A re-examination of text categorization methods. In Proceedings of SIGIR-99, 22nd ACM SIGIR Conference on Research and Development in Information Retrieval, 42–49.
ARROYO, I., SCHAPIRA, A. & WOOLF, B. P. 2001. Authoring and sharing word problems with AWE. In Proceedings of the 10 th International AIED Conference.
BAEZA-YATES, R. AND RIBEIRO-NETO, B. 1999. Modern information retrieval. ACM Press Series/Addison Wesley, 75-82.
BEAL, C. R. 2007. On-line tutoring for math achievement testing: A Controlled Evaluation. In Journal of Interactive Online Learning, 6(1):43-55.
BIRCH, M., & BEAL, C. R. 2008. Problem posing in AnimalWatch: an interactive system for student-authored content. In Proceedings of the 21 st International FLAIRS Conference.
BOBROW, D. 1964. Natural Language Input for a Computer Problem Solving System. PhD Thesis, MIT.
BROWN, S. I. & WALTER, M. I. 1990. The art of problem posing. Hillsdale NJ: Lawrence Erlbaum.
CETINTAS, S., SI, L., CHAKRAVARTY, S., AAGARD, H. P., BOWEN, K. 2010. Learning to identify students’ relevant and irrelevant questions in a micro- blogging supported classroom. In Proceedings of ITS-10, the 10 th International Conference on Intelligent Tutoring Systems, 281-284.
CETINTAS, S., SI, L., AAGARD, H. P., BOWEN, K., & CORDOVA-SANCHEZ, M. (To Appear). Micro-blogging in classroom: Classifying students’ relevant and irrelevant questions in a micro-blogging supported classroom. In IEEE Transactions on Learning Technologies (Accepted October 2010).
CETINTAS, S., SI, L., XIN, Y. P., ZHANG, D., PARK, J. Y. 2009. Automatic text categorization of mathematical word problems. In Proceedings of the 22 nd International FLAIRS Conference, 27-32.
COWELL, R. G., DAWID, A. P., LAURITZEN, S. L. & SPIEGELHALTER, D. J. 1999. In Probabilistic networks and expert systems. Springer.
FRAKES, W. and BAEZA-YATES, R. 1992. Information retrieval: data structures and algorithms. Prentice Hall, Englewood Cliffs, NJ.
GOLLUP, J.P., BERTENTHAL, M., LABOV, J., CURTIS, P. C. 2002. Understanding: improving advanced study of mathematics and science in US high schools. National Academy Press, Washington, DC.
HINTON, G. and SEJONWSKI, T. 1986. Learning and relearning in Boltzmann machines. In Rumelhart, editor, Parallel Distributed Processing, 282–317. MIT Press.
HIRASHIMA, T., YOKOYAMA, T, OKAMOTO, M., & TAKEUCHI, A. 2007. Learning by problem-posing as sentence-integration and experimental use. In Proceedings of the 13 th International AIED Conference.
JOACHIMS, T. 1998. Text categorization with support vector machines: learning with many relevant features. In Proceedings of ECML-98, 10 th European Conference on Machine Learning.
JOACHIMS, T. 1999. Making large-scale support vector machine learning practical. MIT Press.
KO, J. and NYBERG, E. and SI, L. 2007. A probabilistic graphical model for joint answer ranking in question answering. In Proceedings of ACM SIGIR-07, 30th ACM SIGIR Conference on Research and Development in Information Retrieval, 343-350.
KOEDINGER, K. R., ANDERSON, J.R., HADLEY, W.H., & MARK, M. A. 1997. Intelligent tutoring goes to school in the big city. In International Journal of Artificial Intelligence in Education, 8, 30-43.
MALETSKY, E. M., ANDREWS, A. G., BURTON, G. M., JOHNSON, H. C., LUCKIE, L. A. 2004. Harcourt Math (Indiana Edition). Chicago: Harcourt.
MARZOCCHI, G. M., LUCANGELI, D., MEO, T. D., FINI, F., CORNOLDI, C. 2002. The disturbing effect of irrelevant information on arithmetic problem solving in inattentive children. In Developmental Psychology, 21(1): 73-92.
MASTROPIERI, M. A. & SCRUGGS, T. E. 2006. The inclusive classroom: Strategies for effective instruction. New Jersey: Prentice Hall.
MINKA, T. 2003. A comparison of numerical optimizers for logistic regression. Unpublished draft.
PASSOLUNGHI, M. C. AND SIEGEL, L. S. 2001. Short-term memory, working memory, and inhibitory control in children with difficulties in arithmetic problem solving. In Journal of Experimental Child Psychology, 80(1): 47-57.
PORTER, M. F. 1980. An algorithm for suffix stripping. In Program: Electronic Library and Information Systems, 14(3):130–137.
RIJSBERGEN, C. J. v. 1979. Information retrieval, 2nd edition. University of Glasgow.
RITTER, S., ANDERSON, J., CYTRYNOWICZ, M., & MEDVEDEVA, O.1998. Authoring content in the PAT algebra tutor. In Journal of Interactive Media in Education, 98(9).
SCOTT, S. and MATWIN, S. 1999. Feature engineering for text classification. In Proceedings of ICML-99, 16th International Conference on Machine Learning.
SEBASTIANI, F. 2002. Machine learning in automated text categorization. In ACM Computing Surveys, 34(1):1–47.
SHIAH, R., MASTROPIERI, M. A., SCRUGGS, T.E., & FULK, B..M. 1995. The effects of computer-assisted instruction on the mathematical problem solving of students with learning disabilities. In Exceptionality. U.S. DEPT. OF EDUCATION. 2006. No Child Left Behind is working. Retrieved November, 2008, from http://www.ed.gov/nclb/overview/importance/nclbworking.html
VERSCHAFFEL, L., GREER, B., CORTE, D. E. 2000. Making sense of word problems. Taylor & Francis.
YANG, Y. & LIU, X. 1999. A re-examination of text categorization methods. In Proceedings of SIGIR-99, 22nd ACM SIGIR Conference on Research and Development in Information Retrieval, 42–49.
Issue
Section
Articles
Authors who publish with this journal agree to the following terms:
- The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
- Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
- The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons 4.0 License (Attribution-Noncommercial-No Derivatives 4.0 International), or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
- Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
- Noncommercial—other users (including Publisher) may not use this Work for commercial purposes;
- No Derivative Works—other users (including Publisher) may not alter, transform, or build upon this Work,with the understanding that any of the above conditions can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
- The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
- Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
- The Author represents and warrants that:
- the Work is the Author’s original work;
- the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
- the Work is not pending review or under consideration by another publisher;
- the Work has not previously been published;
- the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
- the Work contains no libel, invasion of privacy, or other unlawful matter.
- The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.