A Model-Based Approach to Predicting Graduate-Level Performance Using Indicators of Undergraduate-Level Performance
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Oct 28, 2015
Judith Zimmermann
Kay H. Brodersen
Hans R. Heinimann
Joachim M. Buhmann
Abstract
The graduate admissions process is crucial for controlling the quality of higher education, yet, rules-ofthumb and domain-specific experiences often dominate evidence-based approaches. The goal of the present study is to dissect the predictive power of undergraduate performance indicators and their aggregates. We analyze 81 variables in 171 student records from a Bachelor’s and a Master’s program in Computer Science and employ state-of-the-art methods suitable for high-dimensional data-settings. We consider regression models in combination with variable selection and variable aggregation embedded in a double-layered cross-validation loop. Moreover, bootstrapping is employed to identify the importance of explanatory variables. Critically, the data is not confounded by an admission-induced selection bias, which allows us to obtain an unbiased estimate of the predictive value of undergraduatelevel indicators for subsequent performance at the graduate level. Our results show that undergraduatelevel performance can explain 54% of the variance in graduate-level performance. Significantly, we unexpectedly identified the third-year grade point average as the most significant explanatory variable, whose influence exceeds the one of grades earned in challenging first-year courses. Analyzing the structure of the undergraduate program shows that it primarily assesses a single set of student abilities. Finally, our results provide a methodological basis for deriving principled guidelines for admissions committees.
How to Cite
Zimmermann, J., Brodersen, K. H., Heinimann, H. R., & Buhmann, J. M. (2015). A Model-Based Approach to Predicting Graduate-Level Performance Using Indicators of Undergraduate-Level Performance. Journal of Educational Data Mining, 7(3), 151–176. https://doi.org/10.5281/zenodo.3554733
##plugins.themes.bootstrap3.article.details##
Keywords
regression models, double-layered cross-validation, bootstrapping, explanatory variables, predicting performance at the graduate level
References
AGBONLAHO, R. O., AND OFFOR, U. J. 2008. Predicting success in a Master of Information Science degree programme. Education for Information 26, 3-4, 169-190.
AKAIKE, H. 1974. A new look at the statistical model identification. IEEE Transactions on Automatic Control 9, 6, 716-723.
ASTIN, A. W. 1993. What Matters in College?: Four Critical Years Revisited. Jossey-Bass Publishers, San Francisco, CA, USA.
ATKINSON, R. C., AND GEISER, S. 2009. Reflections on a century of college admissions tests. Educational Researcher 38, 9, 665-676.
BACKHAUS, K., ERICHSON, B., PLINKE, W., AND WEIBER, R. 2006. Multivariate Analysemethoden (11th ed.). Springer, Berlin/Heidelberg, Germany.
BAIRD, J.-A. 2011. Why do people appeal Higher Education grades and what can it tell us about the meaning of standards?. Assessment in Education: Principles, Policy & Practice 18, 1, 1-4.
BAKER, R. S. J. D., AND YACEF, K. 2009. The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining 1, 1, 3-17.
BAKER, R. S. J. D., GOWDA, S. M., AND CORBETT, A. T. 2011. Automatically detecting a student’s preparation for future learning: Help use is key. Proceedings of the 4th International Conference on Educational Data Mining, 179-188.
BARON-BOLDT, J., SCHULER, H., AND FUNKE, U. 1988. Prädiktive Validität von Schulabschlussnoten: Eine Metaanalyse. Zeitschrift für Pädagogische Psychologie 2, 79-90.
BERGIN S., AND REILLY, R. 2006. Predicting introductory programming performance: A multiinstitutional multivariate study. Computer Science Education 16, 4, 303-323.
BIRKEL, P. 1978. Mündliche Prüfungen. Kamp, Bochum, Germany.
BOWERS, A. J. 2011. What's in a grade? The multidimensional nature of what teacher-assigned grades assess in high school. Educational Research and Evaluation 17, 3, 141-159.
BREIMAN, L. 2001. Random forests. Machine Learning 45, 1, 5-32.
BREIMAN, L., AND SPECTOR, P. 1992. Submodel selection and evaluation in regression. The XRandom case. International Statistical Review 60, 3, 291-319.
BRIDGEMAN, B., BURTON, N., AND CLINE, F. 2009. A note on presenting what predictive validity numbers mean. Applied Measurement in Education 22, 2, 109-119.
BÜHLMANN, P. AND VAN DE GEER, S. 2011. Statistics for High-Dimensional Data: Methods, Theory and Applications. Springer, Berlin/Heidelberg, Germany.
CAMARA, W. J. 2005. Broadening criteria of college success and the impact of cognitive redictors. In Choosing Students: Higher Education Admissions Tools for the 21st Century, W. J. Camara and E. W. Kimmel (Eds.). Lawrence Erlbaum Associates, Mahwah, NJ, USA, 53-79.
CHO, Y., AND BRIDGEMAN, B. 2012. Relationship of TOEFL iBT® scores to academic performance: Some evidence from American universities. Language Testing 29, 3, 421-442.
CONLEY, D. T. 2005. College Knowledge: What it Really Takes for Students to Succeed and What We Can Do to Get Them Ready. Jossey-Bass, San Francisco, CA, USA.
COSTELLO, A. B., AND OSBORNE, J. W. 2005. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation 10, 7, 173-178.
CRONBACH, L. J. 1951. Coefficient alpha and the internal structure of tests. Psychometrika 16, 297- 334.
CRONBACH, L. J. 1971. Test validation. In Educational Measurement (2nd ed.), R. L. Thorndike (Ed.). American Council on Education, Washington, DC, USA, 443-507.
CUNY, J., AND ASPRAY, W. 2000. Recruitment and retention of women graduate students in computer science and engineering: Results of a workshop organized by the Computing Research Association. SIGCSE Bulletin 34, 168-174.
DAWES, R. 1975. Graduate admission variables and future success. Science 187, 4178, 721-723.
DE FEYTER, T., CAERS, R., VIGNA, C., AND BERINGS, D. 2012. Unraveling the impact of the big five personality traits on academic performance: The moderating and mediating effects of selfefficacy and academic motivation. Learning and Individual Differences 22, 4, 439-448.
DISTEFANO, C. 2009. Understanding and using factor scores: Considerations for the applied researcher. Practical Assessment, Research & Evaluation 14, 20, 1-11.
DOWNEY, M., COLLINS, M., AND BROWNING, W. 2002. Predictors of success in dental hygiene education: A six-year review. Journal of Dental Education 66, 11, 1269-1273.
EACEA. 2012. The European Higher Education Area in 2012: Bologna Process Implementation Report. Education Audiovisual & Culture Executive Agency (EACEA), Belgium.
EFRON, B., AND TIBSHIRANI, R. J. 1994. An Introduction to the Bootstrap. Monographs on Statistics and Applied Probability 57. Chapman and Hall, London, UK.
EVANS, P., AND WEN, F. K. 2007. Does the medical college admission test predict global academic performance in osteopathic medical school? Journal of the American Osteopathic Association 107, 4, 157-162.
FABRIGAR, L. R., WEGENER, D. T., MACCALLUM, R. C., AND STRAHAN, E. J. 1999. Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods 4, 272-299.
FETTER, J. H. 1997. Questions and Admissions: Reflections on 100,000 Admissions Decisions at Stanford. Stanford University Press, Stanford, CA, USA.
FRANK, M., CHEHREGHANI, M., AND BUHMANN, J. M. 2011. The minimum transfer cost principle for model-order selection. Machine Learning and Knowledge Discovery in Databases. Lecture Notes in Computer Science Series 691. Springer, Berlin/Heidelberg, Germany, 423-438.
FREY, K., AND FREY-EILING, A. 2009. Ausgewählte Methoden der Didaktik. vdf UTB, Stuttgart, Germany.
GEORGE, D., AND MALLERY, P. 2011. SPSS for Windows Step by Step: A Simple Guide and Reference, 18.0 Update (11th ed.). Allyn & Bacon / Pearson, Boston, MA, USA.
GUYON, I., AND ELISSEEFF, A. 2003. An introduction to variable and feature selection. Journal of Machine Learning Research 3, 272-299.
GRAHAM, J.G. 1987. English language proficiency and the prediction of academic success. TESOL Quarterly 21, 3, 505-521.
HARTNETT, R.T., AND WILLINGHAM, W.W. 1980. The criterion problem: What measure of success in graduate education? Applied Psychological Measurement 4, 281-291.
HERZOG S. 2006. Estimating student retention and degree-completion time: Decision trees and neural networks vis-à-vis regression. New Directions for Institutional Research 131, 17-33.
KAISER, H. F. 1958. The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23, 187-200.
KANE, M. T. 2013. Validating the interpretations and uses of test scores. Journal of Educational Measurement 50, 1, 1-37.
KEHM, B. M. 2010. The future of the Bologna Process ‒ The Bologna Process of the future. European Journal of Education 45, 4, 529-534.
KLAPP LEKHOLM, A., AND CLIFFORDSON, C. 2008. Discrepancies between school grades and test scores at individual and school level: Effects of gender and family background. Educational Research and Evaluation 14, 2, 181-199.
KOYS, D. 2010. GMAT versus alternatives: Predictive validity evidence from Central Europe and the Middle East. Journal of Education for Business 85, 3, 180-185.
KUNCEL, N. R., ONES, D. S., AND HEZLETT, S. A. 2001. A comprehensive meta-analysis of the predictive validity of the graduate record examinations: Implications for graduate student selection and performance. Psychological Bulletin 127, 162-181.
LANE, J., LANDE, A., AND COCKERTON, T. 2003. Prediction of postgraduate performance from selfefficacy, class of degree and cognitive ability test scores. Journal of Hospitality, Leisure, Sport and Tourism Education 2, 1, 113-118.
LANGFELDT, H.-R., AND FINGERHUT, W. 1974. Empirische Ansätze zur Aufklärung des Konstruktes "Schulleistung". In Leistungsbeurteilung in der Schule, K. Heller (Ed.). Quelle & Meyer, Heidelberg, Germany.
MEINSHAUSEN, N., AND BÜHLMANN, P. 2010. Stability selection (with discussion). Journal of the Royal Statistical Society B 72, 417-473.
MESSICK, S. 1989. Validity. In Educational Measurement (3rd ed.), R. L. Linn (Ed.). Macmillan New York, NY, USA, 13-103.
NEWTON, P. E. 2012. Questioning the consensus definition of validity. Measurement: Interdisciplinary Research and Perspectives 10, 1-2, 110-122.
NUGENT, G., SOH, L.-K., SAMAL, A., AND LANG, J. 2006. A placement test for computer science: Design, implementation, and analysis. Computer Science Education 16, 1, 19-36.
OSWALD, F. L., SCHMITT, N., KIM, B., RAMSAY, L. J., AND GILLESPIE, M. A. 2001. Developing a biodata measure and situational judgment inventory as predictors of college student performance. Journal of Applied Psychology 89, 187-207.
OWENS, M. K. 2007. Executive Education: Predicting Student Success in 22 Executive MBA Programs. GMAC® Report Series, RR-07-02.
PEÑA-AYALA, A. 2014. Review: Educational data mining: A survey and a data mining-based analysis of recent works. Expert Systems with Applications 41, 4, 1432-1462.
POROPAT, A. 2009. A meta-analysis of the five-factor model of personality and academic performance. Psychological Bulletin 135, 322-338.
PRECKEL, D., AND FREY, K. 2004. Ein Überblick über Prädiktoren für Studienerfolg. Retrieved from: http://www.ifvf.ethz.ch/news/index.
RA, J., AND RHEE, K.-J. 2014. Efficiency of selecting important variable for longitudinal data. Psychology 5, 1, 6-11.
RAFFERTY, A. N., DAVENPORT, J., AND BRUNSKILL, E. 2013. Estimating student knowledge from paired interaction data. Proceedings of The 6th International Conference on Educational Data Mining (EDM 2013).
RAMSEIER, E. 1977. Determinanten des Studienerfolgs: Zusammenfassung der Ergebnisse einer Befragung des schweizerischen Immatrikulationsjahrganges 1965 in einer Pfadanalyse. Revue Suisse de Sociologie 3, 3, 57-73.
RAU, W. 2001. Response: To replicate or not to replicate: Is that Schuman's question? Sociology of Education 74, 1, 75-77.
RAU, W., AND DURAND, A. 2000. The academic ethic and college grades: Does hard work help students to "Make the Grade"? Sociology of Education 73, 1, 19-38.
RINDERMANN, H., AND OUBAID, V. 1999. Auswahl von Studienanfängern durch Universitäten - Kriterien, Verfahren und Prognostizierbarkeit des Studienerfolgs. Zeitschrift für Differentielle und Diagnostische Psychologie 20, 3, 172-191.
RITZEN, J. 2010. A Chance for European Universities or: Avoiding the Looming University Crisis in Europe. University Press, Amsterdam, The Netherlands.
ROMERO, C., AND VENTURA, S. 2010. Educational data mining: A review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics 40, 6, 601-618.
ROMERO C., ROMERO J. R., AND VENTURA S. 2014. A survey on pre-processing educational data. In Educational Data Mining. Studies in Computational Intelligence 524, A. Peña-Ayala (Ed.). Springer International Publishing, Switzerland, 29-64.
SCHUMAN, H. 2001. Comment: Students' effort and reward in college settings. Sociology of Education 74, 1, 73-74.
SCHWARZ, G. 1978. Estimating the dimension of a model. The Annals of Statistics 6(2), 461-464.
SCIME, A. 2008. Globalized computing education: Europe and the United States. Computer Science Education 18, 1, 43-64.
SOMMERLA, G. 1976. Zur Praxis der Leistungsfeststellung und Bewertung in der Schule. Westermanns Pädagogische Beiträge 28, 450-461.
SUELLWOLD, F. 1983. Pädagogische Diagnostik. In Enzyklopädie der Psychologie. Themenbereich B: Methodologie und Methoden. Serie II: Psychologische Diagnostik. Band 2: Intelligenz- und Leistungsdiagnostik, L. Michel (Ed.). Hogrefe, Göttingen, Germany, 307-386.
TENT, L. 1969. Die Auslese von Schülern für weiterführende Schulen: Möglichkeiten und Grenzen. Beiträge zur Theorie und Praxis. Hogrefe, Göttingen, Germany.
THEOBALD R., AND FREEMAN, S. 2014. Is it the intervention or the students? Using linear regression to control for student characteristics in undergraduate STEM education research. CBE-Life Sciences Education 13, 41-48.
THURSTONE, L. L. 1947. Multiple-factor Analysis. The University of Chicago Press, Chicago, IL, USA.
TIMER, J. E., AND CLAUSON, M. I. 2010. The use of selective admissions tools to predict students' success in an advanced standing baccalaureate nursing program. Nurse Education Today 31, 6, 601-606.
THORSEN, C. 2014. Dimensions of norm-referenced compulsory school grades and their relative importance for the prediction of upper secondary school grades. Scandinavian Journal of Educational Research 58, 2, 127-146.
THORSEN, C., AND CLIFFORDSON, C. 2012. Teachers’ grade assignment and the predictive validity of criterion-referenced grades. Educational Research and Evaluation 18, 2, 153-172.
TRAPMANN, S., HELL, B., WEIGAND, S., AND SCHULER, H. 2007. Die Validität von Schulnoten zur Vorhersage des Studienerfolgs: Eine Metaanalyse [The validity of school grades for predicting academic success: A metanalysis]. Zeitschrift für Pädagogische Psychologie 21, 11-27.
TRUELL, A. D., ZHAO, J. J., ALEXANDER, M. W., AND HILL, I. B. 2006. Predicting final student performance in a graduate business program: The MBA. Delta Pi Epsilon Journal 488, 3, 144- 152.
VENTURA JR., P. R. 2005. Identifying predictors of success for an objects-first CS1. Computer Science Education 15, 3, 223-243.
WIKSTRÖM, C., WIKSTRÖM, M., AND LYRÉN, P.-E. 2009. Prediction of study success: Should selection instruments measure cognitive or non-cognitive factors? In Assessment for a Creative World. Paper presented at 35th Annual IAEA Conference, Brisbane, Australia, 13-18 September 2009.
WILLINGHAM, W. W. 1974. Predicting Success in Graduate Education. Science 183, 4122, 273-278.
WILLINGHAM, W. W., YOUNG, J. W., AND MORRIS, M. M. 1985. Success in college: The role of personal qualities and academic ability. College Board Publications, NY, USA.
ZOU, H. 2006. The adaptive lasso and its oracle properties. Journal of the American Statistical Association 101, 476, 1418-1429.
AKAIKE, H. 1974. A new look at the statistical model identification. IEEE Transactions on Automatic Control 9, 6, 716-723.
ASTIN, A. W. 1993. What Matters in College?: Four Critical Years Revisited. Jossey-Bass Publishers, San Francisco, CA, USA.
ATKINSON, R. C., AND GEISER, S. 2009. Reflections on a century of college admissions tests. Educational Researcher 38, 9, 665-676.
BACKHAUS, K., ERICHSON, B., PLINKE, W., AND WEIBER, R. 2006. Multivariate Analysemethoden (11th ed.). Springer, Berlin/Heidelberg, Germany.
BAIRD, J.-A. 2011. Why do people appeal Higher Education grades and what can it tell us about the meaning of standards?. Assessment in Education: Principles, Policy & Practice 18, 1, 1-4.
BAKER, R. S. J. D., AND YACEF, K. 2009. The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining 1, 1, 3-17.
BAKER, R. S. J. D., GOWDA, S. M., AND CORBETT, A. T. 2011. Automatically detecting a student’s preparation for future learning: Help use is key. Proceedings of the 4th International Conference on Educational Data Mining, 179-188.
BARON-BOLDT, J., SCHULER, H., AND FUNKE, U. 1988. Prädiktive Validität von Schulabschlussnoten: Eine Metaanalyse. Zeitschrift für Pädagogische Psychologie 2, 79-90.
BERGIN S., AND REILLY, R. 2006. Predicting introductory programming performance: A multiinstitutional multivariate study. Computer Science Education 16, 4, 303-323.
BIRKEL, P. 1978. Mündliche Prüfungen. Kamp, Bochum, Germany.
BOWERS, A. J. 2011. What's in a grade? The multidimensional nature of what teacher-assigned grades assess in high school. Educational Research and Evaluation 17, 3, 141-159.
BREIMAN, L. 2001. Random forests. Machine Learning 45, 1, 5-32.
BREIMAN, L., AND SPECTOR, P. 1992. Submodel selection and evaluation in regression. The XRandom case. International Statistical Review 60, 3, 291-319.
BRIDGEMAN, B., BURTON, N., AND CLINE, F. 2009. A note on presenting what predictive validity numbers mean. Applied Measurement in Education 22, 2, 109-119.
BÜHLMANN, P. AND VAN DE GEER, S. 2011. Statistics for High-Dimensional Data: Methods, Theory and Applications. Springer, Berlin/Heidelberg, Germany.
CAMARA, W. J. 2005. Broadening criteria of college success and the impact of cognitive redictors. In Choosing Students: Higher Education Admissions Tools for the 21st Century, W. J. Camara and E. W. Kimmel (Eds.). Lawrence Erlbaum Associates, Mahwah, NJ, USA, 53-79.
CHO, Y., AND BRIDGEMAN, B. 2012. Relationship of TOEFL iBT® scores to academic performance: Some evidence from American universities. Language Testing 29, 3, 421-442.
CONLEY, D. T. 2005. College Knowledge: What it Really Takes for Students to Succeed and What We Can Do to Get Them Ready. Jossey-Bass, San Francisco, CA, USA.
COSTELLO, A. B., AND OSBORNE, J. W. 2005. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation 10, 7, 173-178.
CRONBACH, L. J. 1951. Coefficient alpha and the internal structure of tests. Psychometrika 16, 297- 334.
CRONBACH, L. J. 1971. Test validation. In Educational Measurement (2nd ed.), R. L. Thorndike (Ed.). American Council on Education, Washington, DC, USA, 443-507.
CUNY, J., AND ASPRAY, W. 2000. Recruitment and retention of women graduate students in computer science and engineering: Results of a workshop organized by the Computing Research Association. SIGCSE Bulletin 34, 168-174.
DAWES, R. 1975. Graduate admission variables and future success. Science 187, 4178, 721-723.
DE FEYTER, T., CAERS, R., VIGNA, C., AND BERINGS, D. 2012. Unraveling the impact of the big five personality traits on academic performance: The moderating and mediating effects of selfefficacy and academic motivation. Learning and Individual Differences 22, 4, 439-448.
DISTEFANO, C. 2009. Understanding and using factor scores: Considerations for the applied researcher. Practical Assessment, Research & Evaluation 14, 20, 1-11.
DOWNEY, M., COLLINS, M., AND BROWNING, W. 2002. Predictors of success in dental hygiene education: A six-year review. Journal of Dental Education 66, 11, 1269-1273.
EACEA. 2012. The European Higher Education Area in 2012: Bologna Process Implementation Report. Education Audiovisual & Culture Executive Agency (EACEA), Belgium.
EFRON, B., AND TIBSHIRANI, R. J. 1994. An Introduction to the Bootstrap. Monographs on Statistics and Applied Probability 57. Chapman and Hall, London, UK.
EVANS, P., AND WEN, F. K. 2007. Does the medical college admission test predict global academic performance in osteopathic medical school? Journal of the American Osteopathic Association 107, 4, 157-162.
FABRIGAR, L. R., WEGENER, D. T., MACCALLUM, R. C., AND STRAHAN, E. J. 1999. Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods 4, 272-299.
FETTER, J. H. 1997. Questions and Admissions: Reflections on 100,000 Admissions Decisions at Stanford. Stanford University Press, Stanford, CA, USA.
FRANK, M., CHEHREGHANI, M., AND BUHMANN, J. M. 2011. The minimum transfer cost principle for model-order selection. Machine Learning and Knowledge Discovery in Databases. Lecture Notes in Computer Science Series 691. Springer, Berlin/Heidelberg, Germany, 423-438.
FREY, K., AND FREY-EILING, A. 2009. Ausgewählte Methoden der Didaktik. vdf UTB, Stuttgart, Germany.
GEORGE, D., AND MALLERY, P. 2011. SPSS for Windows Step by Step: A Simple Guide and Reference, 18.0 Update (11th ed.). Allyn & Bacon / Pearson, Boston, MA, USA.
GUYON, I., AND ELISSEEFF, A. 2003. An introduction to variable and feature selection. Journal of Machine Learning Research 3, 272-299.
GRAHAM, J.G. 1987. English language proficiency and the prediction of academic success. TESOL Quarterly 21, 3, 505-521.
HARTNETT, R.T., AND WILLINGHAM, W.W. 1980. The criterion problem: What measure of success in graduate education? Applied Psychological Measurement 4, 281-291.
HERZOG S. 2006. Estimating student retention and degree-completion time: Decision trees and neural networks vis-à-vis regression. New Directions for Institutional Research 131, 17-33.
KAISER, H. F. 1958. The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23, 187-200.
KANE, M. T. 2013. Validating the interpretations and uses of test scores. Journal of Educational Measurement 50, 1, 1-37.
KEHM, B. M. 2010. The future of the Bologna Process ‒ The Bologna Process of the future. European Journal of Education 45, 4, 529-534.
KLAPP LEKHOLM, A., AND CLIFFORDSON, C. 2008. Discrepancies between school grades and test scores at individual and school level: Effects of gender and family background. Educational Research and Evaluation 14, 2, 181-199.
KOYS, D. 2010. GMAT versus alternatives: Predictive validity evidence from Central Europe and the Middle East. Journal of Education for Business 85, 3, 180-185.
KUNCEL, N. R., ONES, D. S., AND HEZLETT, S. A. 2001. A comprehensive meta-analysis of the predictive validity of the graduate record examinations: Implications for graduate student selection and performance. Psychological Bulletin 127, 162-181.
LANE, J., LANDE, A., AND COCKERTON, T. 2003. Prediction of postgraduate performance from selfefficacy, class of degree and cognitive ability test scores. Journal of Hospitality, Leisure, Sport and Tourism Education 2, 1, 113-118.
LANGFELDT, H.-R., AND FINGERHUT, W. 1974. Empirische Ansätze zur Aufklärung des Konstruktes "Schulleistung". In Leistungsbeurteilung in der Schule, K. Heller (Ed.). Quelle & Meyer, Heidelberg, Germany.
MEINSHAUSEN, N., AND BÜHLMANN, P. 2010. Stability selection (with discussion). Journal of the Royal Statistical Society B 72, 417-473.
MESSICK, S. 1989. Validity. In Educational Measurement (3rd ed.), R. L. Linn (Ed.). Macmillan New York, NY, USA, 13-103.
NEWTON, P. E. 2012. Questioning the consensus definition of validity. Measurement: Interdisciplinary Research and Perspectives 10, 1-2, 110-122.
NUGENT, G., SOH, L.-K., SAMAL, A., AND LANG, J. 2006. A placement test for computer science: Design, implementation, and analysis. Computer Science Education 16, 1, 19-36.
OSWALD, F. L., SCHMITT, N., KIM, B., RAMSAY, L. J., AND GILLESPIE, M. A. 2001. Developing a biodata measure and situational judgment inventory as predictors of college student performance. Journal of Applied Psychology 89, 187-207.
OWENS, M. K. 2007. Executive Education: Predicting Student Success in 22 Executive MBA Programs. GMAC® Report Series, RR-07-02.
PEÑA-AYALA, A. 2014. Review: Educational data mining: A survey and a data mining-based analysis of recent works. Expert Systems with Applications 41, 4, 1432-1462.
POROPAT, A. 2009. A meta-analysis of the five-factor model of personality and academic performance. Psychological Bulletin 135, 322-338.
PRECKEL, D., AND FREY, K. 2004. Ein Überblick über Prädiktoren für Studienerfolg. Retrieved from: http://www.ifvf.ethz.ch/news/index.
RA, J., AND RHEE, K.-J. 2014. Efficiency of selecting important variable for longitudinal data. Psychology 5, 1, 6-11.
RAFFERTY, A. N., DAVENPORT, J., AND BRUNSKILL, E. 2013. Estimating student knowledge from paired interaction data. Proceedings of The 6th International Conference on Educational Data Mining (EDM 2013).
RAMSEIER, E. 1977. Determinanten des Studienerfolgs: Zusammenfassung der Ergebnisse einer Befragung des schweizerischen Immatrikulationsjahrganges 1965 in einer Pfadanalyse. Revue Suisse de Sociologie 3, 3, 57-73.
RAU, W. 2001. Response: To replicate or not to replicate: Is that Schuman's question? Sociology of Education 74, 1, 75-77.
RAU, W., AND DURAND, A. 2000. The academic ethic and college grades: Does hard work help students to "Make the Grade"? Sociology of Education 73, 1, 19-38.
RINDERMANN, H., AND OUBAID, V. 1999. Auswahl von Studienanfängern durch Universitäten - Kriterien, Verfahren und Prognostizierbarkeit des Studienerfolgs. Zeitschrift für Differentielle und Diagnostische Psychologie 20, 3, 172-191.
RITZEN, J. 2010. A Chance for European Universities or: Avoiding the Looming University Crisis in Europe. University Press, Amsterdam, The Netherlands.
ROMERO, C., AND VENTURA, S. 2010. Educational data mining: A review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics 40, 6, 601-618.
ROMERO C., ROMERO J. R., AND VENTURA S. 2014. A survey on pre-processing educational data. In Educational Data Mining. Studies in Computational Intelligence 524, A. Peña-Ayala (Ed.). Springer International Publishing, Switzerland, 29-64.
SCHUMAN, H. 2001. Comment: Students' effort and reward in college settings. Sociology of Education 74, 1, 73-74.
SCHWARZ, G. 1978. Estimating the dimension of a model. The Annals of Statistics 6(2), 461-464.
SCIME, A. 2008. Globalized computing education: Europe and the United States. Computer Science Education 18, 1, 43-64.
SOMMERLA, G. 1976. Zur Praxis der Leistungsfeststellung und Bewertung in der Schule. Westermanns Pädagogische Beiträge 28, 450-461.
SUELLWOLD, F. 1983. Pädagogische Diagnostik. In Enzyklopädie der Psychologie. Themenbereich B: Methodologie und Methoden. Serie II: Psychologische Diagnostik. Band 2: Intelligenz- und Leistungsdiagnostik, L. Michel (Ed.). Hogrefe, Göttingen, Germany, 307-386.
TENT, L. 1969. Die Auslese von Schülern für weiterführende Schulen: Möglichkeiten und Grenzen. Beiträge zur Theorie und Praxis. Hogrefe, Göttingen, Germany.
THEOBALD R., AND FREEMAN, S. 2014. Is it the intervention or the students? Using linear regression to control for student characteristics in undergraduate STEM education research. CBE-Life Sciences Education 13, 41-48.
THURSTONE, L. L. 1947. Multiple-factor Analysis. The University of Chicago Press, Chicago, IL, USA.
TIMER, J. E., AND CLAUSON, M. I. 2010. The use of selective admissions tools to predict students' success in an advanced standing baccalaureate nursing program. Nurse Education Today 31, 6, 601-606.
THORSEN, C. 2014. Dimensions of norm-referenced compulsory school grades and their relative importance for the prediction of upper secondary school grades. Scandinavian Journal of Educational Research 58, 2, 127-146.
THORSEN, C., AND CLIFFORDSON, C. 2012. Teachers’ grade assignment and the predictive validity of criterion-referenced grades. Educational Research and Evaluation 18, 2, 153-172.
TRAPMANN, S., HELL, B., WEIGAND, S., AND SCHULER, H. 2007. Die Validität von Schulnoten zur Vorhersage des Studienerfolgs: Eine Metaanalyse [The validity of school grades for predicting academic success: A metanalysis]. Zeitschrift für Pädagogische Psychologie 21, 11-27.
TRUELL, A. D., ZHAO, J. J., ALEXANDER, M. W., AND HILL, I. B. 2006. Predicting final student performance in a graduate business program: The MBA. Delta Pi Epsilon Journal 488, 3, 144- 152.
VENTURA JR., P. R. 2005. Identifying predictors of success for an objects-first CS1. Computer Science Education 15, 3, 223-243.
WIKSTRÖM, C., WIKSTRÖM, M., AND LYRÉN, P.-E. 2009. Prediction of study success: Should selection instruments measure cognitive or non-cognitive factors? In Assessment for a Creative World. Paper presented at 35th Annual IAEA Conference, Brisbane, Australia, 13-18 September 2009.
WILLINGHAM, W. W. 1974. Predicting Success in Graduate Education. Science 183, 4122, 273-278.
WILLINGHAM, W. W., YOUNG, J. W., AND MORRIS, M. M. 1985. Success in college: The role of personal qualities and academic ability. College Board Publications, NY, USA.
ZOU, H. 2006. The adaptive lasso and its oracle properties. Journal of the American Statistical Association 101, 476, 1418-1429.
Issue
Section
Articles
Authors who publish with this journal agree to the following terms:
- The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
- Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
- The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons 4.0 License (Attribution-Noncommercial-No Derivatives 4.0 International), or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
- Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
- Noncommercial—other users (including Publisher) may not use this Work for commercial purposes;
- No Derivative Works—other users (including Publisher) may not alter, transform, or build upon this Work,with the understanding that any of the above conditions can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
- The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
- Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
- The Author represents and warrants that:
- the Work is the Author’s original work;
- the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
- the Work is not pending review or under consideration by another publisher;
- the Work has not previously been published;
- the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
- the Work contains no libel, invasion of privacy, or other unlawful matter.
- The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.