Using Demographic Data as Predictor Variables: a Questionable Choice
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Jun 21, 2023
Ryan S. Baker
Lief Esbenshade
Jonathan Vitale
Shamya Karumbaiah
https://orcid.org/0000-0002-0342-4875
Abstract
Predictive analytics methods in education are seeing widespread use and are producing increasingly accurate predictions of students’ outcomes. With the increased use of predictive analytics comes increasing concern about fairness for specific subgroups of the population. One approach that has been proposed to increase fairness is using demographic variables directly in models, as predictors. In this paper we explore issues of fairness in the use of demographic variables as predictors of long-term student outcomes, studying the arguments for and against this practice in the contexts where this literature has been published. We analyze arguments for the inclusion of demographic variables, specifically claims that this approach improves model performance and charges that excluding such variables amounts to a form of ‘color-blind’ racism. We also consider arguments against including demographic variables as predictors, including reduced actionability of predictions, risk of reinforcing bias, and limits of categorization. We then discuss how contextual factors of predictive models should influence case-specific decisions for the inclusion or exclusion of demographic variables and discuss the role of proxy variables. We conclude that, on balance, there are greater benefits to fairness if demographic variables are used to validate fairness rather than as predictors within models.
How to Cite
Baker, R. S., Esbenshade, L., Vitale, J., & Karumbaiah, S. (2023). Using Demographic Data as Predictor Variables: a Questionable Choice. Journal of Educational Data Mining, 15(2), 22–52. https://doi.org/10.5281/zenodo.7702628
##plugins.themes.bootstrap3.article.details##
Keywords
predictive analytics, at-risk prediction, demographic variables, algorithmic bias, algorithmic fairness
References
AGARWAL, A., BEYGELZIMER, A., DUDIK, M., LANGFORD, J., AND WALLACH, H. (2018) A reductions approach to fair classification. In Proceedings of the 35th International Conference on Machine Learning, N. Lawrence, Ed. Proceedings of Machine Learning Research, 60-69.
ALLENSWORTH, E. M., AND EASTON, J. Q. (2007). What matters for staying on-track and graduating in Chicago public high schools: A close look at course grades, failures, and attendance in the freshman year. Research Report. Consortium on Chicago School Research.
ALMEDA, M. V., AND BAKER, R. S. (2020) Predicting student participation in STEM careers: The role of affect and engagement during middle school. Journal of Educational Data Mining 12, 2, 33-47.
ANDERSON, H., BOODHWANI, A., AND BAKER, R. (2019) Assessing the fairness of graduation predictions. Poster paper. In Proceedings of the 12th International Conference on Educational Data Mining (EDM 2019), C. F. Lynch, A. Merceron, M. Desmarais, and R. Nkambou, Eds. International Educational Data Mining Society, 488-491.
ANDRUS, M., AND VILLENEUVE, S. (2022). Demographic-reliant algorithmic fairness: Characterizing the risks of demographic data collection in the pursuit of fairness. In Proceedings of the 5th ACM Conference on Fairness, Accountability, and Transparency (FAccT ’22), Association for Computing Machinery, 1709-1721. https://doi.org/10.1145/3531146.3533226
ANGWIN, J., LARSON, J., MATTU, S., AND KIRCHNER, L. (2016). Machine bias. In Ethics of Data and Analytics, Auerbach Publications, 254-264. http://dx.doi.org/10.1201/9781003278290- 37
ARCHER, K. J., AND KIMES, R. V. (2008). Empirical characterization of random forest variable importance measures. Computational Statistics & Data Analysis 52, 4, 2249-2260. https://doi.org/10.1016/j.csda.2007.08.015
ASBELL-CLARKE, J., ROWE, E., BARDAR, E., EAGLE, M., BROWN, R., BAKER, R., BARNES, T., AND EDWARDS, T. (2015). Leveling Up: Measuring and leveraging implicit STEM learning in games. In Proceedings of the 11th Annual Conference on Games+Learning+Society (GLS 2015), K. E. H. Caldwell, Ed. Carnegie Mellon University, 306-313.
BABUTA, A., AND OSWALD, M. (2019). Data analytics and algorithmic bias in policing. The Royal United Services Institute for Defence and Security Studies.
BAKER, R. S., BERNING, A. W., GOWDA, S. M., ZHANG, S., AND HAWN, A. (2020). Predicting K- 12 dropout. Journal of Education for Students Placed at Risk (JESPAR) 25, 1, 28-54. https://doi.org/10.1080/10824669.2019.1670065
BAKER, R. S., AND HAWN, A. (2021). Algorithmic bias in education. International Journal of Artificial Intelligence in Education 32, 1, 1052-1092. https://doi.org/10.1007/s40593-021- 00285-9
BAKER, R. S. J. D., AND ROSSI, L. M. (2013). Assessing the disengaged behavior of learners. In R. Sottilare, A. Graesser, X. Hu, and H. Holden, (Eds.), Design Recommendations for Intelligent Tutoring Systems – Volume 1 – Learner Modeling, U.S. Army Research Lab, 155-166.
BAROCAS, S., AND SELBST, A. D. (2016). Big Data’s Disparate Impact. 104 California Law Review 671, 671-732. http://dx.doi.org/10.2139/ssrn.2477899
BASSOK, D., AND REARDON, S. F. (2013). “Academic redshirting” in kindergarten: Prevalence, patterns, and implications. Educational Evaluation and Policy Analysis 35, 3, 283-297. https://doi.org/10.3102/0162373713482764
BELCHER, D. C., AND HATLEY, R. V. (1994). A dropout prediction model that highlights middle level variables. Research in Middle Level Education 17, 2, 67-78. https://doi.org/10.1080/10825541.1994.11670032
BERK, R., HEIDARI, H., JABBARI, S., KEARNS, M., AND ROTH, A. (2018). Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research 50, 1, 3–44. https://doi.org/10.1177/0049124118782533
BLECH, E. (2001) Race Policy in France. Washington, DC: Brookings. Retrieved January 26, 2023 from https://www.brookings.edu/articles/race-policy-in-france/
BONILLA-SILVA, E. (2006). Racism without racists: Color-blind racism and the persistence of racial inequality in the United States. Rowman & Littlefield Publishers.
BORNSHEUER, J. N., POLONYI, M. A., ANDREWS, M., FORE, B., AND ONWUEGBUZIE, A. J. (2011). The relationship between ninth-grade retention and on-time graduation in a southeast Texas high school. Journal of At-Risk Issues 16, 2, 9-16.
BOWERS, A. J. (2021). Early warning systems and indicators of dropping out of upper secondary school: The emerging role of digital technologies. OECD Digital Education Outlook 2021: Pushing the Frontiers with AI, Blockchain and Robots, 173-194.
BOWERS, A. J., SPROTT, R., AND TAFF, S. A. (2012). Do we know who will drop out? A review of the predictors of dropping out of high school: Precision, sensitivity, and specificity. The High School Journal 96, 2, 77-100.
BOYCE, J., AND BOWERS, A. J. (2016). Principal turnover: Are there different types of principals who move from or leave their schools? A latent class analysis of the 2007–2008 schools and staffing survey and the 2008–2009 principal follow-up survey. Leadership and Policy in Schools 15, 3, 237-272. https://doi.org/10.1080/15700763.2015.1047033
BREST, P., AND OSHIGE, M. (1995). Affirmative action for whom?. Stanford Law Review 47, 5, 855-900. https://doi.org/10.2307/1229177
CATON, S., AND HAAS, C. (2020). Fairness in machine learning: A survey. arXiv preprint arXiv:2010.04053. https://doi.org/10.48550/arXiv.2010.04053
CHOULDECHOVA, A. (2017). Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data 5, 2, 153-163. https://doi.org/10.1089/big.2016.0047
CHRISTIE, S. T., JARRATT, D. C., OLSON, L. A., AND TAIJALA, T. T. (2019). Machine-learned school dropout early warning at scale. In Proceedings of the 12th International Conference on Educational Data Mining (EDM 2019), C. F. Lynch, A. Merceron, M. Desmarais, and R. Nkambou, Eds. International Educational Data Mining Society, 726-731.
COLEMAN, C., BAKER, R., AND STEPHENSON, S. (2019) A better cold-start for early prediction of student at-risk status in new school districts. In Proceedings of the 12th International Conference on Educational Data Mining (EDM 2019), C. F. Lynch, A. Merceron, M. Desmarais, and R. Nkambou, Eds. International Educational Data Mining Society, 732-737.
COOLEY, S., BYRDO, N., WILLIAMS, M., KING, W., BAIRS, S., HAIZLIP, A., AND LOYAL, K. (2021). Our Voice Our Vision. Seattle Public Schools, Seattle WA.
CORBETT, A. (2001). Cognitive computer tutors: Solving the two-sigma problem. In Proceedings of the 8th International Conference on User Modeling, M. Bauer, P. J. Gmytrasiewicz, and J. Vassileva, Eds. Springer, 137-147. https://doi.org/10.1007/3-540-44566-8 25
CORBETT-DAVIES, S., AND GOEL, S. (2018). The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023. https://doi.org/10.48550/arXiv.1808.00023
CORBETT-DAVIES, S., PIERSON, E., FELLER, A., GOEL, S., AND HUQ, A. (2017, August). Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’17), S. Matwin, S. Yu, and F. Farooq, Eds. Association for Computing Machinery, 797-806. https://doi.org/10.1145/3097983.3098095
CORRIN, W., SEPANIK, S., ROSEN, R., AND SHANE, A. (2016). Addressing early warning indicators: Interim impact findings from the Investing in Innovation (i3) evaluation of Diplomas Now. New York, NY: MDRC.
DALIPI, F., IMRAN, A. S., AND KASTRATI, Z. (2018, April). MOOC dropout prediction using machine learning techniques: Review and research challenges. In Proceedings of the 2018 IEEE Global Engineering Education Conference (EDUCON), IEEE, 1007-1014. https://doi.org/10.1109/EDUCON.2018.8363340
DARENSBOURG, A., PEREZ, E., AND BLAKE, J. J. (2010). Overrepresentation of African American males in exclusionary discipline: The role of school-based mental health professionals in dismantling the school to prison pipeline. Journal of African American Males in Education (JAAME) 1, 3, 196-211.
DARLINGTON, R. B. (1971). Another look at “cultural fairness”. Journal of Educational Measurement 8, 2, 71–82. https://doi.org/10.1111/j.1745-3984.1971.tb00908.x
DEE, T. S., AND PENNER, E. K. (2021). My brother's keeper? The impact of targeted educational supports. Journal of Policy Analysis and Management 40, 4, 1171-1196. https://doi.org/10.1002/pam.22328
DEHO, O. B., JOKSIMOVIC, S., LI, J., ZHAN, C., LIU, J., AND LIU, L. (2022). Should learning analytics models include sensitive attributes? Explaining the why. IEEE Transactions on Learning Technologies 15, 6, 1-13. https://doi.org/10.1109/TLT.2022.3226474
DEKKER, G. W., PECHENIZKIY, M., AND VLEESHOUWERS, J. M. (2009). Predicting students drop out: A case study. Proceedings of the Second International Conference on Educational Data Mining (EDM 2009), T. Barnes, M. Desmarais, C. Romero, and S. Ventura, Eds. International Educational Data Mining Society, 41-50.
DESOCIO, J., VANCURA, M., NELSON, L. A., HEWITT, G., KITZMAN, H., AND COLE, R. (2007). Engaging truant adolescents: Results from a multifaceted intervention pilot. Preventing School Failure: Alternative Education for Children and Youth 51, 3, 3-9. https://doi.org/10.3200/PSFL.51.3.3-11
DIGITAL PROMISE (2021). What Do Edtech and AI Have to Do With Racial Bias? Retrieved June 14, 2022 from https://digitalpromise.org/2021/10/28/what-do-edtech-and-ai-have-to-do-with-race-bias/
D’MELLO, S., LEHMAN, B., SULLINS, J., DAIGLE, R., COMBS, R., VOGT, K., PERKINS, L., AND GRAESSER, A. (2010, June). A time for emoting: When affect-sensitivity is and isn’t effective at promoting deep learning. In Proceedings of the 10th International Conference on Intelligent Tutoring Systems (ITS 2010), V. Aleven, J. Kay, and J. Mostow, Eds. Springer, 245-254. https://doi.org/10.1007/978-3-642-13388-6_29
DOMINA, T., PHARRIS-CIUREJ, N., PENNER, A. M., PENNER, E. K., BRUMMET, Q., PORTER, S. R., AND SANABRIA, T. (2018). Is free and reduced-price lunch a valid measure of educational disadvantage? Educational Researcher 47, 9, 539-555. https://doi.org/10.3102/0013189X18797609
ECKER-LYSTER, M., AND NIILEKSELA, C. (2016). Keeping students on track to graduate: A synthesis of school dropout trends, prevention, and intervention initiatives. Journal of At- Risk Issues 19, 2, 24-31
FANCSALI, S., NIXON, T., AND RITTER, S. (2013, July). Optimal and worst-case performance of mastery learning assessment with Bayesian knowledge tracing. In Proceedings of the International Conference on Educational Data Mining (EDM 2013), S. K. D’Mello, R. A. Calvo, and A. Olney, Eds. International Educational Data Mining Society, 35-42.
FEATHERS, T. (2022) College prep software Naviance is selling advertising access to millions of students. The Markup, Jan. 13, 2022. Retrieved June 14, 2022 from https://themarkup.org/machine-learning/2022/01/13/college-prep-software-naviance-isselling- advertising-access-to-millions-of-students
FELDMAN, M., FRIEDLER, S. A., MOELLER, J., SCHEIDEGGER, C., AND S. VENKATASUBRAMANIAN, S. (2015, August). Certifying and removing disparate impact. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’15), L. Cao, and C. Zhang, Eds. Association for Computing Machinery, 259-268. https://doi.org/10.1145/2783258.2783311
FREEMAN, J., SIMONSEN, B., MCCOACH, D. B., SUGAI, G., LOMBARDI, A., AND HORNER, R. (2015). An analysis of the relationship between implementation of school-wide positive behavior interventions and supports and high school dropout rates. The High School Journal 98, 4, 290-315. http://dx.doi.org/10.1353/hsj.2015.0009
GARCIA, M. (2016). Racist in the machine. World Policy Journal 33, 4, 111-117. http://dx.doi.org/10.1215/07402775-3813015
GARDNER, J., AND BROOKS, C. (2018). Dropout model evaluation in MOOCs. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, AAAI Press, 32(1), 7906-7912. https://doi.org/10.1609/aaai.v32i1.11392
GARDNER, J., BROOKS, C., AND BAKER, R. (2019, March). Evaluating the fairness of predictive student models through slicing analysis. In Proceedings of the 9th International Conference on Learning Analytics & Knowledge (LAK ’19), S. Hsiao, J. Cunnigham, K. McCarthy, and G. Lynch, Eds. Association for Computing Machinery, 225-234. https://doi.org/10.1145/3303772.3303791
GAŠEVIĆ, D., DAWSON, S., ROGERS, T., AND GASEVIC, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education 28, 1, 68-84. https://doi.org/10.1016/j.iheduc.2015.10.002
GELMAN, A., FAGAN, J., AND KISS, A. (2007). An analysis of the New York City police department's “stop-and-frisk” policy in the context of claims of racial bias. Journal of the American Statistical Association 102, 479, 813-823. https://doi.org/10.1198/016214506000001040
GERVET, T., KOEDINGER, K., SCHNEIDER, J., AND MITCHELL, T. (2020). When is deep learning the best approach to knowledge tracing?. Journal of Educational Data Mining 12, 3, 31-54. https://doi.org/10.5281/zenodo.4143614
GOLDHABER, D., KOEDEL, C., ÖZEK, U., AND PARSONS, E. (2022). Using longitudinal student mobility to identify at-risk students. AERA Open 8, 1, 1-13. http://dx.doi.org/10.1177/23328584211071090
GOTTFRIED, M., KIRKSEY, J. J., AND FLETCHER, T. L. (2022). Do high school students with a same-race teacher attend class more often?. Educational Evaluation and Policy Analysis 44, 1, 149-169. https://doi.org/10.3102/01623737211032241
HÄFNER, A., OBERST, V., AND STOCK, A. (2014). Avoiding procrastination through time management: An experimental intervention study. Educational Studies 40, 3, 352-360. https://doi.org/10.1080/03055698.2014.899487
HALDEMAN, D. C. (2000). Gender atypical youth: Clinical and social issues. School Psychology Review 29, 2, 192-200. https://doi.org/10.1080/02796015.2000.12086007
HERODOTOU, C., RIENTIES, B., BOROOWA, A., ZDRAHAL, Z., AND HLOSTA, M. (2019). A largescale implementation of predictive learning analytics in higher education: the teachers’ role and perspective. Educational Technology Research and Development 67, 5, 1273-1306. https://doi.org/10.1007/s11423-019-09685-0
HICKS, B., KITTO, K., PAYNE, L., AND BUCKINGHAM SHUM, S. (2022, March). Thinking with causal models: A visual formalism for collaboratively crafting assumptions. In Proceedings of the 12th International Learning Analytics and Knowledge Conference (LAK ’22), A. F. Wise, R. Martinez-Maldonado, I. Hilliger, Eds. Association of Computing Machinery, 250- 259. https://doi.org/10.1145/3506860.3506899
HOLSTEIN, K., AND DOROUDI, S. (2022). Equity and artificial intelligence in education: Will “AIEd” Amplify or Alleviate Inequities in Education? In The Ethics of Artificial Intelligence in Education: Practices, Challenges, and Debates, K. Porayska-Pomsta, and W. Holmes, Eds. Routledge Press. https://doi.org/10.4324/9780429329067
HU, Q., AND RANGWALA, H. (2020). Towards fair educational data mining: A case study on detecting at-risk students. In Proceedings of the 13th International Conference on Educational Data Mining (EDM 2020), A. N. Rafferty, J. Whitehill, C. Romero, and V. Cavalli-Sforza, Eds. International Educational Data Mining Society, 431-437.
HUTCHINSON, B., AND MITCHELL, M. (2019). 50 years of test (un) fairness: Lessons for machine learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* ’19), Association for Computing Machinery, 49-58. https://doi.org/10.1145/3287560.3287600
HUTT, S., GRAFSGAARD, J. F., AND D'MELLO, S. K. (2019, May). Time to scale: Generalizable affect detection for tens of thousands of students across an entire school year. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19), S. Brewster, G. Fitzpatrick, A. Cox, and V. Kostakos, Eds. Association for Computing Machinery, 1-14. https://doi.org/10.1145/3290605.3300726
HYMAN, S., AUBRY, T., AND KLODAWSKY, F. (2011). Resilient educational outcomes: Participation in school by youth with histories of homelessness. Youth & Society 43, 1, 253- 273. https://doi.org/10.1177/0044118X10365354
JACKSON, J. R. (2018). Algorithmic bias. Journal of Leadership, Accountability and Ethics 15, 4, 55-65.
JOHNSON, G. M. (2021). Algorithmic bias: on the implicit biases of social technology. Synthese 198, 10, 9941-9961. https://doi.org/10.1007/s11229-020-02696-y
KARUMBAIAH, S. (2021) The Upstream Sources of Bias: Investigating Theory, Design, and Methods Shaping Adaptive Learning Systems. Doctoral dissertation, University of Pennsylvania.
KAI, S., ANDRES, J. M. L., PAQUETTE, L., BAKER, R. S., MOLNAR, K., WATKINS, H., AND MOORE, M. (2017) Predicting student retention from behavior in an online orientation course. In Proceedings of the 10th International Conference on Educational Data Mining (EDM 2017), X. Hu, T. Barnes, A. Hershkovitz, and L. Paquette, Eds. International Educational Data Mining Society, 250-255.
KIZILCEC, R. F., AND LEE, H. (2022). Algorithmic fairness in education. In W. Holmes & K. Porayska-Pomsta (Eds.), The Ethics of Artificial Intelligence in Education: Practices, Challenges, and Debates. Routledge Press. https://doi.org/10.4324/9780429329067
KLARE, B. F., BURGE, M. J., KLONTZ, J. C., BRUEGGE, R. W. V., AND JAIN, A. K. (2012). Face recognition performance: Role of demographic information. IEEE Transactions on Information Forensics and Security 7, 6, 1789-1801. https://doi.org/10.1109/TIFS.2012.2214212
KLEINBERG, J., MULLAINATHAN, S., AND RAGHAVAN, M. (2017). Inherent trade-offs in the fair determination of risk scores. In Proceedings of the 8th Innovations in Theoretical Computer Science Conference (ITCS 2017), C. H. Papadimitriou, Ed. Schloss Dagstuhl–Leibniz- Zentrum fuer Informatik, 67, 43:1–43:23. https://doi.org/10.4320/LIPIcs.ITCS.2017.43
KLOFT, M., STIEHLER, F., ZHENG, Z., AND PINKWART, N. (2014, October). Predicting MOOC dropout over weeks using machine learning methods. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), A. Moschitti, B. Pang, and W. Daelmanns, Eds. Association for Computational Linguistics, 60-65. http://dx.doi.org/10.3115/v1/W14-4111
KOEDINGER, K. R., STAMPER, J. C., MCLAUGHLIN, E. A., AND NIXON, T. (2013, July). Using data-driven discovery of better student models to improve student learning. In Proceedings of the 16th International Conference on Artificial Intelligence in Education (AIED 2013), H. C. Lane, K. Yacef, J. Mostow, and P. Pavlik, Eds. Springer, 421-430. https://doi.org/10.1007/978-3-642-39112-5_43
KUSNER, M. J., LOFTUS, J., RUSSELL, C., AND SILVA, R. (2017). Counterfactual fairness. In Advances in Neural Information Processing Systems 30 (NIPS 2017), I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds. Curran Associates, Inc.
LEE, H., AND KIZILCEC, R. F. (2020). Evaluation of fairness trade-offs in predicting student success. ArXiv E-Prints, arXiv:2007.00088. https://arxiv.org/abs/2007.00088. Accessed 1 Oct 2021. https://doi.org/10.48550/arXiv.2007.00088
LEE, S. M. (1993). Racial classifications in the US Census: 1890–1990. Ethnic and racial studies, 16(1), 75-94. https://doi.org/10.1080/01419870.1993.9993773
LEE, M., AND SCHUELE, C. M. (2010). Demographics. Encyclopedia of research design, 347- 348.
LEE, S. J. (2012). New talk about ELL students. Phi Delta Kappan 93, 8, 66-69. https://doi.org/10.1177/003172171209300816
LI, Y., ZOU, X., MA, Z., AND BAKER, R. S. (2022) A multi-pronged redesign to reduce gaming the system. In Proceedings of the 23rd International Conference on Artificial Intelligence in Education (AIED 2022), M. M. Rodrigo, N. Matsuda, A. I. Cristea, and V. Dimitrova, Eds. Springer, 334-337. https://doi.org/10.1007/978-3-031-11647-6_64
LOUKINA, A., MADNANI, N., AND ZECHNER, K. (2019). The many dimensions of algorithmic fairness in educational applications. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, H. Yannakoudakis, E. Kochmar, C. Leacock, N. Madnani, I. Pilan, and T. Zesch, Eds. Association for Computational Linguistics, 1–10. http://dx.doi.org/10.18653/v1/W19-4401
LOWRY, S., AND MACPHERSON, G. (1988). A blot on the profession. British medical journal (Clinical research ed.) 296, 6623, 657. https://doi.org/10.1136%2Fbmj.296.6623.657
MATLOFF, N., AND ZHANG, W. (2022). A novel regularization approach to fair ML. arXiv preprint arXiv:2208.06557. https://doi.org/10.48550/arXiv.2208.06557
MILLIRON, M. D., MALCOLM, L., AND KIL, D. (2014). Insight and action analytics: Three case studies to consider. Research & Practice in Assessment 9, 70-89.
MORRISON, T. G., AND KISS, M. (2017). Modern racism scale. Encyclopedia of personality and individual differences, 1-3. https://doi.org/10.1007/978-3-319-28099-8_1251-1
NAGRECHA, S., DILLON, J. Z., AND CHAWLA, N. V. (2017, April). MOOC dropout prediction: lessons learned from making pipelines interpretable. In Proceedings of the 26th International Conference on World Wide Web Companion (WWW ’17 Companion), R. Barrett, R. Cummings, E. Agichtein, and E. Gabrilovich, Eds. International World Wide Web Steering Committee, 351-359. https://doi.org/10.1145/3041021.3054162
NATIONAL ACADEMIES OF SCIENCES, ENGINEERING, AND MEDICINE. 2022. Measuring Sex, Gender Identity, and Sexual Orientation. Washington, DC: The National Academies Press. https://doi.org/10.17226/26424
O’REILLY-SHAH, V. N., GENTRY, K. R., VAN CLEVE, W., KENDALE, S. M., JABALEY, C. S., AND LONG, D. R. (2020). The COVID-19 pandemic highlights shortcomings in US health care informatics infrastructure: A call to action. Anesthesia and analgesia 131, 2, 340-344. http://dx.doi.org/10.1213/ANE.0000000000004945
PANTIC, K., AND CLARKE-MIDURA, J. (2019). Factors that influence retention of women in the computer science major: A systematic literature review. Journal of Women and Minorities in Science and Engineering 25, 2, 119-145. http://dx.doi.org/10.1615/JWomenMinorScienEng.2019024384
PAQUETTE, L., OCUMPAUGH, J., LI, Z., ANDRES, A., AND BAKER, R. (2020). Who’s learning? Using demographics in EDM research. Journal of Educational Data Mining 12, 3, 1–30. https://doi.org/10.5281/zenodo.4143612
PEARL, J., AND MACKENZIE, D. (2018). The Book of Why: The New Science of Cause and Effect. Basic Books.
ROBISON, S., JAGGERS, J., RHODES, J., BLACKMON, B. J., AND CHURCH, W. (2017). Correlates of educational success: Predictors of school dropout and graduation for urban students in the 30 Deep South. Children and Youth Services Review 73, 1, 37-46. https://doi.org/10.1016/j.childyouth.2016.11.031
ROSENTHAL, R., AND JACOBSON, L. (1968). Pygmalion in the classroom. The Urban Review, 3(1), 16-20. http://dx.doi.org/10.1007/BF02322211
SCHWARTZ, S. E., KANCHEWA, S. S., RHODES, J. E., GOWDY, G., STARK, A. M., HORN, J. P., PARNES, M., AND SPENCER, R. (2018). “I'm having a little struggle with this, can you help me out?”: Examining impacts and processes of a social capital intervention for firstgeneration college students. American Journal of Community Psychology 61, 1-2, 166-178. https://doi.org/10.1002/ajcp.12206
SCHWÖBEL, P., AND REMMERS, P. (2022, June). The long arc of fairness: Formalisations and ethical discourse. In Proceedings of the 5th ACM Conference on Fairness, Accountability, and Transparency (FAccT ’22), Association for Computing Machinery, 2179-2188. https://doi.org/10.1145/3531146.3534635
SHOLLENBERGER, T. L. (2015). Racial disparities in school suspension and subsequent outcomes. In D. Losen (Ed.) Closing the School Discipline Gap: Equitable Remedies for Excessive Exclusion, Teachers College Press, 31-43.
SIEGEL, E. (2013). Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die. John Wiley & Sons.
SNIPP, C. M. (2003). Racial measurement in the American census: Past practices and implications for the future. Annual Review of Sociology 29, 1, 563-588. https://doi.org/10.1146/annurev.soc.29.010202.100006
SURESH, H., AND GUTTAG, J. V. (2019). A framework for understanding unintended consequences of machine learning. arXiv preprint arXiv:1901.10002, 2, 8.
VASQUEZ VERDUGO, J., GITIAUX, X., ORTEGA, C., AND RANGWALA, H. (2022, March). FairEd: A systematic fairness analysis approach applied in a higher educational context. In Proceedings of the 12th International Learning Analytics and Knowledge Conference (LAK ’22), A. F. Wise, R. Martinez-Maldonado, I. Hilliger, Eds. Association of Computing Machinery, 271-281. https://doi.org/10.1145/3506860.3506902 VERBERT, K., DUVAL, E., KLERKX, J., GOVAERTS, S., AND SANTOS, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist 57, 10, 1500-1509. https://doi.org/10.1177/0002764213479363
WEIDLICH, J., GAŠEVIĆ, D., AND DRACHSLER, H. (2022). Causal inference and bias in learning analytics: A primer on pitfalls using directed acyclic graphs. Journal of Learning Analytics 9, 3, 183-199. https://doi.org/10.18608/jla.2022.7577
WOLFF, A., ZDRAHAL, Z., NIKOLOV, A., AND PANTUCEK, M. (2013, April). Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment. In Proceedings of the Third International Conference on Learning Analytics and Knowledge (LAK ‘13), D. Suthers, K. Verbert, E. Duval, and X. Ochoa, Eds. Association for Computing Machinery, 145-149. https://doi.org/10.1145/2460296.2460324
WONG, P. H. (2020). Democratizing algorithmic fairness. Philosophy & Technology 33, 225- 244. https://doi.org/10.1007/s13347-019-00355-w
YON, M. (1995). Educating homeless children in the United States. Equity and Excellence in Education 28, 1, 58-62. https://doi.org/10.1080/1066568950280110
YU, R., LEE, H., AND KIZILCEC, R. F. (2021). Should college dropout prediction models include protected attributes?. In Proceedings of the Eighth ACM Conference on Learning @ Scale (L@S ‘21), C. Meinel, M. Perez-Sanagustin, M. Specht, and A. Ogan, Eds. Association for Computing Machinery, 91–100. https://doi.org/10.1145/3430895.3460139
YU, R., LI, Q., FISCHER, C., DOROUDI, S., AND XU, D. (2020). Towards accurate and fair prediction of college success: evaluating different sources of student data. In Proceedings of the 13th International Conference on Educational Data Mining (EDM 2020), A. N. Rafferty, J. Whitehill, C. Romero, and V. Cavalli-Sforza, Eds. International Educational Data Mining Society, 292-301.
ZAFAR, M. B., VALERA, I., GOMEZ-RODRIGUEZ, M., AND GUMMADI, K. P. (2019). Fairness constraints: A flexible approach for fair classification. The Journal of Machine Learning Research 20, 1, 2737-2778
ALLENSWORTH, E. M., AND EASTON, J. Q. (2007). What matters for staying on-track and graduating in Chicago public high schools: A close look at course grades, failures, and attendance in the freshman year. Research Report. Consortium on Chicago School Research.
ALMEDA, M. V., AND BAKER, R. S. (2020) Predicting student participation in STEM careers: The role of affect and engagement during middle school. Journal of Educational Data Mining 12, 2, 33-47.
ANDERSON, H., BOODHWANI, A., AND BAKER, R. (2019) Assessing the fairness of graduation predictions. Poster paper. In Proceedings of the 12th International Conference on Educational Data Mining (EDM 2019), C. F. Lynch, A. Merceron, M. Desmarais, and R. Nkambou, Eds. International Educational Data Mining Society, 488-491.
ANDRUS, M., AND VILLENEUVE, S. (2022). Demographic-reliant algorithmic fairness: Characterizing the risks of demographic data collection in the pursuit of fairness. In Proceedings of the 5th ACM Conference on Fairness, Accountability, and Transparency (FAccT ’22), Association for Computing Machinery, 1709-1721. https://doi.org/10.1145/3531146.3533226
ANGWIN, J., LARSON, J., MATTU, S., AND KIRCHNER, L. (2016). Machine bias. In Ethics of Data and Analytics, Auerbach Publications, 254-264. http://dx.doi.org/10.1201/9781003278290- 37
ARCHER, K. J., AND KIMES, R. V. (2008). Empirical characterization of random forest variable importance measures. Computational Statistics & Data Analysis 52, 4, 2249-2260. https://doi.org/10.1016/j.csda.2007.08.015
ASBELL-CLARKE, J., ROWE, E., BARDAR, E., EAGLE, M., BROWN, R., BAKER, R., BARNES, T., AND EDWARDS, T. (2015). Leveling Up: Measuring and leveraging implicit STEM learning in games. In Proceedings of the 11th Annual Conference on Games+Learning+Society (GLS 2015), K. E. H. Caldwell, Ed. Carnegie Mellon University, 306-313.
BABUTA, A., AND OSWALD, M. (2019). Data analytics and algorithmic bias in policing. The Royal United Services Institute for Defence and Security Studies.
BAKER, R. S., BERNING, A. W., GOWDA, S. M., ZHANG, S., AND HAWN, A. (2020). Predicting K- 12 dropout. Journal of Education for Students Placed at Risk (JESPAR) 25, 1, 28-54. https://doi.org/10.1080/10824669.2019.1670065
BAKER, R. S., AND HAWN, A. (2021). Algorithmic bias in education. International Journal of Artificial Intelligence in Education 32, 1, 1052-1092. https://doi.org/10.1007/s40593-021- 00285-9
BAKER, R. S. J. D., AND ROSSI, L. M. (2013). Assessing the disengaged behavior of learners. In R. Sottilare, A. Graesser, X. Hu, and H. Holden, (Eds.), Design Recommendations for Intelligent Tutoring Systems – Volume 1 – Learner Modeling, U.S. Army Research Lab, 155-166.
BAROCAS, S., AND SELBST, A. D. (2016). Big Data’s Disparate Impact. 104 California Law Review 671, 671-732. http://dx.doi.org/10.2139/ssrn.2477899
BASSOK, D., AND REARDON, S. F. (2013). “Academic redshirting” in kindergarten: Prevalence, patterns, and implications. Educational Evaluation and Policy Analysis 35, 3, 283-297. https://doi.org/10.3102/0162373713482764
BELCHER, D. C., AND HATLEY, R. V. (1994). A dropout prediction model that highlights middle level variables. Research in Middle Level Education 17, 2, 67-78. https://doi.org/10.1080/10825541.1994.11670032
BERK, R., HEIDARI, H., JABBARI, S., KEARNS, M., AND ROTH, A. (2018). Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research 50, 1, 3–44. https://doi.org/10.1177/0049124118782533
BLECH, E. (2001) Race Policy in France. Washington, DC: Brookings. Retrieved January 26, 2023 from https://www.brookings.edu/articles/race-policy-in-france/
BONILLA-SILVA, E. (2006). Racism without racists: Color-blind racism and the persistence of racial inequality in the United States. Rowman & Littlefield Publishers.
BORNSHEUER, J. N., POLONYI, M. A., ANDREWS, M., FORE, B., AND ONWUEGBUZIE, A. J. (2011). The relationship between ninth-grade retention and on-time graduation in a southeast Texas high school. Journal of At-Risk Issues 16, 2, 9-16.
BOWERS, A. J. (2021). Early warning systems and indicators of dropping out of upper secondary school: The emerging role of digital technologies. OECD Digital Education Outlook 2021: Pushing the Frontiers with AI, Blockchain and Robots, 173-194.
BOWERS, A. J., SPROTT, R., AND TAFF, S. A. (2012). Do we know who will drop out? A review of the predictors of dropping out of high school: Precision, sensitivity, and specificity. The High School Journal 96, 2, 77-100.
BOYCE, J., AND BOWERS, A. J. (2016). Principal turnover: Are there different types of principals who move from or leave their schools? A latent class analysis of the 2007–2008 schools and staffing survey and the 2008–2009 principal follow-up survey. Leadership and Policy in Schools 15, 3, 237-272. https://doi.org/10.1080/15700763.2015.1047033
BREST, P., AND OSHIGE, M. (1995). Affirmative action for whom?. Stanford Law Review 47, 5, 855-900. https://doi.org/10.2307/1229177
CATON, S., AND HAAS, C. (2020). Fairness in machine learning: A survey. arXiv preprint arXiv:2010.04053. https://doi.org/10.48550/arXiv.2010.04053
CHOULDECHOVA, A. (2017). Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data 5, 2, 153-163. https://doi.org/10.1089/big.2016.0047
CHRISTIE, S. T., JARRATT, D. C., OLSON, L. A., AND TAIJALA, T. T. (2019). Machine-learned school dropout early warning at scale. In Proceedings of the 12th International Conference on Educational Data Mining (EDM 2019), C. F. Lynch, A. Merceron, M. Desmarais, and R. Nkambou, Eds. International Educational Data Mining Society, 726-731.
COLEMAN, C., BAKER, R., AND STEPHENSON, S. (2019) A better cold-start for early prediction of student at-risk status in new school districts. In Proceedings of the 12th International Conference on Educational Data Mining (EDM 2019), C. F. Lynch, A. Merceron, M. Desmarais, and R. Nkambou, Eds. International Educational Data Mining Society, 732-737.
COOLEY, S., BYRDO, N., WILLIAMS, M., KING, W., BAIRS, S., HAIZLIP, A., AND LOYAL, K. (2021). Our Voice Our Vision. Seattle Public Schools, Seattle WA.
CORBETT, A. (2001). Cognitive computer tutors: Solving the two-sigma problem. In Proceedings of the 8th International Conference on User Modeling, M. Bauer, P. J. Gmytrasiewicz, and J. Vassileva, Eds. Springer, 137-147. https://doi.org/10.1007/3-540-44566-8 25
CORBETT-DAVIES, S., AND GOEL, S. (2018). The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023. https://doi.org/10.48550/arXiv.1808.00023
CORBETT-DAVIES, S., PIERSON, E., FELLER, A., GOEL, S., AND HUQ, A. (2017, August). Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’17), S. Matwin, S. Yu, and F. Farooq, Eds. Association for Computing Machinery, 797-806. https://doi.org/10.1145/3097983.3098095
CORRIN, W., SEPANIK, S., ROSEN, R., AND SHANE, A. (2016). Addressing early warning indicators: Interim impact findings from the Investing in Innovation (i3) evaluation of Diplomas Now. New York, NY: MDRC.
DALIPI, F., IMRAN, A. S., AND KASTRATI, Z. (2018, April). MOOC dropout prediction using machine learning techniques: Review and research challenges. In Proceedings of the 2018 IEEE Global Engineering Education Conference (EDUCON), IEEE, 1007-1014. https://doi.org/10.1109/EDUCON.2018.8363340
DARENSBOURG, A., PEREZ, E., AND BLAKE, J. J. (2010). Overrepresentation of African American males in exclusionary discipline: The role of school-based mental health professionals in dismantling the school to prison pipeline. Journal of African American Males in Education (JAAME) 1, 3, 196-211.
DARLINGTON, R. B. (1971). Another look at “cultural fairness”. Journal of Educational Measurement 8, 2, 71–82. https://doi.org/10.1111/j.1745-3984.1971.tb00908.x
DEE, T. S., AND PENNER, E. K. (2021). My brother's keeper? The impact of targeted educational supports. Journal of Policy Analysis and Management 40, 4, 1171-1196. https://doi.org/10.1002/pam.22328
DEHO, O. B., JOKSIMOVIC, S., LI, J., ZHAN, C., LIU, J., AND LIU, L. (2022). Should learning analytics models include sensitive attributes? Explaining the why. IEEE Transactions on Learning Technologies 15, 6, 1-13. https://doi.org/10.1109/TLT.2022.3226474
DEKKER, G. W., PECHENIZKIY, M., AND VLEESHOUWERS, J. M. (2009). Predicting students drop out: A case study. Proceedings of the Second International Conference on Educational Data Mining (EDM 2009), T. Barnes, M. Desmarais, C. Romero, and S. Ventura, Eds. International Educational Data Mining Society, 41-50.
DESOCIO, J., VANCURA, M., NELSON, L. A., HEWITT, G., KITZMAN, H., AND COLE, R. (2007). Engaging truant adolescents: Results from a multifaceted intervention pilot. Preventing School Failure: Alternative Education for Children and Youth 51, 3, 3-9. https://doi.org/10.3200/PSFL.51.3.3-11
DIGITAL PROMISE (2021). What Do Edtech and AI Have to Do With Racial Bias? Retrieved June 14, 2022 from https://digitalpromise.org/2021/10/28/what-do-edtech-and-ai-have-to-do-with-race-bias/
D’MELLO, S., LEHMAN, B., SULLINS, J., DAIGLE, R., COMBS, R., VOGT, K., PERKINS, L., AND GRAESSER, A. (2010, June). A time for emoting: When affect-sensitivity is and isn’t effective at promoting deep learning. In Proceedings of the 10th International Conference on Intelligent Tutoring Systems (ITS 2010), V. Aleven, J. Kay, and J. Mostow, Eds. Springer, 245-254. https://doi.org/10.1007/978-3-642-13388-6_29
DOMINA, T., PHARRIS-CIUREJ, N., PENNER, A. M., PENNER, E. K., BRUMMET, Q., PORTER, S. R., AND SANABRIA, T. (2018). Is free and reduced-price lunch a valid measure of educational disadvantage? Educational Researcher 47, 9, 539-555. https://doi.org/10.3102/0013189X18797609
ECKER-LYSTER, M., AND NIILEKSELA, C. (2016). Keeping students on track to graduate: A synthesis of school dropout trends, prevention, and intervention initiatives. Journal of At- Risk Issues 19, 2, 24-31
FANCSALI, S., NIXON, T., AND RITTER, S. (2013, July). Optimal and worst-case performance of mastery learning assessment with Bayesian knowledge tracing. In Proceedings of the International Conference on Educational Data Mining (EDM 2013), S. K. D’Mello, R. A. Calvo, and A. Olney, Eds. International Educational Data Mining Society, 35-42.
FEATHERS, T. (2022) College prep software Naviance is selling advertising access to millions of students. The Markup, Jan. 13, 2022. Retrieved June 14, 2022 from https://themarkup.org/machine-learning/2022/01/13/college-prep-software-naviance-isselling- advertising-access-to-millions-of-students
FELDMAN, M., FRIEDLER, S. A., MOELLER, J., SCHEIDEGGER, C., AND S. VENKATASUBRAMANIAN, S. (2015, August). Certifying and removing disparate impact. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’15), L. Cao, and C. Zhang, Eds. Association for Computing Machinery, 259-268. https://doi.org/10.1145/2783258.2783311
FREEMAN, J., SIMONSEN, B., MCCOACH, D. B., SUGAI, G., LOMBARDI, A., AND HORNER, R. (2015). An analysis of the relationship between implementation of school-wide positive behavior interventions and supports and high school dropout rates. The High School Journal 98, 4, 290-315. http://dx.doi.org/10.1353/hsj.2015.0009
GARCIA, M. (2016). Racist in the machine. World Policy Journal 33, 4, 111-117. http://dx.doi.org/10.1215/07402775-3813015
GARDNER, J., AND BROOKS, C. (2018). Dropout model evaluation in MOOCs. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, AAAI Press, 32(1), 7906-7912. https://doi.org/10.1609/aaai.v32i1.11392
GARDNER, J., BROOKS, C., AND BAKER, R. (2019, March). Evaluating the fairness of predictive student models through slicing analysis. In Proceedings of the 9th International Conference on Learning Analytics & Knowledge (LAK ’19), S. Hsiao, J. Cunnigham, K. McCarthy, and G. Lynch, Eds. Association for Computing Machinery, 225-234. https://doi.org/10.1145/3303772.3303791
GAŠEVIĆ, D., DAWSON, S., ROGERS, T., AND GASEVIC, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education 28, 1, 68-84. https://doi.org/10.1016/j.iheduc.2015.10.002
GELMAN, A., FAGAN, J., AND KISS, A. (2007). An analysis of the New York City police department's “stop-and-frisk” policy in the context of claims of racial bias. Journal of the American Statistical Association 102, 479, 813-823. https://doi.org/10.1198/016214506000001040
GERVET, T., KOEDINGER, K., SCHNEIDER, J., AND MITCHELL, T. (2020). When is deep learning the best approach to knowledge tracing?. Journal of Educational Data Mining 12, 3, 31-54. https://doi.org/10.5281/zenodo.4143614
GOLDHABER, D., KOEDEL, C., ÖZEK, U., AND PARSONS, E. (2022). Using longitudinal student mobility to identify at-risk students. AERA Open 8, 1, 1-13. http://dx.doi.org/10.1177/23328584211071090
GOTTFRIED, M., KIRKSEY, J. J., AND FLETCHER, T. L. (2022). Do high school students with a same-race teacher attend class more often?. Educational Evaluation and Policy Analysis 44, 1, 149-169. https://doi.org/10.3102/01623737211032241
HÄFNER, A., OBERST, V., AND STOCK, A. (2014). Avoiding procrastination through time management: An experimental intervention study. Educational Studies 40, 3, 352-360. https://doi.org/10.1080/03055698.2014.899487
HALDEMAN, D. C. (2000). Gender atypical youth: Clinical and social issues. School Psychology Review 29, 2, 192-200. https://doi.org/10.1080/02796015.2000.12086007
HERODOTOU, C., RIENTIES, B., BOROOWA, A., ZDRAHAL, Z., AND HLOSTA, M. (2019). A largescale implementation of predictive learning analytics in higher education: the teachers’ role and perspective. Educational Technology Research and Development 67, 5, 1273-1306. https://doi.org/10.1007/s11423-019-09685-0
HICKS, B., KITTO, K., PAYNE, L., AND BUCKINGHAM SHUM, S. (2022, March). Thinking with causal models: A visual formalism for collaboratively crafting assumptions. In Proceedings of the 12th International Learning Analytics and Knowledge Conference (LAK ’22), A. F. Wise, R. Martinez-Maldonado, I. Hilliger, Eds. Association of Computing Machinery, 250- 259. https://doi.org/10.1145/3506860.3506899
HOLSTEIN, K., AND DOROUDI, S. (2022). Equity and artificial intelligence in education: Will “AIEd” Amplify or Alleviate Inequities in Education? In The Ethics of Artificial Intelligence in Education: Practices, Challenges, and Debates, K. Porayska-Pomsta, and W. Holmes, Eds. Routledge Press. https://doi.org/10.4324/9780429329067
HU, Q., AND RANGWALA, H. (2020). Towards fair educational data mining: A case study on detecting at-risk students. In Proceedings of the 13th International Conference on Educational Data Mining (EDM 2020), A. N. Rafferty, J. Whitehill, C. Romero, and V. Cavalli-Sforza, Eds. International Educational Data Mining Society, 431-437.
HUTCHINSON, B., AND MITCHELL, M. (2019). 50 years of test (un) fairness: Lessons for machine learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* ’19), Association for Computing Machinery, 49-58. https://doi.org/10.1145/3287560.3287600
HUTT, S., GRAFSGAARD, J. F., AND D'MELLO, S. K. (2019, May). Time to scale: Generalizable affect detection for tens of thousands of students across an entire school year. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19), S. Brewster, G. Fitzpatrick, A. Cox, and V. Kostakos, Eds. Association for Computing Machinery, 1-14. https://doi.org/10.1145/3290605.3300726
HYMAN, S., AUBRY, T., AND KLODAWSKY, F. (2011). Resilient educational outcomes: Participation in school by youth with histories of homelessness. Youth & Society 43, 1, 253- 273. https://doi.org/10.1177/0044118X10365354
JACKSON, J. R. (2018). Algorithmic bias. Journal of Leadership, Accountability and Ethics 15, 4, 55-65.
JOHNSON, G. M. (2021). Algorithmic bias: on the implicit biases of social technology. Synthese 198, 10, 9941-9961. https://doi.org/10.1007/s11229-020-02696-y
KARUMBAIAH, S. (2021) The Upstream Sources of Bias: Investigating Theory, Design, and Methods Shaping Adaptive Learning Systems. Doctoral dissertation, University of Pennsylvania.
KAI, S., ANDRES, J. M. L., PAQUETTE, L., BAKER, R. S., MOLNAR, K., WATKINS, H., AND MOORE, M. (2017) Predicting student retention from behavior in an online orientation course. In Proceedings of the 10th International Conference on Educational Data Mining (EDM 2017), X. Hu, T. Barnes, A. Hershkovitz, and L. Paquette, Eds. International Educational Data Mining Society, 250-255.
KIZILCEC, R. F., AND LEE, H. (2022). Algorithmic fairness in education. In W. Holmes & K. Porayska-Pomsta (Eds.), The Ethics of Artificial Intelligence in Education: Practices, Challenges, and Debates. Routledge Press. https://doi.org/10.4324/9780429329067
KLARE, B. F., BURGE, M. J., KLONTZ, J. C., BRUEGGE, R. W. V., AND JAIN, A. K. (2012). Face recognition performance: Role of demographic information. IEEE Transactions on Information Forensics and Security 7, 6, 1789-1801. https://doi.org/10.1109/TIFS.2012.2214212
KLEINBERG, J., MULLAINATHAN, S., AND RAGHAVAN, M. (2017). Inherent trade-offs in the fair determination of risk scores. In Proceedings of the 8th Innovations in Theoretical Computer Science Conference (ITCS 2017), C. H. Papadimitriou, Ed. Schloss Dagstuhl–Leibniz- Zentrum fuer Informatik, 67, 43:1–43:23. https://doi.org/10.4320/LIPIcs.ITCS.2017.43
KLOFT, M., STIEHLER, F., ZHENG, Z., AND PINKWART, N. (2014, October). Predicting MOOC dropout over weeks using machine learning methods. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), A. Moschitti, B. Pang, and W. Daelmanns, Eds. Association for Computational Linguistics, 60-65. http://dx.doi.org/10.3115/v1/W14-4111
KOEDINGER, K. R., STAMPER, J. C., MCLAUGHLIN, E. A., AND NIXON, T. (2013, July). Using data-driven discovery of better student models to improve student learning. In Proceedings of the 16th International Conference on Artificial Intelligence in Education (AIED 2013), H. C. Lane, K. Yacef, J. Mostow, and P. Pavlik, Eds. Springer, 421-430. https://doi.org/10.1007/978-3-642-39112-5_43
KUSNER, M. J., LOFTUS, J., RUSSELL, C., AND SILVA, R. (2017). Counterfactual fairness. In Advances in Neural Information Processing Systems 30 (NIPS 2017), I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds. Curran Associates, Inc.
LEE, H., AND KIZILCEC, R. F. (2020). Evaluation of fairness trade-offs in predicting student success. ArXiv E-Prints, arXiv:2007.00088. https://arxiv.org/abs/2007.00088. Accessed 1 Oct 2021. https://doi.org/10.48550/arXiv.2007.00088
LEE, S. M. (1993). Racial classifications in the US Census: 1890–1990. Ethnic and racial studies, 16(1), 75-94. https://doi.org/10.1080/01419870.1993.9993773
LEE, M., AND SCHUELE, C. M. (2010). Demographics. Encyclopedia of research design, 347- 348.
LEE, S. J. (2012). New talk about ELL students. Phi Delta Kappan 93, 8, 66-69. https://doi.org/10.1177/003172171209300816
LI, Y., ZOU, X., MA, Z., AND BAKER, R. S. (2022) A multi-pronged redesign to reduce gaming the system. In Proceedings of the 23rd International Conference on Artificial Intelligence in Education (AIED 2022), M. M. Rodrigo, N. Matsuda, A. I. Cristea, and V. Dimitrova, Eds. Springer, 334-337. https://doi.org/10.1007/978-3-031-11647-6_64
LOUKINA, A., MADNANI, N., AND ZECHNER, K. (2019). The many dimensions of algorithmic fairness in educational applications. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, H. Yannakoudakis, E. Kochmar, C. Leacock, N. Madnani, I. Pilan, and T. Zesch, Eds. Association for Computational Linguistics, 1–10. http://dx.doi.org/10.18653/v1/W19-4401
LOWRY, S., AND MACPHERSON, G. (1988). A blot on the profession. British medical journal (Clinical research ed.) 296, 6623, 657. https://doi.org/10.1136%2Fbmj.296.6623.657
MATLOFF, N., AND ZHANG, W. (2022). A novel regularization approach to fair ML. arXiv preprint arXiv:2208.06557. https://doi.org/10.48550/arXiv.2208.06557
MILLIRON, M. D., MALCOLM, L., AND KIL, D. (2014). Insight and action analytics: Three case studies to consider. Research & Practice in Assessment 9, 70-89.
MORRISON, T. G., AND KISS, M. (2017). Modern racism scale. Encyclopedia of personality and individual differences, 1-3. https://doi.org/10.1007/978-3-319-28099-8_1251-1
NAGRECHA, S., DILLON, J. Z., AND CHAWLA, N. V. (2017, April). MOOC dropout prediction: lessons learned from making pipelines interpretable. In Proceedings of the 26th International Conference on World Wide Web Companion (WWW ’17 Companion), R. Barrett, R. Cummings, E. Agichtein, and E. Gabrilovich, Eds. International World Wide Web Steering Committee, 351-359. https://doi.org/10.1145/3041021.3054162
NATIONAL ACADEMIES OF SCIENCES, ENGINEERING, AND MEDICINE. 2022. Measuring Sex, Gender Identity, and Sexual Orientation. Washington, DC: The National Academies Press. https://doi.org/10.17226/26424
O’REILLY-SHAH, V. N., GENTRY, K. R., VAN CLEVE, W., KENDALE, S. M., JABALEY, C. S., AND LONG, D. R. (2020). The COVID-19 pandemic highlights shortcomings in US health care informatics infrastructure: A call to action. Anesthesia and analgesia 131, 2, 340-344. http://dx.doi.org/10.1213/ANE.0000000000004945
PANTIC, K., AND CLARKE-MIDURA, J. (2019). Factors that influence retention of women in the computer science major: A systematic literature review. Journal of Women and Minorities in Science and Engineering 25, 2, 119-145. http://dx.doi.org/10.1615/JWomenMinorScienEng.2019024384
PAQUETTE, L., OCUMPAUGH, J., LI, Z., ANDRES, A., AND BAKER, R. (2020). Who’s learning? Using demographics in EDM research. Journal of Educational Data Mining 12, 3, 1–30. https://doi.org/10.5281/zenodo.4143612
PEARL, J., AND MACKENZIE, D. (2018). The Book of Why: The New Science of Cause and Effect. Basic Books.
ROBISON, S., JAGGERS, J., RHODES, J., BLACKMON, B. J., AND CHURCH, W. (2017). Correlates of educational success: Predictors of school dropout and graduation for urban students in the 30 Deep South. Children and Youth Services Review 73, 1, 37-46. https://doi.org/10.1016/j.childyouth.2016.11.031
ROSENTHAL, R., AND JACOBSON, L. (1968). Pygmalion in the classroom. The Urban Review, 3(1), 16-20. http://dx.doi.org/10.1007/BF02322211
SCHWARTZ, S. E., KANCHEWA, S. S., RHODES, J. E., GOWDY, G., STARK, A. M., HORN, J. P., PARNES, M., AND SPENCER, R. (2018). “I'm having a little struggle with this, can you help me out?”: Examining impacts and processes of a social capital intervention for firstgeneration college students. American Journal of Community Psychology 61, 1-2, 166-178. https://doi.org/10.1002/ajcp.12206
SCHWÖBEL, P., AND REMMERS, P. (2022, June). The long arc of fairness: Formalisations and ethical discourse. In Proceedings of the 5th ACM Conference on Fairness, Accountability, and Transparency (FAccT ’22), Association for Computing Machinery, 2179-2188. https://doi.org/10.1145/3531146.3534635
SHOLLENBERGER, T. L. (2015). Racial disparities in school suspension and subsequent outcomes. In D. Losen (Ed.) Closing the School Discipline Gap: Equitable Remedies for Excessive Exclusion, Teachers College Press, 31-43.
SIEGEL, E. (2013). Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die. John Wiley & Sons.
SNIPP, C. M. (2003). Racial measurement in the American census: Past practices and implications for the future. Annual Review of Sociology 29, 1, 563-588. https://doi.org/10.1146/annurev.soc.29.010202.100006
SURESH, H., AND GUTTAG, J. V. (2019). A framework for understanding unintended consequences of machine learning. arXiv preprint arXiv:1901.10002, 2, 8.
VASQUEZ VERDUGO, J., GITIAUX, X., ORTEGA, C., AND RANGWALA, H. (2022, March). FairEd: A systematic fairness analysis approach applied in a higher educational context. In Proceedings of the 12th International Learning Analytics and Knowledge Conference (LAK ’22), A. F. Wise, R. Martinez-Maldonado, I. Hilliger, Eds. Association of Computing Machinery, 271-281. https://doi.org/10.1145/3506860.3506902 VERBERT, K., DUVAL, E., KLERKX, J., GOVAERTS, S., AND SANTOS, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist 57, 10, 1500-1509. https://doi.org/10.1177/0002764213479363
WEIDLICH, J., GAŠEVIĆ, D., AND DRACHSLER, H. (2022). Causal inference and bias in learning analytics: A primer on pitfalls using directed acyclic graphs. Journal of Learning Analytics 9, 3, 183-199. https://doi.org/10.18608/jla.2022.7577
WOLFF, A., ZDRAHAL, Z., NIKOLOV, A., AND PANTUCEK, M. (2013, April). Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment. In Proceedings of the Third International Conference on Learning Analytics and Knowledge (LAK ‘13), D. Suthers, K. Verbert, E. Duval, and X. Ochoa, Eds. Association for Computing Machinery, 145-149. https://doi.org/10.1145/2460296.2460324
WONG, P. H. (2020). Democratizing algorithmic fairness. Philosophy & Technology 33, 225- 244. https://doi.org/10.1007/s13347-019-00355-w
YON, M. (1995). Educating homeless children in the United States. Equity and Excellence in Education 28, 1, 58-62. https://doi.org/10.1080/1066568950280110
YU, R., LEE, H., AND KIZILCEC, R. F. (2021). Should college dropout prediction models include protected attributes?. In Proceedings of the Eighth ACM Conference on Learning @ Scale (L@S ‘21), C. Meinel, M. Perez-Sanagustin, M. Specht, and A. Ogan, Eds. Association for Computing Machinery, 91–100. https://doi.org/10.1145/3430895.3460139
YU, R., LI, Q., FISCHER, C., DOROUDI, S., AND XU, D. (2020). Towards accurate and fair prediction of college success: evaluating different sources of student data. In Proceedings of the 13th International Conference on Educational Data Mining (EDM 2020), A. N. Rafferty, J. Whitehill, C. Romero, and V. Cavalli-Sforza, Eds. International Educational Data Mining Society, 292-301.
ZAFAR, M. B., VALERA, I., GOMEZ-RODRIGUEZ, M., AND GUMMADI, K. P. (2019). Fairness constraints: A flexible approach for fair classification. The Journal of Machine Learning Research 20, 1, 2737-2778
Issue
Section
EDM 2023 Journal Track
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:
- The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
- Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
- The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons 4.0 License (Attribution-Noncommercial-No Derivatives 4.0 International), or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
- Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
- Noncommercial—other users (including Publisher) may not use this Work for commercial purposes;
- No Derivative Works—other users (including Publisher) may not alter, transform, or build upon this Work,with the understanding that any of the above conditions can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
- The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post online a pre-publication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see The Effect of Open Access). Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
- Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
- The Author represents and warrants that:
- the Work is the Author’s original work;
- the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
- the Work is not pending review or under consideration by another publisher;
- the Work has not previously been published;
- the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
- the Work contains no libel, invasion of privacy, or other unlawful matter.
- The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.