Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

"This is not a data problem": Algorithms and Power in Public Higher Education in Canada (2403.13969v2)

Published 20 Mar 2024 in cs.HC, cs.AI, and cs.CY

Abstract: Algorithmic decision-making is increasingly being adopted across public higher education. The expansion of data-driven practices by post-secondary institutions has occurred in parallel with the adoption of New Public Management approaches by neoliberal administrations. In this study, we conduct a qualitative analysis of an in-depth ethnographic case study of data and algorithms in use at a public college in Ontario, Canada. We identify the data, algorithms, and outcomes in use at the college. We assess how the college's processes and relationships support those outcomes and the different stakeholders' perceptions of the college's data-driven systems. In addition, we find that the growing reliance on algorithmic decisions leads to increased student surveillance, exacerbation of existing inequities, and the automation of the faculty-student relationship. Finally, we identify a cycle of increased institutional power perpetuated by algorithmic decision-making, and driven by a push towards financial sustainability.

View on arXiv
Definition Search Book Streamline Icon: https://streamlinehq.com
References (70)
  1. Nasir Abdul Jalil and Mikkay Wong Ei Leen. 2021. Learning Analytics in Higher Education: The Student Expectations of Learning Analytics. In 2021 5th International Conference on Education and E-Learning (ICEEL 2021). Association for Computing Machinery, New York, NY, USA, 249–254. https://doi.org/10.1145/3502434.3502463
  2. Ali Alkhatib and Michael Bernstein. 2019. Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Glasgow Scotland Uk, 1–13. https://doi.org/10.1145/3290605.3300760
  3. Street-Level Algorithms and AI in Bureaucratic Decision-Making: A Caseworker Perspective. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (April 2021), 1–23. https://doi.org/10.1145/3449114
  4. Human-centered data science: an introduction. The MIT Press, Cambridge, Massachusetts.
  5. Swati Bajpai and S. Mani. 2017. Big Data in Education and Learning Analytics. TechnoLearn: An International Journal of Educational Technology 7, 1and2 (2017), 45. https://doi.org/10.5958/2249-5223.2017.00005.5
  6. Ruha Benjamin. 2019. Race after technology : abolitionist tools for the New Jim Code. Polity Press, Cambridge, UK. https://librarysearch.library.utoronto.ca/discovery/fulldisplay/alma991106344770306196/01UTORONTO_INST:UTORONTO
  7. Michael Brown and Carrie Klein. 2020. Whose Data? Which Rights? Whose Power? A Policy Discourse Analysis of Student Privacy Policy Documents. The Journal of Higher Education 91, 7 (Nov. 2020), 1149–1178. https://doi.org/10.1080/00221546.2020.1770045 Publisher: Routledge _eprint: https://doi.org/10.1080/00221546.2020.1770045.
  8. Lisa Ruth Brunner. 2023. Higher education institutions as eyes of the state: Canada’s international student compliance regime. Globalisation, Societies and Education 21, 2 (March 2023), 236–251. https://doi.org/10.1080/14767724.2022.2037407 Publisher: Routledge _eprint: https://doi.org/10.1080/14767724.2022.2037407.
  9. Alan Burton-Jones. 1999. Knowledge capitalism: business, work, and learning in the new economy. Oxford University Press, Oxford [England] ;.
  10. Centennial College. 2020a. Academic Plan 2021-2025. Technical Report. Centennial College. https://centennialcollege.widen.net/s/7qmftcr52w/c1_02_sep20_academic_plan_2021-2025_fa_spread
  11. Centennial College. 2020b. Universal Design for Learning. http://www.centennialcollege.ca
  12. Centennial College. 2023. Vision and Mission Statement. http://www.centennialcollege.ca
  13. Who is the ”Human” in Human-Centered Machine Learning: The Case of Predicting Mental Health from Social Media. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (Nov. 2019), 1–32. https://doi.org/10.1145/3359249
  14. Chun-Wei Chiang and Ming Yin. 2021. You’d Better Stop! Understanding Human Reliance on Machine Learning Models under Covariate Shift. In 13th ACM Web Science Conference 2021. ACM, Virtual Event United Kingdom, 120–129. https://doi.org/10.1145/3447535.3462487
  15. Reconciling data-driven crime analysis with human-centered algorithms. Cities 124 (May 2022), 103604. https://doi.org/10.1016/j.cities.2022.103604
  16. Julie E. Cohen. 2018. The Biopolitical Public Domain: the Legal Construction of the Surveillance Economy. Philosophy & Technology 31, 2 (June 2018), 213–233. https://doi.org/10.1007/s13347-017-0258-2
  17. A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). IEEE, Raleigh, NC, 690–704. https://doi.org/10.1109/SaTML54575.2023.00050
  18. DataRobot. 2023. DataRobot AI Platform. https://www.datarobot.com/
  19. An Uncommon Task: Participatory Design in Legal AI. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (April 2022), 51:1–51:23. https://doi.org/10.1145/3512898
  20. A. Desrosières and C. Naish. 1998. The Politics of Large Numbers: A History of Statistical Reasoning. Harvard University Press, Cambridge, Massachusetts. https://books.google.ca/books?id=hqO0Qvjy4HsC
  21. Writing Ethnographic Fieldnotes, Second Edition. University of Chicago Press, Chicago, IL. https://press.uchicago.edu/ucp/books/book/chicago/W/bo12182616.html
  22. Patterns of Persistence in Postsecondary Education: New Evidence for Ontario. Technical Report. Higher Education Quality Council of Ontario.
  23. Asbjørn Ammitzbøll Flügge. 2020. Algorithmic Decision Making in Public Administration: A CSCW-Perspective. In Companion of the 2020 ACM International Conference on Supporting Group Work (GROUP ’20). Association for Computing Machinery, New York, NY, USA, 15–24. https://doi.org/10.1145/3323994.3371016
  24. Statistics Canada Government of Canada. 2022. Trends in private and public funding in Canadian colleges, 2019/2020. https://www150.statcan.gc.ca/n1/daily-quotidien/220120/dq220120c-eng.htm Last Modified: 2022-01-20.
  25. Government of Ontario. 2021. Ontario Expanding High-Quality, Accessible Virtual Learning. https://news.ontario.ca/en/release/1001058/ontario-expanding-high-quality-accessible-virtual-learning
  26. Government of Ontario. 2023a. Ontario Putting Public Colleges and Universities on Stable Footing Now and into the Future. https://news.ontario.ca/en/release/1002771/ontario-putting-public-colleges-and-universities-on-stable-footing-now-and-into-the-future
  27. Government of Ontario. 2023b. Published plans and annual reports 2022–2023: Ministry of Colleges and Universities | ontario.ca. http://www.ontario.ca/page/published-plans-and-annual-reports-2022-2023-ministry-colleges-and-universities
  28. Counterfactual Prediction Under Outcome Measurement Error. In 2023 ACM Conference on Fairness, Accountability, and Transparency. ACM, Chicago IL USA, 1584–1598. https://doi.org/10.1145/3593013.3594101
  29. Alan Harrison. 2023. Ensuring Financial Sustainability for Ontario’s Postsecondary Sector. Technical Report. Ministry of Colleges and Universities.
  30. Shifting Concepts of Value: Designing Algorithmic Decision-Support Systems for Public Services. In Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society. ACM, Tallinn Estonia, 1–12. https://doi.org/10.1145/3419249.3420149
  31. The Principles of Data-Centric AI. Commun. ACM 66, 8 (Aug. 2023), 84–92. https://doi.org/10.1145/3571724
  32. Jeffrey Alan Johnson. 2018. Structural Information Justice. In Toward Information Justice: Technology, Politics, and Policy for Data in Higher Education Administration, Jeffrey Alan Johnson (Ed.). Springer International Publishing, Cham, 133–159. https://doi.org/10.1007/978-3-319-70894-2_6
  33. A Human-Centered Systematic Literature Review of Cyberbullying Detection Algorithms. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (Oct. 2021), 1–34. https://doi.org/10.1145/3476066
  34. Royce Kimmons and George Veletsianos. 2021. Proctoring Software in Higher Ed: Prevalence and Patterns. https://er.educause.edu/articles/2021/2/proctoring-software-in-higher-ed-prevalence-and-patterns
  35. Understanding Frontline Workers’ and Unhoused Individuals’ Perspectives on AI Used in Homeless Services. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23). Association for Computing Machinery, New York, NY, USA, 1–17. https://doi.org/10.1145/3544548.3580882
  36. Jan-Erik Lane. 2000. New public management. Routledge, London ;.
  37. Algorithms and Decision-Making in the Public Sector. Annual Review of Law and Social Science 17, 1 (2021), 309–334. https://doi.org/10.1146/annurev-lawsocsci-041221-023808 _eprint: https://doi.org/10.1146/annurev-lawsocsci-041221-023808.
  38. Michael P. Lillis. 2011. Faculty Emotional Intelligence and Student-Faculty Interactions: Implications for Student Retention. Journal of college student retention : Research, theory & practice 13, 2 (2011), 155–178. https://doi.org/10.2190/CS.13.2.b Place: Los Angeles, CA Publisher: SAGE Publications.
  39. David Lyon. 2010. Surveillance, Power and Everyday Life. In Emerging Digital Spaces in Contemporary Society: Properties of Technology, Phillip Kalantzis-Cope and Karim Gherab-Martín (Eds.). Palgrave Macmillan UK, London, 107–120. https://doi.org/10.1057/9780230299047_18
  40. Implications of AI (un-)fairness in higher education admissions: the effects of perceived AI (un-)fairness on exit, voice and organizational reputation. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT* ’20). Association for Computing Machinery, New York, NY, USA, 122–130. https://doi.org/10.1145/3351095.3372867
  41. A Human-Centered Review of Algorithms in Decision-Making in Higher Education. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, Hamburg Germany, 1–15. https://doi.org/10.1145/3544548.3580658
  42. A Survey on Bias and Fairness in Machine Learning. Comput. Surveys 54, 6 (July 2021), 115:1–115:35. https://doi.org/10.1145/3457607
  43. Mark Olssen and Michael A. Peters. 2005. Neoliberalism, higher education and the knowledge economy: from the free market to knowledge capitalism. Journal of Education Policy 20, 3 (2005), 313–345. https://doi.org/10.1080/02680930500108718 Publisher: Taylor and Francis Ltd.
  44. Power Dynamics and Value Conflicts in Designing and Maintaining Socio-Technical Algorithmic Processes. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (March 2022), 1–21. https://doi.org/10.1145/3512957
  45. Michael A. Peters. 2007. Higher education, globalisation and the knowledge economy: Reclaiming the cultural mission. Ubiquity 2007, May (May 2007), 1267047.1276155. https://doi.org/10.1145/1267047.1276155
  46. Nelli Piattoeva and Rebecca Boden. 2020. Escaping numbers? The ambiguities of the governance of education through data. International Studies in Sociology of Education 29, 1-2 (April 2020), 1–18. https://doi.org/10.1080/09620214.2020.1725590 Publisher: Routledge _eprint: https://doi.org/10.1080/09620214.2020.1725590.
  47. Paul Prinsloo. 2019. A social cartography of analytics in education as performative politics. British Journal of Educational Technology 50, 6 (Nov. 2019), 2810–2823. https://doi.org/10.1111/bjet.12872 Publisher: John Wiley & Sons, Ltd.
  48. Paul Prinsloo. 2020. Of ‘black boxes’ and algorithmic decision-making in (higher) education – A commentary. Big data & society 7, 1 (2020), 205395172093399–. https://doi.org/10.1177/2053951720933994 Place: London, England Publisher: SAGE Publications.
  49. Sabita Ramlal. 2021. The New Public Management and Post-Secondary Institutions: the Evolution of the Accountability Regime of Ontario Universities. thesis. Toronto Metropolitan University. https://doi.org/10.32920/ryerson.14663628.v1
  50. A Human-Centered Systematic Literature Review of the Computational Approaches for Online Sexual Risk Detection. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (Oct. 2021), 465:1–465:38. https://doi.org/10.1145/3479609
  51. A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare. http://arxiv.org/abs/2107.03487 arXiv:2107.03487 [cs].
  52. A Human-Centered Review of Algorithms used within the U.S. Child Welfare System. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, Honolulu HI USA, 1–15. https://doi.org/10.1145/3313831.3376229
  53. Devansh Saxena and Shion Guha. 2023. Algorithmic Harms in Child Welfare: Uncertainties in Practice, Organization, and Street-level Decision-Making. ACM Journal on Responsible Computing (Sept. 2023). https://doi.org/10.1145/3616473 Just Accepted.
  54. Rethinking ”Risk” in Algorithmic Systems Through A Computational Narrative Analysis of Casenotes in Child-Welfare. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, Hamburg Germany, 1–19. https://doi.org/10.1145/3544548.3581308
  55. Unpacking Invisible Work Practices, Constraints, and Latent Power Relationships in Child Welfare through Casenote Analysis. In CHI Conference on Human Factors in Computing Systems. ACM, New Orleans LA USA, 1–22. https://doi.org/10.1145/3491102.3517742
  56. From Human to Data to Dataset: Mapping the Traceability of Human Subjects in Computer Vision Datasets. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (April 2023), 55:1–55:33. https://doi.org/10.1145/3579488
  57. Ben Shneiderman. 2022. Human-Centered AI. Oxford University Press, Oxford, New York.
  58. Sharon Slade and Paul Prinsloo. 2013. Learning Analytics: Ethical Issues and Dilemmas. American Behavioral Scientist 57, 10 (Oct. 2013), 1510–1529. https://doi.org/10.1177/0002764213479366 Publisher: SAGE Publications Inc.
  59. Manuel Souto-Otero and Roser Beneito-Montagut. 2016. From governing through data to governmentality through data: Artefacts, strategies and the digital turn. European educational research journal EERJ 15, 1 (2016), 14–33. https://doi.org/10.1177/1474904115617768 Place: London, England Publisher: SAGE Publications.
  60. Gary Spraakman. 2009. New Public Management and Ceremontial Change at Ontario Colleges and Universities. https://papers.ssrn.com/abstract=1325358
  61. Harini Suresh and John Guttag. 2021. A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle. In Equity and Access in Algorithms, Mechanisms, and Optimization. ACM, – NY USA, 1–9. https://doi.org/10.1145/3465416.3483305
  62. Rachel L. Thomas and David Uminsky. 2022. Reliance on metrics is a fundamental challenge for AI. Patterns 3, 5 (May 2022), 100476. https://doi.org/10.1016/j.patter.2022.100476
  63. Vincent Tinto. 1989. Misconceptions Mar Campus Discussions of Student Retention. https://www.chronicle.com/article/misconceptions-mar-campus-discussions-of-student-retention/ Section: News.
  64. Brittany Warren. 2020. Limits to government powers in the regulation of colleges and universities. https://www.stewartmckelvey.com/thought-leadership/limits-to-government-powers-in-the-regulation-of-colleges-and-universities/
  65. Kimberly Williamson and René F. Kizilcec. 2021. Learning Analytics Dashboard Research Has Neglected Diversity, Equity and Inclusion. In Proceedings of the Eighth ACM Conference on Learning @ Scale (L@S ’21). Association for Computing Machinery, New York, NY, USA, 287–290. https://doi.org/10.1145/3430895.3460160
  66. Towards Accurate and Fair Prediction of College Success: Evaluating Different Sources of Student Data. Technical Report. International Educational Data Mining Society. https://eric.ed.gov/?id=ED608066 Publication Title: International Educational Data Mining Society ERIC Number: ED608066.
  67. Deliberating with AI: Improving Decision-Making for the Future through Participatory AI Design and Stakeholder Deliberation. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (April 2023), 1–32. https://doi.org/10.1145/3579601
  68. Shoshana Zuboff. 2015. Big other: Surveillance Capitalism and the Prospects of an Information Civilization. Journal of information technology 30, 1 (2015), 75–89. https://doi.org/10.1057/jit.2015.5 Place: London, England Publisher: SAGE Publications.
  69. Shoshana Zuboff. 2019. Surveillance Capitalism and the Challenge of Collective Action. New labor forum 28, 1 (2019), 10–29. https://doi.org/10.1177/1095796018819461 Place: Los Angeles, CA Publisher: Sage Publications, Inc.
  70. Shoshana Zuboff. 2020. The age of surveillance capitalism: the fight for a human future at the new frontier of power / Shoshana Zuboff. (first trade paperback edition ed.). PublicAffairs, New York.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Kelly McConvey (2 papers)
  2. Shion Guha (23 papers)

Summary

"This is not a data problem": Examining the Impact of Algorithms and Power in Canadian Public Higher Education

Introduction to the Study

A recent qualitative analysis conducted on data and algorithm usage within a public college in Ontario, Canada provides valuable insights into how post-secondary institutions are integrating data-driven practices. This significant shift towards reliance on algorithmic decision-making is largely influenced by internal and external pressures towards financial sustainability. The paper contends that such reliance leads to heightened student surveillance, magnifies existing inequalities, and automates elements of the faculty-student relationship. Notably, it highlights a vicious cycle where algorithmic decision-making perpetuates increased institutional power, driven by the overarching goal of financial sustainability.

Data-Driven Practices and Algorithmic Decision-Making

The paper uncovers that data-driven decision-making at the college aims primarily at resource allocation, revenue forecasting, and potentially commercialization of algorithms. This aligns with government policies and the neoliberal trend towards greater accountability and efficiency in higher education. The adoption of these practices responds to financial challenges but raises ethical concerns related to student privacy, equity, and the transformation of educational values.

Algorithms at Work

The analysis details the college's foray into algorithmic decision-making with three primary models focused on enroLLMent prediction and assessing student success. The use of automated machine learning platforms, such as DataRobot, simplifies model building but limits transparency and control over the machine learning process. Concerns arise regarding model accuracy, bias mitigation, and the absence of clear ownership and responsibility for algorithmic decisions.

Impact on Stakeholder Relationships

The integration of algorithms and data-driven systems redefines relationships within the institution. This shift particularly affects the academic and student experience divisions, with notable discrepancies in perceptions of the college's capabilities in handling data and algorithms. As decision-making increasingly relies on data, divisions with direct data access, particularly those linked to business and administration, gain undue influence over institutional priorities.

Consequences for Students

The emphasis on data and algorithms profoundly affects students through increased surveillance, reinforced inequities, and a reduced human element in education. The paper discusses how the need for extensive data for algorithm training leads to pervasive student monitoring, raising ethical concerns. The opaque nature of algorithmic scoring and categorization can exacerbate existing biases and unfairly impact students from marginalized groups. Moreover, the automation of identifying and supporting at-risk students traditionally a faculty role signals a shift towards viewing education through a business lens, potentially undermining the quality of student support and the faculty-student relationship.

The ASP-HEI Cycle: A New Framework for Understanding Power Dynamics

The paper introduces the "ASP-HEI Cycle" as a conceptual framework to illustrate how algorithms, student data, and institutional power interact within higher education institutions. This cycle reveals how financial sustainability goals dictate the adoption of data-driven practices, which in turn amplify institutional control at the expense of student autonomy and equity. The framework underscores the cyclic nature of these dynamics, where financial pressures drive further reliance on algorithmic practices, perpetuating institutional power and potentially distancing higher education from its core democratic and inclusive values.

Implications and Future Directions

This research underscores the need for a careful examination of how higher education institutions adopt algorithmic decision-making. The findings call for a more human-centered approach in the design and implementation of algorithms, emphasizing transparency, equity, and stakeholder participation. There is a clear necessity to balance efficiency and financial considerations with the educational mission's qualitative aspects. Future research should further explore the consequences of these practices on the educational experience and how institutions can mitigate potential harms while embracing the opportunities presented by technological advancements.

Concluding Thoughts

The paper sounds a cautionary note on the growing reliance on data and algorithms within higher education, highlighting the complex interplay between financial sustainability, institutional power, and the risk to democratic educational values. It presents a critical perspective that invites further dialogue and reflection among policymakers, educators, and researchers on navigating the challenges of integrating technology in education while preserving its core values of inclusivity, fairness, and human connection.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown