Fairness Theatre: Evaluating Post-Hoc Fairness Interventions in Vendor-Controlled Early Warning Systems

Public institutions are increasingly procuring proprietary AI systems, such as Early Warning Systems (EWS) in higher education, where they lack the ability to inspect or modify the system's design. This situation necessitates the use of post-hoc fairness interventions to address inequities, raising questions about the coordination of fairness efforts among various stakeholders with unequal power dynamics. Research evaluating six such interventions on a simulated EWS in a Canadian public college identified how these interventions impact fairness, accuracy, and demographic disparities, and redistribute false positives and false negatives.

The proliferation of opaque AI systems in public sectors introduces significant governance and ethical challenges, particularly regarding fairness and accountability. Institutions must understand the limitations and potential impacts of proprietary AI to develop effective strategies for risk mitigation and equitable deployment. This research highlights the critical need for robust evaluation frameworks and clearer power dynamics in the procurement and management of AI technologies.

Public institutions are acquiring AI systems whose internal design and operation cannot be inspected or altered by the procuring entity.

Forward consideration, not a verified fact.