Applied and Filtered: An End-to-End Algorithmic Fairness Audit of A Public Employment Agency

A research paper from arXiv details an independent, end-to-end algorithmic fairness audit of a semi-automated hiring system used by Barcelona Activa, a public employment agency. The audit, covering approximately 497,000 candidate-vacancy pipeline entries over five years, evaluated the third-party TalentClue platform. While aggregate outcomes across binary genders appeared statistically indistinguishable, this parity was found to mask substantial underlying disparities within the system.

Why it matters

This research highlights that relying solely on aggregate fairness metrics can be misleading, as underlying biases and disparities may persist within complex algorithmic systems. For organizations deploying AI in critical functions, it underscores the necessity of comprehensive, granular audits to ensure true equity and mitigate hidden risks.

Key insights

• An independent, end-to-end algorithmic fairness audit was conducted on a semi-automated hiring system used by a public employment agency (Barcelona Activa).
• The audited system involved the third-party TalentClue platform for candidate search and shortlisting.
• The analysis covered approximately 497,000 candidate-vacancy pipeline entries from September 2017 to September 2022.
• Seven pipeline stages were examined, encompassing automated processing, human discretion, candidate data, and employer decisions.
• Aggregate outcomes across binary genders were statistically indistinguishable, implying overall fairness at a high level.
• This aggregate parity was found to conceal substantial underlying disparities within the system.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2608.13022