Research Summary: Algorithm Validation as a Policy Audit: Evidence from Race-blind Charging

Research from arXiv highlights an algorithm developed to automate race-blind review in legal charging decisions, as mandated by California. This LLM-based tool, bc2, redacted race-related proxies in over 119,000 cases in 2025. The study evaluates the algorithm's compliance with state requirements and the effectiveness of those requirements in achieving race-blind decision-making, utilizing a corpus of nearly 5,000 police reports.

Why it matters

This development underscores the increasing integration of artificial intelligence into critical public sector operations and policy implementation. It highlights the necessity of rigorously validating AI tools to ensure compliance with regulatory mandates and to assess their effectiveness in achieving intended policy outcomes, particularly in sensitive areas like fairness and equity. Organizations must consider the implications of AI on procedural integrity and ethical standards.

Key insights

• California mandated 'race-blind charging' decisions for prosecutors, requiring the redaction of race-related proxies from case documents.
• An open-source, LLM-based algorithm, bc2, was developed to automate this redaction process.
• The bc2 algorithm facilitated race-blind review in over 119,000 real-world cases during 2025.
• The research aims to validate whether bc2 faithfully implements the state's requirements.
• The study also investigates if the state's requirements, even when faithfully implemented, advance the goal of race-blind decision-making.
• The validation and evaluation are based on a corpus of nearly 5,000 real-world police reports from various U.S. jurisdictions.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.13174