Small Changes, Big Impact: Demographic Bias in LLM-Based Hiring Through Subtle Sociocultural Markers in Anonymised Resumes

A recent research paper highlights that Large Language Models (LLMs) used in resume screening pipelines can introduce or perpetuate demographic bias, even when explicit Personally Identifiable Information (PII) is removed. This bias stems from subtle sociocultural markers present in anonymised resumes. A study in Singapore found that 18 different LLMs exhibited bias across four ethnicities and two genders, indicating that current AI-driven hiring practices may not be as fair as presumed.

Why it matters

This research reveals a significant risk in the deployment of AI for critical human capital processes, particularly hiring. Unchecked, such biases can lead to discriminatory outcomes, undermine diversity initiatives, and expose organizations to reputational and regulatory risks. Addressing these subtle biases is crucial for maintaining equitable talent acquisition and fostering inclusive workplaces.

Key insights

• LLMs are increasingly used in resume screening, raising concerns about fairness and bias.
• Bias can persist in LLM-based hiring even after explicit PII (e.g., names) is redacted.
• Subtle sociocultural markers (languages, co-curricular activities, volunteering, hobbies) can act as demographic proxies.
• A stress-test framework, instantiated in Singapore, created 4100 resume variants from 100 neutral resumes, differing only in job-irrelevant markers for various ethnicities and genders.
• Evaluation of 18 LLMs in both direct comparison and score/shortlist settings, with and without rationale prompting, revealed demographic bias.
• The study confirms that LLMs can inadvertently discriminate based on job-irrelevant sociocultural cues.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2603.05189