Research Summary: Retrieval Sensitivity to Identity Signals in Queries

Research indicates that dense retrieval systems, crucial for determining document access and influencing language models, exhibit sensitivity to identity signals embedded in user queries. Specifically, these systems show a bias towards retrieving content aligned with the political ideology expressed in a query and perform less effectively for queries framed in African American Language compared to White Mainstream English. This sensitivity was observed across multiple retrieval models in political news and consumer-health domains.

Why it matters

The observed biases in information retrieval systems have significant implications for fair access to information, potentially reinforcing existing disparities and echo chambers. Organizations relying on these systems must address these sensitivities to ensure equitable and unbiased information dissemination, especially in critical domains like health and public discourse.

Key insights

• Retrieval systems, including dense retrievers and a sparse baseline, consistently return articles aligning with the political lean expressed in a user's query.
• Retrieval performance is demonstrably worse for questions formulated in African American Language (AAL) compared to White Mainstream English (WME).
• Evaluations were conducted across two distinct domains: political news and consumer-health questions.
• The observed biases were present across five different dense retrievers and a sparse baseline model.
• Controlled synthetic sets were used to isolate the impact of identity signals, alongside naturalistic queries.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.36534