Research Summary: White Men Without Degrees Receive the Lowest Ratings from Large Language Models

Recent research from arXiv:2610.00185v1 indicates that large language models (LLMs) consistently assign the lowest average ratings to 'White men without an undergraduate degree' across evaluations for credit, hiring, and rental applications. This finding emerged from full-factorial vignette experiments involving 18 LLMs from 12 developer groups, which assessed 32 profiles varied by gender, race, age, citizenship, and education while maintaining constant financial or occupational circumstances. The average ratings for this demographic were 75.87 in credit, 92.71 in hiring, and 86.62 in rental housing on a 0-100 scale.

Why it matters

This research highlights potential systemic biases embedded within large language models that could significantly impact equitable access to essential services and opportunities. Organizations deploying or developing AI in critical decision-making processes must understand and mitigate these biases to ensure fairness and prevent unintended discriminatory outcomes. Failure to address these issues could lead to regulatory scrutiny, reputational damage, and erosion of public trust in AI systems.

Key insights

• Large language models (LLMs) provide the lowest average ratings to 'White men without an undergraduate degree' when evaluating credit, hiring, and rental applications.
• The evaluations were conducted using 18 LLMs from 12 developer groups, assessing 32 profiles across various demographic dimensions.
• Financial and occupational circumstances were held constant in the vignettes, allowing isolation of demographic biases.
• Average ratings for 'White men without degrees' were 75.87 (credit), 92.71 (hiring), and 86.62 (rental housing) on a 0-100 scale.
• The study involved 17,280 ratings, demonstrating a robust dataset for analysis.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2610.00185