Following the Preference, Missing the Optimum: Compliance Without Optimization in AI Housing Recommendation

Research indicates that AI-driven housing recommendation systems, despite their growing role as initial points of contact for consumers, may fail to optimize outcomes for users. By prioritizing compliance without explicitly considering optimization, these models can overlook suitable options, potentially steering users away from their ideal choices. This suggests a critical gap in the design and auditing of AI systems in high-stakes domains where user preferences and legal stipulations intersect.

Why it matters

This research highlights a fundamental challenge in the deployment of AI systems in critical public services and regulated industries: the potential conflict between regulatory compliance and optimal user outcomes. Organizations deploying or relying on AI for recommendation or decision support must understand that mere compliance may not equate to user satisfaction or ethical performance, necessitating a re-evaluation of AI design and audit methodologies.

Key insights

• Large language models (LLMs) are increasingly serving as the primary interface for consumer searches in sectors with significant material stakes and explicit legal frameworks, such as housing.
• Prior audits have demonstrated that AI models can influence housing seekers based on perceived identity, raising concerns about fairness and bias.
• A key limitation identified is the inability of existing audits to quantify what a user foregoes when a recommender fails to present an optimal option, due to a lack of a comprehensive inventory for scoring omissions.
• The research audited AI housing recommendations by creating 150 synthetic renter scenarios in New York City.
• For each scenario, a pool of 120 real listings with verified rent, bedrooms, and transit commute data was used to identify the exact set of listings satisfying stated renter constraints and to derive its Pareto frontier.
• The primary outcome of the audit focused on the discrepancy between preference following and optimal outcome delivery, indicating a potential 'compliance without optimization' paradigm.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.10856