When Personalization Becomes Bias: Structural and Discursive Religious Framing in AI-Generated Financial Advice

Recent research from arXiv highlights a critical issue regarding the integration of Large Language Models (LLMs) into financial advisory systems: the pervasive reproduction of religious bias. A study examining ChatGPT, Gemini, and Grok found that these AI models exhibited structural biases in financial advice across various religious identities and decision contexts. Unbiased advice was generated in only a small fraction of interactions, indicating a significant systemic problem.

Why it matters

This research reveals a significant risk of unfair and potentially discriminatory outcomes when AI is applied to sensitive domains like financial advice. Organizations deploying or developing LLM-based financial tools must address these biases to maintain trust, ensure ethical operations, and comply with potential future regulations regarding AI fairness and discrimination.

Key insights

• Large Language Models (LLMs) integrated into financial advisory systems reproduce religious bias.
• The study used 432 simulated advisor-client interactions with ChatGPT, Gemini, and Grok.
• Religious identity pairings included Christian, Muslim, Hindu, and non-religious individuals.
• Bias was observed across core financial decisions: stock investment, house purchase, and life insurance.
• Structural biases were identified across all tested models and decision contexts.
• Discursive mechanisms, or linguistic enactments, were found to facilitate these biases.
• Only 12-18% of AI-generated advice in the study was deemed unbiased.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2608.16909