Large language models underestimate and partly misrepresent cultural variation in everyday norms

A recent study comparing Large Language Models (LLMs) against human benchmark data from the Global Study of Everyday Norms (GSEN) found that leading LLMs consistently underestimate and partially misrepresent cultural variation in everyday behavioral norms. Specifically, the models estimated societal differences to be less than half their measured size, indicating a significant accuracy gap in their understanding of cross-cultural nuances.

The observed underestimation and misrepresentation of cultural variation by LLMs highlight critical limitations in their ability to accurately process and reflect diverse human societal norms. This has significant implications for the reliability of AI systems deployed in globally diverse contexts, impacting decision-making, policy development, and cross-cultural interactions mediated or informed by these models.

Leading LLMs (GPT-5, GPT-5.4, Claude Opus 4.6, Gemini 3.1 Pro) were benchmarked against the Global Study of Everyday Norms (GSEN), which covers 150 scenarios across 90 societies.

Forward consideration, not a verified fact.