Accurate in space, unreliable in time: how LLMs represent national cultural change

Recent research from arXiv:2609.01902v1 indicates that Large Language Models (LLMs) are proficient at representing national cultural alignment at a given point in time but demonstrate unreliability in accurately capturing how cultural values change over historical periods. Current evaluation methods for LLMs predominantly treat culture as static, overlooking the dynamic nature of cultural evolution identified by cultural psychology.

Why it matters

The accurate representation of cultural change over time is crucial for LLMs, as their applications increasingly intersect with diverse global contexts requiring nuanced historical and societal understanding. Failing to account for cultural evolution could lead to misinterpretations, biased outputs, and ineffective or inappropriate interactions in domains where cultural sensitivity and historical context are paramount.

Key insights

• Assessments of cultural alignment are critical for LLM development and improvement.
• Most LLM cultural evaluations treat culture as a 'single snapshot,' focusing only on current societal representation.
• Cultural psychology research indicates that cultural values evolve over time, exhibiting varied rates and directions of change.
• A truly 'culturally aware' LLM should model both current cultural states and historical cultural trajectories.
• The study examined LLMs' ability to capture cultural trajectories across 40 countries using over two decades of World Values Survey data.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.01902