Why are all LLMs Obsessed with Japanese Culture? On the Hidden Cultural and Regional Biases of LLMs

Recent research indicates that Large Language Models (LLMs) exhibit specific cultural and regional biases, particularly demonstrating a tendency to favor Japanese culture in responses to generic culture-related questions. This finding contrasts with some prior studies on cultural bias and highlights a significant limitation in LLMs' cultural coverage and competence, irrespective of the prompting language used.

Why it matters

The presence of regional and cultural biases in LLMs necessitates a re-evaluation of their reliability and applicability in diverse global contexts. This can impact strategies related to product development, market penetration, and information dissemination, as biased outputs may inadvertently misrepresent or exclude certain cultural perspectives.

Key insights

• LLMs possess limitations in cultural coverage and competence.
• Specific cultural biases are present in LLMs.
• Previous studies did not specifically investigate regional preferences in generic culture-related questions.
• A new dataset, Culture-Related Open Questions (CROQ), was developed for evaluating LLMs, available in 24 languages.
• Evaluation revealed a clear tendency for LLMs to emphasize countries like Japan in their cultural answers.
• This bias was observed even when prompting in various languages, suggesting it is not language-dependent.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2604.21751