Research Summary: Geopolitical Divisions Across Languages in Large Language Models

Recent research reveals that large language models (LLMs) like GPT, Claude, and Gemini demonstrate geopolitical biases in their responses, which vary significantly based on the language of the query. When evaluating statements about the Ukraine conflict, the balance of Russia-leaning versus Ukraine-leaning answers differs across languages, often mirroring real-world political alignments of countries where those languages are official.

Why it matters

This research highlights a critical vulnerability in the perceived neutrality and reliability of AI systems, particularly as they are increasingly used for information dissemination on sensitive global events. The observed linguistic bias can inadvertently influence public opinion, propagate misperceptions, and undermine efforts to provide balanced information across diverse linguistic and cultural contexts.

Key insights

• LLMs exhibit differing geopolitical biases in their responses depending on the query language.
• Specifically, the balance of Russia-leaning and Ukraine-leaning responses to questions about the Ukraine conflict varies across 112 languages.
• This linguistic-driven bias aligns with geopolitical divisions, where responses in a country's official language tend to reflect that country's public sentiment towards Russia or its voting patterns in the United Nations regarding Ukraine.
• The study collected 67,200 responses across GPT, Claude, and Gemini, indicating a systemic observation across major AI platforms.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.20005