Safety Alignment Illusion: The Cross-Lingual Safety Gap in LLMs

Research identifies a critical safety alignment gap in Large Language Models (LLMs) when used in non-English languages, particularly for diverse linguistic communities. Current safety training is predominantly English-centric, leading to the failure of safety filters and the potential propagation of harmful biases, such as stereotype-reinforcing outputs in voice assistants and spoken dialogue systems. This issue is particularly pronounced in linguistically diverse regions like India. A new multilingual evaluation benchmark, INCLUDE, has been introduced to quantify Indian-centric societal biases and address this cross-lingual safety challenge.

Why it matters

The identified cross-lingual safety gap in LLMs poses a significant risk to the equitable and ethical deployment of AI technologies globally. Addressing this gap is crucial for maintaining trust in AI systems and preventing the exacerbation of societal biases, particularly in diverse linguistic markets. It also highlights the need for a more inclusive and culturally sensitive approach to AI development and deployment strategies.

Key insights

• Safety alignment training for Large Language Models (LLMs) is heavily English-centric.
• Safety filters in LLMs often fail when applied to non-English languages.
• The failure of non-English safety filters can lead to user-facing consequences, including stereotype-reinforcing outputs.
• Harmful biases can be propagated to non-English speaking communities through technologies like voice assistants.
• Linguistically diverse populations, such as in India, represent a critical failure mode for current LLM safety approaches.
• The INCLUDE (Indian Cultural Lens for Understanding and Detecting Embedded Biases) benchmark has been introduced to evaluate cross-lingual safety for Indian-centric societal biases.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2608.18131