Research Summary: The One-Word Census: Answer-Choice Conformity Across 105 Language Models

A research study evaluating 105 language models from over twenty laboratories demonstrates pronounced answer-choice conformity across large option spaces, finding that models repeatedly converge on identical single-word responses. Using 96 category-based prompts tested across eight iterations per model, the analysis revealed severe behavioral homogeneity, with single answers capturing at least 80% of total responses in 28 categories.

Why it matters

This matters because pervasive monoculture and response convergence across ostensibly competing language models create systemic blind spots and correlated points of failure. Organizations deploying artificial intelligence solutions risk false consensus and reduced diversity of thought, mistaking shared training artifacts for objective truth. Consequently, strategic decision-making that relies on multi-model consensus may inherit unmonitored epistemic bias across distinct vendors.

Key insights

• Across 105 language models from more than 20 labs, asking the models to 'pick a word' resulted in the selection of 'serendipity' 46% of the time.
• In 28 out of 96 tested categories with many valid answers, a single answer accounted for at least 80% of all generated responses.
• The study measured model convergence using 96 single-turn prompts queried eight times per model, quantified via exact-match scoring without embeddings or automated judges.
• Model answer-choice surprisal was formulated as the average negative log-base-2 probability of a model's choices against the pooled outputs of all other models.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2607.12796