Language-model groups overstate consensus when replaying human deliberation on a reasoning task

Research comparing human and large language model (LLM) groups on a reasoning task found that LLM agent groups consistently exhibited higher rates of consensus than human groups, irrespective of how consensus was defined or participation levels were matched. This overstatement of consensus by LLMs appears to be influenced by their higher participation rates compared to humans.

This research highlights a critical difference in collaborative behavior and outcome between human and AI-driven groups, particularly concerning consensus formation. Understanding these dynamics is vital for organizations deploying AI in decision-making, collaborative environments, or simulations, as AI might present an artificially high level of agreement, potentially masking underlying complexities or dissenting views.

Human group full-consensus rates on a Wason reasoning task ranged from 24.0% to 57.0% depending on scoring definitions.

Forward consideration, not a verified fact.