Research Summary: All four leading LLMs talk more than they listen to personality-verified synthetic help-seekers

Research indicates that leading large language models (LLMs) tend to dominate conversations with synthetic users seeking help during crises, rather than exhibiting responsive listening. An evaluation using personality-verified synthetic help-seekers, facing a caregiver's crisis, revealed that LLMs' responses did not adapt sufficiently to diverse psychological profiles, even when those profiles could be accurately inferred by human auditors from dialogue.

Why it matters

This research highlights a critical limitation in current LLM capabilities, specifically their ability to engage adaptively and empathetically in sensitive, multi-turn interactions. As LLMs are increasingly deployed in support roles, their failure to 'listen' and respond contextually to diverse human psychological needs poses significant challenges for user trust and effective intervention.

Key insights

• Four leading LLMs exhibited a tendency to 'talk more than they listen' when interacting with synthetic help-seekers in acute crisis scenarios.
• The evaluation involved personality-verified synthetic help-seekers, each assigned a psychometrically specified profile, simulating a caregiver learning of a dementia diagnosis.
• Human auditors, unaware of the initial profile prompts, were able to accurately infer the specified psychological profiles (including Big Five, coping style, coping self-efficacy, resilience, and reactance) from the dialogue alone.
• The LLMs' responses did not adequately differentiate based on the distinct personality and coping profiles of the synthetic users, suggesting a lack of adaptive interaction.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2608.22425