Research Summary: Moral Advice as Interactional Negotiation: Framing, User Pressure, and Social Position in Large Language Model Responses

Research from arXiv reveals that Large Language Models (LLMs), specifically GPT-4o-mini, engage in 'moral advice as interactional negotiation,' influenced significantly by framing, sustained user pressure, and the social position presented by the 'moral subject.' A factorial vignette experiment on eldercare dilemmas showed that LLM responses varied based on whether the situation was framed as caregiving and the user's persistence, indicating that LLMs do not offer fixed moral guidance but rather negotiate it based on interactional dynamics.

Why it matters

This research is strategically important because it highlights the dynamic and negotiable nature of moral guidance from AI systems, rather than a fixed ethical stance. Understanding these interactional sensitivities is critical for designing and deploying AI ethically, especially as LLMs become more integrated into decision-making and advisory roles across various sectors.

Key insights

• LLMs increasingly interpret and legitimize morally contested choices, serving as a source of everyday guidance.
• Moral advice provided by LLMs is an interactional negotiation, shaped by the framing of the dilemma, sustained user pressure, and the moral subject's social position.
• Using GPT-4o-mini, a factorial vignette experiment explored responses to eldercare dilemmas.
• Caregiving affirmation in the dilemma framing led to near-uniform endorsement from the LLM.
• Non-caregiving framing resulted in different responses, implying sensitivity to context.
• The study involved 1,620 configuration-framing cells, repeated three times, for a total of 4,860 conversational runs, demonstrating robust data collection.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.05345