The Judgment-Consequence Gap: LLM Moral Reasoning in Healthcare Decisions

Research from arXiv investigates the moral reasoning of large language models (LLMs) in healthcare, specifically concerning resource allocation when patient actions contribute to illness. The study evaluates various LLMs across different families and capabilities using clinical vignettes. A key finding is the 'judgment-consequence gap,' where LLMs generally align with human judgment regarding patient responsibility for health-harming behaviors but diverge when considering the consequences for care denial.

The integration of LLMs into critical sectors like healthcare requires robust understanding of their decision-making processes, especially in morally complex scenarios. Misaligned moral reasoning could lead to inequitable or ethically questionable resource allocation decisions, undermining trust and potentially causing significant societal impact.

LLMs are being integrated into high-stakes domains, including healthcare, necessitating an understanding of their moral reasoning.

Forward consideration, not a verified fact.