Research Summary: Language Models Reproduce Human Reductionist Bias and Decision Inconsistency in Neurodevelopmental Disorders Assessment

Recent research from arXiv reveals that large language models (LLMs) and human professionals (physicians and psychologists) exhibit similar biases and inconsistencies in assessing neurodevelopmental disorders. Specifically, both groups demonstrate a disconnect between descriptive functional level assessments and final eligibility decisions for support, indicating a reductionist bias and decision inconsistency. This suggests that LLMs currently mirror human cognitive limitations in complex, value-laden mental health assessments.

Why it matters

This finding highlights critical challenges for the responsible deployment of AI in sensitive domains like healthcare and mental health, particularly where complex, value-laden judgments are required. It underscores the need for robust auditing frameworks to ensure fairness and consistency, and to address the potential for algorithmic bias to mirror or amplify human biases in critical decision-making processes.

Key insights

• LLMs and human professionals show comparable decision inconsistencies in neurodevelopmental disorder assessments.
• Both groups' ratings of patients' functional levels are not significantly associated with support-eligibility decisions.
• This indicates a potential disconnect between descriptive assessments and final evaluative judgments.
• The study introduces a mixed-methods human-LLM auditing framework for complex mental-health decisions.
• The assessment process involves factual evidence alongside value-laden interpretations, which LLMs currently reproduce similarly to humans.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2608.17105