Research Summary: Accountable AI with Grounded, Faithful, Consistent, Actionable Rationales: A Case Study in Clinical Trial Matching with VERDICT

The field of Artificial Intelligence, particularly with Large Language Models (LLMs), faces significant challenges in achieving accountability in decision-making processes. Traditional LLM outputs can be ungrounded, incomplete, or unfaithful to the underlying decision logic. This research proposes that accountability requires verifiable rationales, clear articulation of assumptions, policy consistency, and identification of pivotal conditions. It introduces 'self-faithfulness' as a method to test accountability, demonstrating its application in the high-stakes domain of clinical trial matching.

Why it matters

As AI systems become more prevalent in critical decision-making across various sectors, ensuring their accountability is paramount for trust and effective governance. This research offers a framework and a testing methodology to achieve greater transparency and justification in AI outcomes, which is crucial for managing ethical considerations and regulatory compliance.

Key insights

• Accountability in AI decisions is defined by the ability to examine, justify, and contest outcomes.
• Large Language Models (LLMs) present difficulties for accountability due to potential for ungrounded, incomplete, or unfaithful outputs.
• Key components for achieving AI accountability include verified rationales, stated assumptions, consistent policy application, and identification of pivotal conditions.
• The concept of 'self-faithfulness' is introduced as an automatic test for accountability, where changing pivotal conditions should alter the decision.
• The research applies this accountability framework to clinical trial matching, a critical and high-stakes task in evidence-based medicine.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.03366