Research Summary: Trust Is Not a Score: Runtime Assurance Contracts for High-Risk AI Agents

New research introduces the concept of a Runtime Assurance Contract (RAC) to bridge the 'assurance-transition gap' in high-risk AI agents. This gap refers to the missing framework for how observed evidence should dynamically alter an AI agent's operational authority during critical tasks. RACs propose a formal policy-level schema that defines autonomy boundaries, component eligibility, evidence states, transition policies, human review capacities, and non-compensatory gates, ensuring that aggregate performance does not unilaterally authorize action.

Why it matters

The assurance-transition gap presents a critical challenge for the safe and responsible deployment of AI in high-stakes environments. Addressing this gap through formal contracts like RACs is vital for managing AI risk, building public trust, and ensuring regulatory compliance. It enables organizations to confidently integrate advanced AI systems into operations where failures could have severe consequences.

Key insights

• A significant 'assurance-transition gap' exists, where current benchmarks, audits, and protocols fail to define how real-time evidence should modify an AI agent's authority in consequential tasks.
• Runtime Assurance Contracts (RACs) are proposed as a formal policy-level schema to address this gap, binding various operational parameters for high-risk AI agents.
• RACs specify autonomy boundaries, component eligibility, evidence states, transition policies, human-review capacity, and non-compensatory gates.
• Under a RAC, soft performance metrics can inform routing decisions, but a failed or unknown mandatory gate necessitates specific actions like retry, switch, escalation, deferral, or stopping the task.
• Aggregate performance metrics alone are insufficient to authorize action; critical safety and operational gates must be met individually.
• The framework defines the contract itself, an evidence record, a permission rule, and five invariants to ensure robust operation.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.39717