Research Summary: Runtime Governance for Agentic AI: Action-Boundary Control with Trusted Provenance and Fail-Closed Execution

The advent of agentic AI systems introduces a new class of safety challenges, shifting from harmful content generation to harmful operational side effects caused by AI-initiated actions such as file modifications or workflow changes. Traditional prompt-level governance is insufficient to contain these risks as it lacks an execution boundary. A proposed solution, Aegis, addresses this by implementing a runtime governance system that interposes a trusted decision layer between the AI model's action proposals and their execution. This system evaluates proposals against established policies, ensures trusted provenance, operates on a fail-closed principle, and can route complex decisions for quorum-based authorization.

Why it matters

The safe and effective deployment of agentic AI systems is critical for organizations seeking to leverage their autonomous capabilities without incurring significant operational risks. Establishing robust governance mechanisms at the execution layer is paramount to preventing unintended or malicious actions, thereby safeguarding organizational assets and integrity. This approach directly addresses the emerging challenge of AI systems having direct operational impact, which is distinct from traditional content-based AI risks.

Key insights

• Agentic AI systems pose new safety risks through their ability to request and perform operational actions (e.g., modifying files, sending messages, changing workflow state).
• The primary safety concern for agentic AI transitions from harmful text generation to potential harmful operational side effects.
• Prompt-level governance alone is inadequate for managing agentic AI risks because it does not establish an execution boundary for AI-proposed actions.
• Aegis is introduced as a runtime governance system designed to mediate AI model outputs as action proposals.
• The system operates with a 'model proposes; trusted runtime decides' paradigm, inserting a decision layer before tool execution.
• Key features of Aegis include evaluating proposals against policy state, resolving provenance server-side, failing closed under uncertainty, and routing certain cases for quorum-based authorization ('Senate-style settlement').

Source

arXiv — Computers and Society — https://arxiv.org/abs/2608.16891