Silent Updates: Measuring and Closing the Post-Deployment Disclosure Gap

Research identifies 'silent updates' to deployed foundation models as a critical governance challenge. Providers can modify model behavior post-deployment without public disclosure, versioning, or re-evaluation. This practice undermines the foundational assumption of AI governance frameworks that links evaluated models to those in use, creating a 'disclosure gap' that hinders oversight and accountability.

Why it matters

The practice of silent updates introduces significant opacity into the deployment of AI systems, directly impacting trust, safety, and accountability. Organizations relying on these models, or those tasked with their governance, face challenges in verifying performance, compliance, and ethical standards if the underlying systems are changing without notice. This issue necessitates a re-evaluation of current AI governance models to ensure they account for the dynamic nature of deployed AI.

Key insights

• Deployed foundation models are frequently modified by providers through various methods including fine-tuning, classifier updates, prompt revisions, retrieval changes, and routing adjustments.
• These post-deployment modifications are often 'silent,' meaning they occur without any public disclosure, version increment, or re-evaluation of the system.
• Silent updates challenge a core assumption of current AI governance frameworks, which rely on an externally verifiable chain of custody between evaluated models and those served to users.
• The research aims to measure the extent of this 'post-deployment disclosure gap' by examining practices across first-party API providers and inference hosts.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2608.11803