Regulatory Approval Is Not Enough: Gaps in Trustworthy AI Reporting in FDA-Cleared Medical Devices

Research indicates significant gaps in the publicly available regulatory documentation for AI/ML-enabled medical devices, specifically concerning the reporting of 'trustworthy AI' principles. An analysis of FDA summary reports for devices cleared between 2021 and 2025 found insufficient evidence to independently assess crucial dimensions of trustworthiness, despite the increasing integration of these systems into clinical decision-making.

Why it matters

The lack of comprehensive public reporting on AI trustworthiness in regulated medical devices poses a significant challenge to fostering confidence in these technologies. Addressing these transparency gaps is critical for maintaining public trust, ensuring patient safety, and facilitating the responsible adoption of AI in sensitive sectors.

Key insights

• AI/ML-enabled medical devices are increasingly deployed in healthcare, operating within evolving regulatory frameworks.
• There is a growing expectation for these systems to demonstrate 'trustworthy AI' dimensions to build confidence among clinicians, patients, and the public.
• Analysis of FDA AI/ML-enabled medical device summary reports from 2021-2025 revealed that publicly available documentation often lacks sufficient evidence.
• The identified gaps hinder the independent assessment of the trustworthiness of cleared AI systems, particularly against established principles like FUTURE-AI.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2608.12360