Research Summary: Context and Symmetry in Auditing: A Case Study of Skeleton Inference in Motion Capture

The research introduces 'contextual auditing' as a method for evaluating Artificial Intelligence (AI) systems, particularly those that observe and make inferences about human behavior. This approach emphasizes understanding AI system behavior within the specific practices and contexts that generate data, aiming to bridge the gap between how an AI system actually performs and how it is expected to perform. The paper argues for the necessity of this method to effectively interrogate assumptions in AI auditing.

Why it matters

As AI systems become more prevalent across various sectors, their reliable and ethical operation is paramount. Implementing robust auditing methodologies ensures AI systems function as intended, mitigating risks associated with inaccurate inferences and fostering trust in AI deployments. This directly impacts operational integrity and compliance with future regulatory frameworks.

Key insights

• AI systems are increasingly making inferences about humans, necessitating robust evaluation methods.
• Standard AI auditing involves identifying actual system behavior and contrasting it with expected behavior.
• Contextual auditing is proposed as a method to audit measurements within the context of their producing practices.
• This method facilitates the interrogation of assumptions made during AI auditing.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2608.10194