Can We Trust AI Agents in the Supermarket? Sugar Content Inference from Product Images

Research evaluating AI agent systems for inferring sugar content from supermarket product images across four national contexts reveals significant performance variations. AI agents demonstrate inconsistent reliability in accurately identifying lower-sugar options based solely on front-of-pack images, challenging their potential as a substitute for regulated nutritional labeling.

Why it matters

The growing reliance on AI for consumer information, particularly in health-related areas like nutrition, necessitates careful assessment of its accuracy and reliability. Inconsistent AI performance across different markets could lead to misinformed consumer choices and impact public health, necessitating strategic consideration of AI's role in information dissemination and regulatory oversight.

Key insights

• Nutritional labels are often presented in small print, which reduces readability for consumers.
• Consumers are increasingly relying on AI nutrition lenses and vision-capable conversational agents for dietary guidance.
• A study using a Two-Alternative Forced Choice game evaluated AI agents' ability to infer which of two packaged foods had less sugar from front-of-pack images.
• The evaluation covered four national supermarket contexts: Sweden, the USA, Australia, and Kazakhstan, with 132 comparisons.
• The results indicate a significant performance divide for AI agents depending on the specific national context and product origin (global vs. local).
• The current reliability of AI-mediated advice may not meaningfully substitute for regulated nutritional labeling.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2608.12359