Research Summary: Vulnerabilities in Personalization: Assessing Health Privacy Risks in ChatGPT Logs and Memory

A large-scale computational audit of ChatGPT conversations across India, Nigeria, Brazil, and Pakistan revealed significant disclosures of personal health information, with 21.31% of audited conversations containing such data. A notable 3.62% of these conversations carried high-to-extreme privacy risks due to the inclusion of stigmatized conditions, direct identifiers, and precise locations. The analysis also identified a substantial disconnect between corporate communication and the system's actual behavior regarding memory entries, as over 95% of profile entries were implicitly extracted.

Why it matters

This research highlights significant and widespread privacy risks associated with the use of conversational AI platforms, particularly concerning sensitive personal health information. Such vulnerabilities can erode user trust, expose organizations to regulatory non-compliance, and necessitate a re-evaluation of data handling policies and technology governance for AI systems.

Key insights

• 21.31% of 179,057 audited conversations across four countries contained personal health data.
• 3.62% of these conversations posed high-to-extreme privacy risks, involving stigmatized conditions, direct identifiers, and precise locations.
• The study audited conversations from India, Nigeria, Brazil, and Pakistan.
• An evaluation of ChatGPT's memory entries showed that over 95% of profile entries were implicitly extracted, contrasting with corporate framing of system behavior.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.14697