Research Summary: Evaluating Personal Information Output from Conversational Interactions in Generative AI Systems

An exploratory pilot study evaluated the scope and perceived accuracy of personal information output from generative AI systems (GPT-5.2 Instant and GPT-5.2 Thinking) during conversational interactions. The study, involving 15 Japanese participants, categorized outputs as Fact, Inference, and Confidence. Key findings indicate that model design differences had limited impact on personal information output patterns. 'Fact' type outputs demonstrated a more conservative pattern than 'Inference' types. Core Personal attributes, such as identification information, were handled conservatively, while Behavioral and Linguistic attributes showed higher output rates.

Why it matters

This research highlights the varying propensity of generative AI systems to output different categories of personal information during conversations. Understanding these tendencies is crucial for developing robust data governance frameworks and privacy safeguards, particularly as these systems become more integrated into business operations and public-facing services. Organizations must assess the risks associated with such outputs to maintain trust and ensure compliance with data protection regulations.

Key insights

• Differences in generative AI model design (GPT-5.2 Instant and GPT-5.2 Thinking) had limited impact on personal information output tendencies during conversational interactions.
• Personal information outputs were categorized into Fact, Inference, and Confidence types.
• The 'Fact' output type exhibited a more conservative output pattern for personal information compared to the 'Inference' type.
• Core Personal attributes, those associated with identification, were treated relatively conservatively by the AI systems.
• Behavioral and Linguistic attributes showed higher rates of personal information output from the AI systems.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.22204