Research Summary: When Disability Disclosure Travels: Memory, Privacy, and Contextual Integrity in Conversational AI

A recent research study investigated how individuals with disabilities in the United States interact with large language model (LLM)-based conversational AI, focusing on the disclosure of disability information. The study found that users often disclose disability by translating it into task-specific instructions rather than explicit identification, driven by immediate needs. A key observation was the tension between perceiving AI as a non-judgmental conversational partner and a data-holding entity, leading to conflicting norms around disclosure. The research highlights that AI's memory features can alleviate repeated disclosures, but raise concerns about privacy and the contextual integrity of shared information.

Why it matters

This research reveals critical insights into user behavior and privacy perceptions regarding sensitive personal information shared with AI. Understanding the nuanced ways individuals disclose disability, and their concerns about data handling, is vital for developing ethical AI systems and data governance frameworks. It underscores the need for strategies that balance utility with privacy and trust.

Key insights

• Disabled individuals often disclose their disability to conversational AI systems based on need, translating it into task-scoped instructions rather than direct identification.
• Users evaluate disclosure against two distinct entities: the AI as a non-judging interlocutor and the company as a data holder, which elicits contradictory norms regarding information sharing.
• The memory features of LLM-based assistants can reduce the burden of repeated disclosure for users.
• The study specifically focused on LLM-based assistants like ChatGPT, Claude, and Gemini.
• The research used contextual integrity as an analytic framework to understand disclosure practices and privacy perceptions.
• Participants were 12 adults with disabilities in the United States who use these AI systems.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.22720