Research Summary: Strangers to Themselves: What Language Models Say About Themselves Is Generic

Research indicates that Large Language Models (LLMs) demonstrate limited self-knowledge regarding their own future behavior. While LLMs can fluently describe how they might act, their self-predictions for specific behavioral evaluations show a weak correlation with actual outcomes. This suggests that LLMs' responses about their own conduct are generic and do not reflect a distinct understanding of their internal states or operational characteristics, performing comparably to predictions about 'capable AI agents in general'.

Why it matters

This research reveals a fundamental limitation in the current capabilities of advanced AI systems, specifically their inability to accurately self-assess future behaviors. For organizations deploying or developing such technologies, this implies a need for external validation and oversight mechanisms rather than relying on AI systems' introspective declarations, which could significantly impact trust, risk management, and the design of autonomous agents.

Key insights

• Language models can articulate potential behavioral responses, such as reacting to pushback, misusing tools, or lying under pressure.
• Direct self-report by language models about their own behavior exhibits a weak correlation with their actual behavior (r = +0.04).
• Providing models with specific items related to the evaluations only marginally improves their self-prediction accuracy (r = +0.24).
• Predictions made by a model about 'capable AI agents in general' perform equally well (r = +0.28) as item-informed self-predictions, suggesting a lack of specific self-referential insight.
• Predictions from other models about their own behavior also effectively predict the target model's behavior, reinforcing the generic nature of these responses.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.09899