Interrupting the Chain: Human Perception of AI-Generated Disinformation Through a Kill Chain Lens

A recent study investigates human perception of AI-generated disinformation, utilizing a cybersecurity kill chain framework. Key findings indicate a significant perception-accuracy gap, where increased suspicion of AI-generated content does not correlate with improved detection. Modern large language models (LLMs) are shown to produce text largely indistinguishable from human-generated content, and prolonged engagement with detection tasks leads to cognitive fatigue, degrading accuracy in identifying fake news. These insights highlight critical challenges in combating AI-driven disinformation campaigns.

Why it matters

The ability of AI to generate indistinguishable disinformation at scale presents a formidable challenge to information integrity and public trust. These findings underscore the need for advanced countermeasures beyond human discernment, impacting strategic approaches to digital security, public communications, and regulatory frameworks globally.

Key insights

• There is a perception-accuracy gap where heightened suspicion of AI-generated content does not improve detection rates.
• Modern Large Language Models (LLMs) frequently produce text that is indistinguishable from human-generated content.
• An asymmetric cognitive fatigue effect causes fake-news detection accuracy to degrade by 10.2 percentage points under sustained effort.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2608.21389