Research Summary: Interrupting the Chain: Human Perception of AI-Generated Disinformation Through a Kill Chain Lens

Generative AI facilitates the production of customized misinformation at scale, outpacing current reactive defense mechanisms. A human-subject study involving 504 participants demonstrated a significant challenge in distinguishing AI-generated content from human-produced content, and identifying its veracity. Key findings indicate a perception-accuracy gap, the indistinguishability of modern LLM outputs, and an asymmetric cognitive fatigue effect that degrades fake-news detection.

Why it matters

The proliferation of sophisticated AI-generated disinformation presents a significant threat to information integrity, public trust, and decision-making processes across all sectors. Organizations must recognize the diminishing human capacity to accurately discern fabricated content, which can compromise operational security, reputation, and strategic communication efforts.

Key insights

• A perception-accuracy gap exists, meaning increased suspicion among individuals does not correlate with improved detection of AI-generated disinformation.
• Modern Large Language Models (LLMs) are capable of producing text that is frequently indistinguishable from human-generated content.
• An asymmetric cognitive fatigue effect was observed, leading to a 10.2 percentage point degradation in fake-news detection under certain conditions.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2608.21389