Research Summary: Citing Less Critically: LLMs Reshape the Rhetoric and Reach of Scientific Citation

Research indicates that Large Language Models (LLMs) used in scientific writing may alter the rhetorical intent of citations compared to human authors. A study employing a masked-citation task found that LLM-generated citation sentences differ in what, whom, and how they cite, potentially influencing the perceived support, contrast, or neutrality of cited works.

Why it matters

This research is strategically important because the increasing integration of LLMs into scientific communication could fundamentally reshape how knowledge is attributed, supported, or challenged. It highlights potential shifts in the rhetorical landscape of academic discourse, impacting the perception of scientific consensus and dissent within and across disciplines.

Key insights

• Scientific citations inherently carry rhetorical intent, categorized as positive (supporting), negative (contrasting), or neutral (mentioning).
• The study investigates whether Large Language Models (LLMs) reproduce citation rhetorical intent similarly to human authors.
• A masked-citation task was developed to compare human and LLM-generated citation behaviors, producing counterfactual corpora.
• Analysis focuses on 'what,' 'whom,' and 'how' models cite.
• An LLM-as-a-judge method was used to classify citation intent.
• Social distance between cited authors was measured using a 20-million-edge coauthorship network.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.01432