Local AI pre-screening for human triple-blind peer review in health sciences

Academic peer review systems, particularly in major conferences like NeurIPS, ICLR, and ICML, are experiencing significant strain due to an exponential increase in submission volume that exceeds the availability of qualified human reviewers. This has led to the undisclosed integration of large language models (LLMs) into the review process, with independent analysis indicating substantial AI involvement, including fully AI-generated reviews. This trend introduces risks such as hallucinated citations and malicious prompt injection within manuscripts to manipulate AI reviewers. A multi-LLM pre-screening framework is proposed to address these challenges.

Why it matters

The integrity and efficiency of peer review processes are critical for the advancement of research and the credibility of academic output across all sectors. The unmanaged introduction of AI into these processes poses significant risks to quality assurance, ethical standards, and trust in published knowledge, necessitating strategic consideration of AI integration and robust countermeasures to safeguard research validity.

Key insights

• Academic peer review systems are under significant strain due to a rapid increase in submission volumes, outstripping the supply of qualified human reviewers.
• Large Language Models (LLMs) are increasingly, and often undisclosed, being used in the peer review process; for ICLR 2026, approximately 21% of reviews were fully AI-generated, and over half showed some AI involvement.
• The use of AI in peer review introduces documented risks, including hallucinated citations in accepted papers and hidden prompt-injection instructions designed to manipulate AI reviewers.
• A triple-blind, multi-LLM pre-screening framework is being developed to address these issues, specifically for health sciences, though the full scope is not detailed in the abstract.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2608.14625