MultiGhostBench: A Multilingual Benchmark for Long-Form LLM-Generated Text Attribution under Distribution Shifts

Recent research introduces MultiGhostBench, a comprehensive multilingual benchmark for attributing long-form, Large Language Model (LLM)-generated text. Comprising 928 books across five LLMs, six languages, and three scripts, the benchmark addresses limitations of existing tools which are often restricted to English, controlled environments, or older models. Initial evaluations indicate that current attribution methods struggle with consistency and experience performance degradation when confronted with distribution shifts in domain, author, or language.

Why it matters

The development of robust methods for identifying LLM-generated content is crucial for maintaining trust in information, intellectual property, and content integrity across various sectors. The demonstrated weaknesses of current attribution methods under diverse conditions highlight a significant vulnerability that necessitates strategic investment in research and development to mitigate potential risks associated with unverified LLM output.

Key insights

• Existing LLM authorship attribution (AA) benchmarks are limited, primarily focusing on English, controlled settings, or outdated models, with short text considerations in multilingual studies.
• MultiGhostBench is a new multilingual benchmark containing 928 books generated by five recent LLMs across six languages and three scripts.
• The average length of texts in MultiGhostBench is approximately 59,000 words per book, significantly longer than previous benchmarks.
• The benchmark supports robust evaluation of attribution methods under domain, author, and language distribution shifts.
• Evaluation of representative AA methods shows no single method consistently outperforms others across all settings.
• Performance of AA methods generally degrades significantly when encountering distribution shifts.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.02379