Safety-Flag: A Unified Benchmark for the Reliability and Calibration of LLM Content Moderators

This research introduces Safety-Flag, a unified benchmark designed to assess the reliability and calibration of Large Language Models (LLMs) when used for content moderation. It consolidates seven existing safety benchmarks into a standardized flag/do-not-flag protocol, enabling a more comprehensive evaluation beyond simple aggregate accuracy. The study highlights that conventional accuracy metrics fail to capture critical aspects of moderator reliability, such as error direction, probability calibration, and the utility of confidence scores for human review, which often diverge.

The increasing reliance on LLMs for content moderation necessitates robust and multidimensional evaluation methods to ensure their reliability and ethical deployment. This research offers a standardized framework to better understand the nuanced performance of moderation systems, moving beyond simplistic accuracy metrics to address critical aspects like calibration and error types, which are vital for trust and effective risk management.

Safety-Flag unifies seven existing safety benchmarks (BeaverTails, XSTest, Ethics, WildGuard, Aegis, ToxiChat, ToxiGen) into a single, balanced flag/do-not-flag content moderation protocol.

Forward consideration, not a verified fact.