Research Summary: Used, Mentioned, or Condemned? A Controlled Contrast-Set Diagnostic for the Use-Mention Distinction in Code-Mixed Hinglish Misogyny Detection

Existing lexicon-driven misogyny detection systems struggle to differentiate between the harmful use of slurs and their mention in contexts like counter-speech, which can lead to misclassification and unintended moderation outcomes. This research diagnoses this issue within code-mixed Hinglish, identifies evaluation artifacts in public datasets, and contributes to addressing this critical distinction for effective content moderation.

Why it matters

The inability of current detection systems to differentiate between the use and mention of harmful language undermines the effectiveness of content moderation strategies. This issue impacts user safety, freedom of expression, and the perceived fairness of digital platforms, necessitating a more nuanced approach to algorithmic content analysis.

Key insights

• Lexicon-driven misogyny detectors are inherently unable to distinguish between the use of a slur (misogynistic) and its mention (e.g., in counter-speech).
• This inability governs whether content moderation effectively protects individuals or inadvertently silences discussions about abuse.
• Two evaluation artifacts were identified in a publicly available corpus: category-encoding anonymization placeholders that leak labels, and lexically disjoint registers between misogynistic and benign comments.
• These artifacts can lead to inflated performance metrics (e.g., macro-F1 of approximately 1.00 for bag-of-words models) under random cross-validation, which collapses under more robust evaluation methods.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2609.22261