Mitigating GenAI-Powered Evidence Pollution for Out-Of-Context Misinformation Detection

Generative Artificial Intelligence (GenAI) models pose a growing threat to online information security due to their capacity to create deceptive content. Specifically, GenAI is being used to 'pollute' evidence used by out-of-context (OOC) multimodal misinformation detection systems. Traditional detection methods, which rely on web-retrieved evidence, are increasingly challenged by this GenAI-driven evidence pollution, as existing work often assumes a 'clean' evidence corpus.

The proliferation of GenAI-generated deceptive content undermines trust in digital information and the effectiveness of established misinformation detection mechanisms. Addressing this threat is critical for maintaining information integrity, protecting brand reputation, and ensuring the reliability of data used for decision-making across various sectors.

GenAI models are increasingly used to generate deceptive content, raising concerns for online information security.

Forward consideration, not a verified fact.