🚨 NeurIPS 2024 🚨How robust are AI-Generated Image Detectors?
🤔 Can they detect various magnitudes of image augmentations?
💡 Does performance fluctuate across scenes?
🚀 Find out with Semi-Truths: 1.5 million images for the targeted evaluation of AI-generated images. https://arxiv.org/abs/2411.07472
🤔 Can they detect various magnitudes of image augmentations?
💡 Does performance fluctuate across scenes?
🚀 Find out with Semi-Truths: 1.5 million images for the targeted evaluation of AI-generated images. https://arxiv.org/abs/2411.07472
Comments
🤔 However, the majority of the SOTA systems are trained exclusively on end-to-end fully generated images, or on data from very constrained distributions.
🔍 One such case is known as “Sleepy Joe”, where a video of Joe Biden was changed only in the facial region to make it appear as though he fell asleep at a podium.
It includes a wide array of scenes & subjects, as well as various magnitudes of image augmentation. We define “magnitude” by size of the augmented region and the semantic change achieved.
We perturb entity & image captions with LLMs, then apply different diffusion models and augmentation techniques to alter images.
💡 UniversalFakeDetector suffers >35 point performance drop on different scenes, and >5 points on magnitude of change.
🤗HF: https://huggingface.co/semi-truths
👾Github: https://github.com/J-Kruk/SemiTruths/