Mechanism underlying Gemini’s comparison noise

Determine whether Gemini’s reduced accuracy in pairwise real-versus-synthetic image classification results from instruction-tuning that makes Gemini assess visual differences between paired images differently from single images in isolation, by replacing the authentic FFHQ reference with a second synthetic image in a controlled experiment.

Background

In the pairwise experiment, Gemini 2.5 and Gemini 3 performed worse when shown one authentic image alongside one synthetic image, contrary to the expected contextual advantage. The paper labels this possible interference “comparison noise,” but does not establish its causal mechanism.

The authors propose that Gemini’s instruction-tuning may cause it to process visual differences between paired images differently from a single image. A controlled follow-up using two synthetic images instead of an FFHQ image would test this proposed explanation.

References

Comparison noise remains a proposed explanation rather than a confirmed mechanism. One alternative is that Gemini’s instruction-tuning causes it to assess visual differences between paired images differently than it processes a single image in isolation, a hypothesis that could be isolated by replacing the FFHQ control group image with a second synthetic image in a controlled follow-up experiment.

— Beyond Generation: Using Multimodal LLMs as Detectors of LLM-Generated Deepfake Facial Images  (2610.21760 - Salamon et al., 15 Oct 2026) in Section 5.5, Future Work