Frozen DINO Localizes Image Edits Without a Localizer

Abstract: Localized image edits can change a photograph's meaning while leaving most of it authentic, so forensic analysis must identify where an edit occurred. We show that patch-level perturbation responses from frozen DINO encoders are themselves localization maps. Training-free Localization of AI-image Edits from patch-token Drift (TRAIL) applies one global Haar perturbation and maps cosine drift between corresponding patch tokens. On 80 source-disjoint CocoGlide test images, TRAIL reaches .903 patch AUROC versus .912 for the mask-supervised Detective SAM; fixed-threshold Dice is .619 versus .709, while an oracle threshold raises TRAIL to .790. Transferred unchanged to Poisson image interpolation, TRAIL reaches .855 AUROC versus .864, showing that the cue persists without a generator. Across sixteen DINO encoders, the best block lies at normalized depth .80-.94. Global context matters: AUROC falls from .903 globally to .857 for local-in-canvas perturbations and .735 for independently encoded crops. Frozen DINO patch tokens therefore contain a strong late-layer localization signal whose visibility depends on the perturbation and preserved context. Code: https://github.com/VishalJ99/trail-image-edit-localization.

• The paper introduces TRAIL, a training-free method that compares frozen DINO patch-token embeddings before and after global Haar high-frequency attenuation, achieving .903 patch AUROC on CocoGlide—statistically comparable to the supervised Detective SAM baseline.
• TRAIL’s localization signal transfers beyond generative inpainting, reaching .855 AUROC on Poisson Image Interpolation and .850 on Foreign Patch Interpolation, showing that it can identify classical as well as AI-generated edits.
• The method produces strong soft localization maps but weaker fixed-threshold masks, with .619 Dice versus .790 using per-image oracle thresholds, highlighting calibration and domain transfer as key challenges for practical deployment.

Overview

TRAIL (Training-free Localization of AI-image Edits from patch-token Drift) demonstrates that the patch-level perturbation response discarded by image-level, training-free forgery detectors is itself a usable localization map (2608.18968). The method applies a single global Haar high-frequency attenuation to an image, passes both the original and perturbed versions through a frozen DINO-family encoder, and computes cosine drift between corresponding patch tokens at a late transformer block. No learned head, decoder, or segmentation model is involved; "training-free" means no parameter is fitted to manipulation labels or masks, though the block, spatial filter, and operating threshold are selected on development data.

The paper positions TRAIL against two lines of prior work. Image-level training-free detectors such as RIGID, MINDER, and WaRPAD collapse perturbation sensitivity into a single score per image, while supervised localizers such as TruFor and Detective SAM are trained directly from masks. The central question is narrow: how much localization capability already exists in the perturbation response of a frozen general-purpose vision encoder?

Method

Let $f_{\ell,p}(x)$ denote patch token $p$ after block $\ell$ of a frozen encoder. The perturbation removes a fraction of the second-level Haar detail component from the full canvas:

$$T(x) = x - \alpha\,\mathrm{HF}_2(x), \qquad \alpha = 0.2$$

The score at each patch is $$s_\ell(p) = 1 - \cos(f_{\ell,p}(x), f_{\ell,p}(T(x)))$$, reshaped to the token grid and passed through one reflect-padded $3\times3$ median filter. The headline configuration uses DINOv3 ViT-7B/16 at $448\times448$ resolution and block 36, yielding a $28\times28$ map with a development-selected threshold $\tau=0.0026$. Inference requires one wavelet transform and two frozen-encoder passes. A per-image oracle threshold, which maximizes Dice against ground truth for each map individually, is reported as a diagnostic of calibration headroom rather than as an achievable operating point.

Main results

On 80 source-disjoint CocoGlide test images (GLIDE inpainting edits of COCO sources), TRAIL reaches .903 patch AUROC versus .912 for Detective SAM, with a paired 95% bootstrap interval crossing zero ($[-.049,.033]$). This near-parity between a fully unsupervised readout and a mask-supervised segmentation pipeline is the paper's strongest claim. Controls rule out trivial explanations: identically filtered raw-pixel change reaches only .641 AUROC, and an aligned-real control—running TRAIL on the unedited source and applying the edit mask post hoc—reaches .697. The fake-minus-aligned-real increment of +.206 [.167,.246] indicates the ranking is specific to the edit rather than to pixel-space perturbation magnitude or ordinary content in the edited region.

Thresholded overlap tells a different story. At prespecified operating points, TRAIL's fixed Dice (.619) trails Detective SAM's (.709) by .090, a gap excluding zero. However, the per-image oracle raises the same TRAIL maps to .790 Dice, an increase of .171 [.134,.209]. Oracle thresholds span .00033 to .01342 across images, so the development-frozen cut misses in both directions; the soft maps remain well placed while single-threshold calibration does not. Grid resolution is not the bottleneck: the best achievable Dice for any binary $p$0 map averages .926 against the pixel masks.

The paper also tests whether the cue depends on generator artifacts. With every setting transferred unchanged, Poisson Image Interpolation—a classical edit constructed without any generative model—yields .855 AUROC versus .864 for Detective SAM, and Foreign Patch Interpolation yields .850 versus .894. Patch-token drift therefore carries spatial information even when no generator produced the edit.

TGIF2 provides the main boundary condition. Across 720 splices, TRAIL reaches .774 AUROC versus .839 for Detective SAM, with the deficit concentrated on semantic-mask splices ($p$1 AUROC, $p$2 oracle Dice, both excluding zero); on random masks all paired intervals include zero. Notably, the aligned-real control rises to .743 under semantic masks, meaning some drift inside semantic object regions exists before editing. Because mask geometry, semantics, and generator co-vary between groups, the authors correctly frame this as a regime difference rather than a causal finding about semantics.

Depth and scale across sixteen encoders

Across sixteen DINO-family encoders spanning ViT-S to ViT-7B, the development-selected block lies at normalized depth .80–.94 in every model tested, regardless of pretraining recipe. The choice is locally stable: blocks 35–37 of DINOv3-7B give .9029, .9037, and .9032 development AUROC.

Model size affects a different quantity more consistently than raw localization. In DINOv3, fake-image AUROC barely moves past ViT-B (.885, .893, .888, .903 for B/L/H+/7B), while the fake-minus-real increment rises monotonically from .089 at ViT-S to .206 at 7B. DINOv2-with-registers shows increments up to .293 at ViT-g, exceeding plain DINOv2's .264 at g—but the ordering reverses at small scales, which rules out a uniform register advantage despite consistency with register tokens absorbing high-norm artifacts. DINO itself shows no scaling trend (increments of .051 and .043). Raw ranking and edit-specific contrast select different encoders: DINOv3-7B has the highest fake-image AUROC, DINOv2-g-with-registers the largest increment. Capacity therefore does not uniformly raise the same localization score; where a trend exists, it more reliably separates edit-associated drift from baseline content drift.

Perturbation geometry matters more than dose

A sweep over three perturbations (Haar attenuation, Gaussian noise, Gaussian blur) crossed with three application geometries shows a consistent ordering: global application beats local-in-canvas application beats independently encoded crops, for every perturbation and on both datasets. Haar moves from .903 to .857 to .735 AUROC; noise from .798 to .738 to .527; blur from .830 to .646 to .610. Configuration ranks agree across CocoGlide and TGIF2 (Spearman $p$3, $p$4).

The three geometries disentangle perturbation locality from encoding context. The largest degradation occurs when the surrounding image is removed during encoding, indicating that corresponding tokens must be compared within shared full-image context for drift to function as a localization signal. This contrasts with WaRPAD's image-level result, where independent crops help detection: a stitched version of WaRPAD's construction reaches only .808 AUROC as a localization map here. The exact Haar dose is comparatively unimportant—$p$5 yields .900–.908 development AUROC.

Limitations

The method outputs a coarse $p$6 map rather than full-resolution segmentation, and the headline configuration requires two passes through a 7-billion-parameter encoder. The supervised reference is block-averaged to the same grid, so the comparison does not capture Detective SAM's native-resolution advantage. The oracle threshold consumes ground truth and is diagnostic only; selecting a stable threshold across new domains remains open. The method is training-free in parameters but not validation-free, since block, filter, and threshold were tuned on development masks. Evaluation covers inpainting-style edits, TGIF2 splices, and two classical families; global retouching, fully generated images, recompression, resizing, and adversarial post-processing are untested. The TGIF2 semantic/random comparison confounds several factors and should not be read causally.

Conclusion

The paper establishes that a late-layer localization signal is already present in frozen DINO patch tokens before any localization model is fitted: reading cosine drift under a single global Haar perturbation achieves .903 AUROC on CocoGlide, statistically indistinguishable from a mask-supervised localizer, and transfers to generator-free edits. The signal is consistently late-layer across sixteen encoders, scales more reliably in edit-specific contrast than in raw ranking, and degrades sharply when global encoding context is removed. The clearest open problem left by the results is threshold calibration: the spatial ordering is strong, but no single fixed cut recovers it as binary masks across images.