Compression Asymmetry and Trajectory Binding in Noise-Anchored Diffusion Inversion

Abstract: Real-image diffusion inversion is governed by a tight quality-cost trade-off, with costs incurred in computation, storage, or per-image optimization. We study this trade-off through the forward Gaussian noise anchor that defines a diffusion trajectory and isolate two mechanisms behind effective stored-noise inversion. First, diffusion noise exhibits an element-wise compression asymmetry: int8 full-dimensional anchors preserve reconstruction, whereas low-dimensional subspace summaries are much less reliable, often collapsing even at comparable or smaller payloads; the element-wise over subspace ordering persists across five stored-noise inversion methods. Second, inversion is trajectory-bound and score-prior coupled: the matched forward anchor and a trained score network are both necessary, arguing against a purely algebraic-identity explanation. Together, these findings specify what to store and how to use it. They lead to Noise-Anchored Reverse Correction (NARC), a training-free inversion primitive that stores a single int8 latent anchor and reuses it with a fixed, noise-level-dependent anchor-weight schedule: strong anchoring when the reverse trajectory is noise-dominated, then relaxed anchoring as image detail emerges. On PIE-Bench++ with Stable Diffusion 1.5, NARC outperforms five modern non-exact baselines and improves PSNR by +3.24 dB over PnP DirectInv while using about 400x less inversion storage than PnP DirectInv. The compression asymmetry, anchor specificity, and editing plug-in also transfer to SDXL 1024^2.

• The paper identifies an element-wise compression advantage in diffusion noise anchors and shows that int8 full-coordinate storage preserves reconstruction quality, while subspace summaries lose up to 1.41 dB PSNR and worsen LPIPS.
• The paper demonstrates trajectory binding and score-prior coupling by showing that matched anchors, trained denoisers, and intermediate correction weights are essential for high-fidelity inversion rather than algebraic cancellation alone.
• The paper introduces NARC, a training-free method using a scheduled anchor blend that reaches 24.72 dB PSNR with a 16 KiB cache, outperforming non-exact baselines by 7.77–9.79 dB while using 32–75% fewer UNet calls.

Overview

This paper studies real-image diffusion inversion through the lens of the cached forward-noise anchor that defines a diffusion trajectory. Rather than proposing another point on the conventional "quality versus NFE/storage/per-image optimization" trade-off axis, the authors ask what the closed-form DDPM forward marginal already reveals: sampling $\epsilon \sim \mathcal{N}(0,I)$ once per image and forming $$x_T = \sqrt{\bar\alpha_T}\,x_0 + \sqrt{1-\bar\alpha_T}\,\epsilon$$ yields a timestep-independent anchor of the trajectory, which can be blended with the model's noise prediction at every reverse step. The paper's stated central contribution is two empirical mechanism findings about this anchor, from which the method—Noise-Anchored Reverse Correction (NARC)—is derived.

Finding 1: element-wise versus subspace compression asymmetry

The first finding contradicts the standard image-processing intuition that subspace projections (DCT low bands, low-rank or random-projection summaries) are the efficient way to compress structured signals. For diffusion noise anchors, the ordering reverses. Under a strict cache-accounted fp32-runtime audit—where the stored payload is decoded once and used both to rebuild the reverse start and in the correction term—int8 quantization of the full anchor is statistically indistinguishable from the fp32 anchor (e.g., $\Delta$PSNR $=+0.00008$ dB, $p=0.894$ on SD 1.5; $+0.00006$ dB, $p=0.744$ on SDXL). Packed int4 stays within 0.12 dB. By contrast, DCT, random-projection, spatial-mask, and block-average summaries at comparable or smaller payloads lose 0.99–1.41 dB PSNR and worsen LPIPS by 0.156–0.180 on SD 1.5. On SDXL the failure mode is subtler and the paper is careful about it: support-reduced summaries can raise PSNR and SSIM through smoothing while worsening LPIPS by 0.042–0.060, so the robust claim is Pareto dominance of full-coordinate retention, not universal PSNR collapse.

Crucially, the authors test the same compression ladder across five stored-noise inversion methods—NARC's sampled Gaussian anchor, Edit-friendly DDPM's $T{+}1$ noise maps, LEDITS++'s stored maps, DDIM-Inv's terminal latent, and PnP DirectInv's residual list—and the element-wise-over-subspace ordering holds in all five, with a three-tier magnitude gradient (strong / transition / boundary) governed by two conditions: per-dimension cache magnitude must exceed the quantization step, and cache information must be isotropically dispersed across ambient space. This makes the asymmetry a paradigm-wide property of the noise latent space rather than an artifact of the proposed method.

Finding 2: trajectory binding and score-prior coupling

The blend rule admits a degenerate algebraic identity at $\lambda_t \equiv 1$: substituting into the DDIM step under the forward-trajectory assumption exactly cancels the model term and recovers the SD VAE reconstruction ceiling (27.00 dB). The authors therefore run three independent tests to determine whether their operating points are merely numerical relatives of this identity, and all three refute that hypothesis at the tested settings.

Anchor specificity. The trajectory-matched forward anchor reconstructs at 24.72 dB on the main schedule, while random, mismatched-image, shuffled, and sign-flipped anchors collapse to 6.49–6.55 dB—a paired gap of +18.17 to +18.23 dB (and +20.74 to +20.83 dB on SDXL). The authors note honestly that wrong-anchor collapse alone does not falsify the algebraic hypothesis, since it is also predicted algebraically for $\lambda < 1$.

Trained-score dependence. Replacing the UNet with $$x_T = \sqrt{\bar\alpha_T}\,x_0 + \sqrt{1-\bar\alpha_T}\,\epsilon$$0 random weights collapses reconstruction by −11.75 dB on 140/140 images ($$x_T = \sqrt{\bar\alpha_T}\,x_0 + \sqrt{1-\bar\alpha_T}\,\epsilon$$1), monotone in $$x_T = \sqrt{\bar\alpha_T}\,x_0 + \sqrt{1-\bar\alpha_T}\,\epsilon$$2 (−7.90 to −13.39 dB across fixed-$$x_T = \sqrt{\bar\alpha_T}\,x_0 + \sqrt{1-\bar\alpha_T}\,\epsilon$$3 comparators). An algebraic identity would be insensitive to UNet weights. The claim is explicitly scoped: it is not a theorem over arbitrary denoisers.

Unimodal cfg sensitivity. The cfg-induced PSNR drop peaks near $$x_T = \sqrt{\bar\alpha_T}\,x_0 + \sqrt{1-\bar\alpha_T}\,\epsilon$$4 at 2.60 dB and decays to 0.81 dB at $$x_T = \sqrt{\bar\alpha_T}\,x_0 + \sqrt{1-\bar\alpha_T}\,\epsilon$$5; the main schedule retains a positive paired drop of +0.694 dB, confirming a model-coupled regime rather than the algebraic endpoint band.

The NARC method

NARC operationalizes both findings as a training-free primitive: a closed-form forward jump (zero forward UNet calls), followed by deterministic DDIM reverse sampling in which the classifier-free-guided noise prediction is blended with the cached anchor via $$x_T = \sqrt{\bar\alpha_T}\,x_0 + \sqrt{1-\bar\alpha_T}\,\epsilon$$6. Finding 1 dictates what to store (a 16 KiB int8 anchor for SD 1.5 latents); Finding 2 dictates how to use it (an intermediate, noise-aware $$x_T = \sqrt{\bar\alpha_T}\,x_0 + \sqrt{1-\bar\alpha_T}\,\epsilon$$7 with the model in the loop).

The schedule keys on the noise coefficient: $$x_T = \sqrt{\bar\alpha_T}\,x_0 + \sqrt{1-\bar\alpha_T}\,\epsilon$$8 with $$x_T = \sqrt{\bar\alpha_T}\,x_0 + \sqrt{1-\bar\alpha_T}\,\epsilon$$9, giving strong anchoring while the reverse trajectory is noise-dominated and relaxation as image detail emerges. Three geometric observations motivate this shape: anchor-error leverage on $\Delta$0 scales with $\Delta$1, the $\Delta$2-prediction model is least reliable at large $\Delta$3, and a fixed high $\Delta$4 suppresses the model's accumulated knowledge at low noise, producing perceptual blur (LPIPS 0.504 at fixed $\Delta$5 versus 0.147 under the schedule). A mean-$\Delta$6 matched fixed control ($\Delta$7) reaches only 22.99 dB versus 24.72 dB, confirming that schedule shape, not average anchor weight, drives the gain. NARC uses $\Delta$8 UNet calls with the CFG split and stores only the int8 anchor.

Main results

On PIE-Bench++ (SD 1.5, cfg 7.5, $\Delta$9, paired Wilcoxon by image ID), NARC is paired +7.77 to +9.79 dB above DDIM-Inv, ReNoise, DCI, FreeInv, and taba on PSNR (all $=+0.00008$0, the test floor), with LPIPS gains of −0.185 to −0.351, while using 32–75% fewer UNet calls. Within its own paradigm, it reverses the gap to PnP DirectInv: +3.24 dB ($=+0.00008$1) at roughly $=+0.00008$2 less storage (16 KiB versus 6.4 MB). Against the large-cache or exact/optimization ceilings—Edit-friendly DDPM (3.2 MB, 26.70 dB), EDICT (26.98 dB), and Null-text (25.98 dB)—NARC remains 1.16–2.25 dB below, though the schedule shrinks these gaps by roughly 4× relative to the fixed-$=+0.00008$3 configuration. Auxiliary cross-checks are consistent: FID drops from 146.88 at the conservative reference to 43.32 under the main schedule, and CLIP-I reaches 0.9538, above the modern non-exact baselines and PnP DirectInv but below the large-cache references. The compression asymmetry, anchor specificity, and editing plug-in transfer to SDXL at $=+0.00008$4 (27.12 dB PSNR, 0.211 LPIPS under $=+0.00008$5). In editing, the mask-background variant improves background PSNR by about +5.1 dB and reduces background LPIPS by 0.29–0.31 on both backbones while keeping CLIP-T within 0.009–0.010 of the no-NARC baseline; whole-image blending achieves better raw background fidelity but degrades target alignment.

Limitations and open questions

The paper is explicit about several boundaries. First, the 16 KiB figure is additional-cache accounting for retained-source editing; if the source image or clean diffusion state is discarded, $=+0.00008$6 or equivalent information must also be stored, so the claim does not apply to standalone archival compression. Second, int8 is the only fully validated element-wise scheme; packed int4 passes the compression audits but has not been revalidated across the full reconstruction/editing suite. Third, the SDXL results expose a metric split in which support-reduced summaries can raise PSNR while worsening LPIPS, complicating any single-metric summary of Finding 1. Fourth, transfer to DiT-based backbones rests on a suggestive but weak SD 3/MMDiT probe: the trained-versus-random gap is only +4.87 dB there, versus −11.75 dB for the SD UNet, and the flow-matching coordinate cannot inherit the DDIM $=+0.00008$7 schedule verbatim. Fifth, deeper integration with editing frameworks such as P2P/MasaCtrl remains future work, and the mechanism claims are scoped to the tested samplers, schedules, and CFG settings rather than stated as theorems. An open question the paper leaves is whether the two-condition account of the compression gradient (magnitude above quantization step, isotropic dispersion) can be made predictive for arbitrary cached objects rather than post hoc.

Conclusion

The paper links two empirical mechanism findings—an element-wise-over-subspace compression asymmetry in diffusion noise caches, and trajectory binding with score-prior coupling at the tested operating points—to a minimal training-free inversion primitive. NARC establishes a 16 KiB extreme-compression operating point within the stored-noise paradigm, outperforming five modern non-exact baselines by +7.77 to +9.79 dB and PnP DirectInv by +3.24 dB at $=+0.00008$8 less storage, while conceding the remaining gap to large-cache and exact-inversion ceilings. The mechanism-to-method structure—what to store, and how to couple it back into the reverse trajectory—is the paper's substantive contribution beyond the raw operating point.