EditInfinity: Text-Driven Image Editing

• EditInfinity is a text-driven image editing framework that uses binary-quantized tokens to provide exact inversion supervision and bypass common diffusion-based approximation errors.
• It combines text prompting rectification, LoRA-based style preservation, and a holistic smoothing strategy for spatially controlled, multi-scale token editing.
• Evaluations on PIE-Bench demonstrate its superior performance in add, change, and delete operations, balancing source fidelity with precise text prompt alignment.

EditInfinity is a text-driven image editing framework that adapts Infinity, a binary-quantized autoregressive generative model, for image editing rather than following the more common diffusion-model pipeline. Its defining premise is that VQ-style discrete latent representations allow exact inversion supervision, thereby avoiding the approximation errors that usually constrain diffusion-based editing. The method combines an inversion stage based on exact quantized token targets with text prompting rectification, image style preservation via LoRA, and a holistic smoothing strategy for token-space editing. On PIE-Bench, the framework is reported to achieve strong performance across “add”, “change”, and “delete” operations while preserving high fidelity to the source image and aligning edits precisely with target text prompts (Wang et al., 23 Oct 2025).

1. Problem formulation and motivation

EditInfinity is situated within the general paradigm of training-free image editing methods that first perform image inversion and then perform editing along the inferred generative trajectory. In this standard formulation, a source image is reconstructed into the model’s latent process and then modified under a target text prompt. The paper identifies a central limitation of this paradigm for diffusion models: the exact forward trajectory that produced the source image is unknown, so inversion must be approximated with deterministic samplers or optimization-based finetuning. These approximation errors in intermediate latent states degrade editing quality and create a persistent trade-off between source fidelity and editability (Wang et al., 23 Oct 2025).

The paper characterizes the downstream effects of inversion error as including poor reconstruction of the source image, drift in background/unchanged regions, weak semantic alignment to the target prompt, and a general trade-off between fidelity and editability. Within this framing, EditInfinity does not primarily claim a stronger base generator; rather, it attributes its improvement to replacing approximate diffusion inversion with an editing framework in which intermediate targets are exactly available in a discrete latent space (Wang et al., 23 Oct 2025).

This positions EditInfinity against a classical diffusion-editing bottleneck. A plausible implication is that the framework treats editing quality as fundamentally limited by the invertibility of the underlying generative representation, not only by the strength of the editing guidance.

2. Binary-quantized foundation and exact inversion supervision

The framework uses Infinity, described as a binary-quantized autoregressive text-to-image model, as its base generator. Its key property for editing is that images are tokenized into discrete binary quantized residual tokens, so the exact latent tokens of a source image can be obtained directly by quantization. The paper contrasts this with diffusion models, where inversion remains approximate (Wang et al., 23 Oct 2025).

Infinity is presented as a bitwise multi-scale residual quantization model. For an image $I \in \mathbb{R}^{H \times W \times 3}$ encoded into a feature $F$, residuals are formed across scales as

$$z_k = \text{down}\left(F - F_{k-1}, (h_k, w_k)\right) \in \mathbb{R}^{h_k \times w_k \times d}.$$

The model then applies Binary Spherical Quantization (BSQ):

$$R_k = \mathcal{Q}(z_k) = \frac{1}{\sqrt{d}} \operatorname{sign}\left(\frac{z_k}{\|z_k\|}\right),$$

with cumulative feature reconstruction

$F_k = \sum_{i=1}^k \text{up}(R_i, (h_K,w_K)).$

Autoregressive modeling proceeds across scales:

$$p(R_{1:K}|\Psi(t)) = \prod_{k=1}^K p\big(R_k \mid R_1,\dots,R_{k-1}, \Psi(t)\big),$$

where $\Psi(t)$ is the text embedding from the text encoder, identified in the paper as Flan-T5 (Wang et al., 23 Oct 2025).

These formulations matter because EditInfinity’s inversion stage operates against exact token targets. Given a source image $I_{sou}$ and source prompt $t_{sou}$, the method first quantizes the image into exact source tokens $R_{1:K}^{sou}$, then optimizes a learnable prompt to reconstruct those tokens using the inversion loss

$F$0

The paper’s central claim is that, unlike diffusion inversion, this objective uses cross-entropy over token predictions with exact supervision. This suggests that inversion is recast from approximate latent trajectory recovery into supervised token reconstruction, which directly affects downstream edit fidelity (Wang et al., 23 Oct 2025).

3. Inversion mechanism: prompt rectification and style preservation

EditInfinity augments exact token supervision with two mechanisms: text prompting rectification and image style preservation. The first addresses the mismatch between a user-provided source prompt and the actual source image; the second addresses preservation of style and structure during inversion and editing (Wang et al., 23 Oct 2025).

For text prompting rectification, the method begins with the source prompt $F$1, adds an instruction prompt $F$2 such as “the language style of this prompt is”, and appends 20 learnable prompt tokens $F$3. The system encodes $F$4 and $F$5 through Infinity’s text encoder $F$6 and concatenates

$F$7

The backbone remains frozen, and only the learnable prompt tokens are optimized using $F$8. The paper describes this as a way to rectify the semantic condition so that it better matches the actual source image, thereby reducing semantic mismatch during inversion (Wang et al., 23 Oct 2025).

For image style preservation, EditInfinity adds LoRA to the FFN layers of Infinity. The role of LoRA is described as preserving global appearance and style after prompt rectification. The paper motivates this with the low-rank bias of LoRA under the update

$F$9

Only the low-rank matrices are updated, and the learned $$z_k = \text{down}\left(F - F_{k-1}, (h_k, w_k)\right) \in \mathbb{R}^{h_k \times w_k \times d}.$$0 is retained during editing. The ablation summary in the paper states that removing both prompt rectification and LoRA causes serious inconsistency; prompt rectification alone improves prompt alignment but can shift style; adding LoRA restores fidelity (Wang et al., 23 Oct 2025).

Taken together, these two mechanisms split the inversion problem into semantic alignment and appearance preservation. A plausible implication is that EditInfinity treats inversion not as a single reconstruction objective, but as coordinated control over prompt semantics and source-style retention.

4. Editing stage and holistic smoothing

The editing stage uses a token-level blending strategy that mixes source and target tokens in a spatially controlled, multi-scale manner. For each scale $$z_k = \text{down}\left(F - F_{k-1}, (h_k, w_k)\right) \in \mathbb{R}^{h_k \times w_k \times d}.$$1, the model generates target tokens $$z_k = \text{down}\left(F - F_{k-1}, (h_k, w_k)\right) \in \mathbb{R}^{h_k \times w_k \times d}.$$2 under the edited prompt, blends them with source tokens $$z_k = \text{down}\left(F - F_{k-1}, (h_k, w_k)\right) \in \mathbb{R}^{h_k \times w_k \times d}.$$3 using a smoothing kernel $$z_k = \text{down}\left(F - F_{k-1}, (h_k, w_k)\right) \in \mathbb{R}^{h_k \times w_k \times d}.$$4, and then feeds the blended tokens forward to the next scale. The paper emphasizes that smoothing is not applied only as a final compositing step; it is propagated autoregressively across scales (Wang et al., 23 Oct 2025).

The smoothing kernel is defined using Manhattan distance to the edit mask $$z_k = \text{down}\left(F - F_{k-1}, (h_k, w_k)\right) \in \mathbb{R}^{h_k \times w_k \times d}.$$5:

$$z_k = \text{down}\left(F - F_{k-1}, (h_k, w_k)\right) \in \mathbb{R}^{h_k \times w_k \times d}.$$6

with

$$z_k = \text{down}\left(F - F_{k-1}, (h_k, w_k)\right) \in \mathbb{R}^{h_k \times w_k \times d}.$$7

The main paper uses $$z_k = \text{down}\left(F - F_{k-1}, (h_k, w_k)\right) \in \mathbb{R}^{h_k \times w_k \times d}.$$8 and $$z_k = \text{down}\left(F - F_{k-1}, (h_k, w_k)\right) \in \mathbb{R}^{h_k \times w_k \times d}.$$9 (Wang et al., 23 Oct 2025).

At each scale, token blending is given by

$$R_k = \mathcal{Q}(z_k) = \frac{1}{\sqrt{d}} \operatorname{sign}\left(\frac{z_k}{\|z_k\|}\right),$$0

If $$R_k = \mathcal{Q}(z_k) = \frac{1}{\sqrt{d}} \operatorname{sign}\left(\frac{z_k}{\|z_k\|}\right),$$1, the edited result is downsampled and used as the autoregressive state for the next scale (Wang et al., 23 Oct 2025).

The paper attributes two functions to this holistic smoothing strategy. First, it improves fidelity by preserving unedited background and structure. Second, it improves semantic alignment by keeping the edited region guided by the target prompt. The ablations reported in the paper state that removing smoothing leads to visible boundary discontinuities, that the linear kernel outperforms a Gaussian alternative in the chosen setting, and that autoregressive blending at every scale is better than blending only at the end (Wang et al., 23 Oct 2025).

This design distinguishes EditInfinity from editing systems that rely on approximate latent-noise inversion followed by guided denoising. Here, editing is closer to controlled autoregressive token replacement under exact source-token supervision.

5. Evaluation on PIE-Bench

The main empirical evaluation is conducted on PIE-Bench, described as containing 700 test cases and 9 editing types, with each case including a source image, source prompt, target editing prompt, and editing mask. The paper highlights qualitative results for add, change, and delete, and additionally reports edit types such as change content, change pose, change color, change background, change material, and change style (Wang et al., 23 Oct 2025).

Compared baselines include P2P, MasaCtrl, P2P-Zero, NTI, PnP-Inv, NP, StableFlow, RF-Edit, and Gemini 2.0 as a closed-source reference. The reported evaluation criteria cover both preservation and alignment: PSNR, LPIPS, MSE, and SSIM for background preservation, and CLIP score on whole image, CLIP score on edited region, and IR (ImageReward) for text alignment (Wang et al., 23 Oct 2025).

The paper reports the following main scores for EditInfinity on PIE-Bench:

Metric	Score	Role
PSNR	27.95	Background preservation
LPIPS	33.08	Background preservation
MSE	24.27	Background preservation
SSIM	92.12	Background preservation
CLIP Whole	26.41	Text alignment
CLIP Edited	23.47	Edited-region alignment
IR	5.88	Overall reward-style evaluation

These results are presented as the best overall balance on PIE-Bench, with the paper specifically stating that EditInfinity outperforms diffusion-based baselines across the trade-off between preserving background or unchanged regions and aligning edits to text prompts. It also states that the method outperforms RF-Edit, identified as the strongest diffusion-transformer baseline, on both preservation and alignment, and also exceeds the reported metrics of Gemini 2.0. The paper further stresses that this occurs even though Infinity’s base generative model is not obviously stronger than FLUX on GenEval, which it uses to argue that the improvement arises from the editing framework itself rather than only from the underlying generator (Wang et al., 23 Oct 2025).

The reported runtime characteristics show a marked asymmetry between inversion and editing: inversion: 107.06 s and per-editing: 3.64 s. The paper presents this as advantageous when multiple edits are performed on the same source image, since inversion is paid once and subsequent edits are relatively fast (Wang et al., 23 Oct 2025).

6. Ablations, relation to diffusion editing, and limitations

The ablation studies emphasize three components: learnable prompt + LoRA, smoothing kernel, and multi-scale autoregressive blending. The paper states that both learnable prompt and LoRA are important; prompt rectification improves semantic alignment, while LoRA restores source style and structure. It also notes that training LoRA too long overfits and hurts editability (Wang et al., 23 Oct 2025).

For the smoothing mechanism, the paper reports that performance is not overly sensitive to whether masks come from user-provided masks or from cross-attention maps. More importantly, it reports that without $$R_k = \mathcal{Q}(z_k) = \frac{1}{\sqrt{d}} \operatorname{sign}\left(\frac{z_k}{\|z_k\|}\right),$$2, boundary artifacts appear, that linear smoothing outperforms Gaussian smoothing in the chosen setting, and that autoregressive blending at every scale is superior to a non-autoregressive strategy (Wang et al., 23 Oct 2025).

In relation to diffusion-based pipelines, EditInfinity is explicitly framed as replacing approximate latent-noise inversion + guided denoising with exact discrete reconstruction + controlled autoregressive token replacement. That comparison aligns it with a broader set of editing systems that seek stronger control over localization and preservation, but its technical route is distinct: it relies on exact intermediate token supervision rather than time-step inversion in diffusion space (Wang et al., 23 Oct 2025).

The paper also states a limitation. EditInfinity can be less ideal in extreme style-change cases, especially when no background preservation is needed and the image contains detailed structural patterns. In such cases, token blending may constrain full transformation and slightly degrade fidelity. This suggests that the same mechanism that preserves local continuity and background structure may also limit radical global transformation when preservation is not the objective (Wang et al., 23 Oct 2025).

Within the paper’s own framing, EditInfinity’s main contributions are fivefold: a new editing framework for binary-quantized generative models; exact-supervision inversion using quantized source tokens; text prompting rectification and LoRA-based style preservation; holistic smoothing for spatially controlled editing; and strong empirical performance on PIE-Bench across localized editing operations. Its broader significance lies in showing that image editing quality can be improved by redesigning the inversion substrate itself rather than only improving prompt guidance or diffusion-time control (Wang et al., 23 Oct 2025).