WarpI2I: Image Warping for Image-to-Image Translation

Abstract: Image-to-image (I2I) translation has achieved strong results in tasks like human relighting and driving scene translation using latent diffusion models (LDMs). However, compact LDMs often struggle to preserve fine-grained structures because the encoder compresses high-resolution inputs into a spatially downsampled latent space. To address this issue, we propose a simple saliency-guided warp-unwarp framework that reallocates spatial representation toward salient regions before encoding, enabling better preservation of structural details without increasing latent resolution. The warped image is processed by the original diffusion model and then mapped back via an inverse warp. In addition, we propose a simple and efficient outpainting-based synthetic data generation pipeline to produce high-quality paired data for image relighting. Our method is model-agnostic, requires no architectural modification, and introduces negligible computational overhead. Experiments on human relighting, driving scene relighting, and translation demonstrate improved structural preservation, lighting faithfulness, and image quality, with our framework extending naturally to video via frame-by-frame application with good temporal stability. Project Webpage: https://shenzheng2000.github.io/WarpI2I.github.io

• The paper proposes a plug-and-play, saliency-guided warping method that enlarges salient regions pre-encoding to preserve fine details in latent diffusion models.
• The method employs differentiable, invertible warping and unwarping steps, ensuring high-fidelity structure preservation without altering the original network architecture.
• Extensive experiments demonstrate significant improvements in facial detail, semantic consistency, and robustness across various I2I tasks with negligible computational overhead.

Saliency-Guided Image Warping for Latent I2I Diffusion: An Expert Summary of "WarpI2I"

Motivation and Problem Formulation

Recent advances in image-to-image (I2I) translation, notably those leveraging latent diffusion models (LDMs), have demonstrated strong performance for tasks such as image relighting and scene translation. However, practical deployment—especially with open-source, compact LDMs—remains challenging for scenarios requiring preservation of fine-grained detail, such as human faces, textual objects, or salient scene artifacts. The performance degradation is attributed primarily to the aggressive spatial downsampling characteristic of LDM encoders, which results in a latent space too coarse to maintain localized, high-frequency information.

A naïve solution—simply raising the input resolution or the latent's spatial size—incurs quadratic increases in computational cost and memory, rendering it inapplicable for real-time or large-scale settings. Prior approaches based on multi-scale pipelines or test-time refinement introduce non-trivial overheads and typically require architectural changes.

This work proposes a model-agnostic, computationally inexpensive, and plug-and-play framework: saliency-guided spatial image warping before encoding, followed by inverse warping (unwarping) post-generation (Figure 1). The essential insight is to densify the spatial representation allocated to salient regions (e.g., faces, small objects) before latent encoding, while maintaining the global structure by inverting the warp after translation. This mechanism preserves critical details without altering the LDM’s architecture or increasing the size of the spatial latent.

Figure 1: Overview of the warp-unwarp strategy. Salient image regions are enlarged pre-encoding and restored post-translation, boosting detail preservation in standard latent diffusion frameworks.

Methodology

Saliency Map Construction and Application

The core of the method leverages semantically-guided saliency maps to identify regions warranting greater spatial allocation in the latent. For human relighting, this equates to fine-grained, part-level annotation (e.g., face and eye bounding boxes), while for driving scenes, saliency regions correspond to object-level spatial extents. The saliency signal is sourced either from ground-truth annotations or detector-generated bounding boxes (e.g., YOLO-World, InsightFace).

The warping process is then realized as a differentiable, invertible mapping—parameterized as a piecewise bilinear transformation—applied before encoding. Upon completion of I2I translation, the result undergoes inverse warping, thus restoring the original image geometry (Figure 2).

Figure 2: The original, warped, and reconstructed images show negligible pixel-wise discrepancy after warp-unwarp, ensuring high-fidelity structure preservation.

This approach introduces a minimal computational penalty (3 ms per image for both warping and unwarping) and requires no additional learnable parameters.

Synthetic Paired Data Generation

Paired data is a bottleneck for supervised I2I training, especially for relighting. The authors introduce an efficient synthetic data pipeline based on the FLUX text-to-image model. The pipeline employs outpainting, depth conditioning, and prompt engineering, with ChatGPT used both for initial annotation and automated, quantitative filtering of poor samples (Figure 3). Salient features—such as identity and pose—are strictly controlled throughout this process, with a high (≈96%) usable pair yield.

Figure 3: The synthetic data pipeline combines FLUX outpainting, background prompt substitution, depth-based conditioning, and ChatGPT filtering to yield robust paired training data.

Experimental Evaluation

Human Relighting

Extensive experiments on VITON-HD and StreetTryOn reveal that, compared to state-of-the-art diffusion-based relighting systems (IC-Light, DreamLight), the proposed framework yields improvements in person identity, clothing identity, image quality, and lighting faithfulness. Notably, both quantitative metrics (FID, LPIPS, ArcFace, CLIP score) and user study results indicate noticeable advantages for WarpI2I, with marked improvements in fine-scale detail retention (Figure 4, Figure 5).

Figure 4: Warping leads to significant improvement in facial detail fidelity, outperforming baseline img2img-turbo.

Figure 5: Detailed comparison demonstrates the superior preservation of facial features with warping.

Ablation studies show the contribution of both warping and the Background Prompt Substitution (BPS) step, with the combination resulting in the highest mean user ratings across all criteria. The system also exhibits robustness to detector noise and bandwidth hyperparameter variation.

Driving Scene Relighting and Translation

On both paired (ROADWork) and unpaired (CycleGAN-Turbo on BDD100K, Cityscapes, ACDC, DarkZurich) settings, the warp-unwarp approach demonstrates improved semantic consistency and structural fidelity, especially for small or distant objects (Figure 6, Figure 7). Quantitative gains are observed in FID, KID, Clean-FID, and DINO-Struct across standard cross-domain and domain-shifted settings.

Figure 6: In driving scene relighting, the method better preserves small semantic details and global illumination effects compared to prior art.

Figure 7: Warping yields qualitatively superior background and object consistency in complex scene translation tasks.

Inference latency remains unaffected for all practical purposes, preserving the real-time applicability of baseline models.

Comparison With Commercial and Non-Diffusion Baselines

When compared with commercial APIs such as ChatGPT img2img, WarpI2I demonstrates both order-of-magnitude speed-ups and significantly better identity preservation, especially in challenging relighting conditions. Commercial models are also brittle to spatially warped input, whereas the presented framework preserves scale and structure (Figure 8).

Figure 8: ChatGPT img2img alters face and clothing identity and fails on warped inputs; WarpI2I robustly maintains both.

Implications, Limitations, and Future Directions

The saliency-based warping framework addresses a crucial bottleneck of modern latent I2I models—retention of fine-grained details—without the complex overhead of architectural expansion, multi-scale recomposition, or intensive refinement. Given its model-agnosticism and negligible additional cost, WarpI2I is well suited for broad integration in both image and video translation pipelines.

A direct implication is the enabling of higher-quality, open-source diffusion-based I2I applications in computationally constrained or real-time contexts, ranging from photo editing to autonomous perception. The robust synthetic data generation pipeline further lowers the entry barrier for new domains with limited paired data, potentially benefiting tasks such as scene rendering and robotic manipulation.

Limitations include evaluation restricted to efficient, single-step diffusion backbones (due to compute constraints), and some frame-level temporal instability when applying the algorithm naively to video. Addressing high-speed motion with temporal priors (e.g., flow-based warping) and systematic integration into multi-step diffusion and transformer architectures are promising directions for future research.

Conclusion

WarpI2I introduces a principled, effective saliency-guided image warping method for detail-preserving latent diffusion-based image translation. Integrated with a scalable synthetic data generation pipeline, it achieves robust gains over advanced baselines in both paired and unpaired I2I tasks, with negligible computational overhead and strong adaptability. The approach provides a compelling solution for the detail preservation problem in latent generative models, with immediate opportunities for both research and deployment in vision systems.

(2606.31018)