RemEdit Diffusion Framework

• The paper introduces a novel diffusion-based framework that leverages Riemannian manifold navigation and dual-SLERP blending to achieve high-fidelity and controllable image edits.
• It employs goal-aware prompt enrichment via a vision-language model and task-specific attention pruning to enhance semantic consistency and accelerate inference.
• Empirical benchmarks on datasets like CelebA-HQ and LSUN-Church demonstrate superior accuracy (S_dir up to 0.1982) and rapid runtimes, affirming its practical effectiveness.

RemEdit is a diffusion-based image editing framework designed to reconcile the trade-off between semantic fidelity and inference speed in controllable generative AI. It achieves this through a synergistic integration of Riemannian geometry for latent space navigation, dual spherical interpolation (SLERP), grounded prompt enrichment, and task-specific attention pruning. RemEdit demonstrates state-of-the-art editing accuracy and real-time performance benchmarks across various datasets, substantiating its practicality and robustness for high-fidelity image manipulation (Adhikarla et al., 25 Jan 2026).

1. Riemannian Manifold Navigation in Latent Space

RemEdit models the U-Net bottleneck feature space ("h-space") as a Riemannian manifold $(\mathcal M, g)$ of dimension $N = C \times H \times W$. The metric tensor $g_{ij}(h) = \langle \partial_i, \partial_j \rangle$ induces inner products on the tangent space $T_h\mathcal M$, enabling the computation of geodesics—paths following the data distribution instead of mere straight Euclidean offsets. The framework learns a non-trivial affine connection $\nabla$ parameterized by Christoffel symbols $\Gamma^k_{ij}(h)$, which underlie geodesic calculations in feature space.

A lightweight "Mamba" network $f_\theta$ takes the concatenated feature vector $y_0 = \mathrm{concat}(h,\;\mathrm{timestep})$ and predicts the Christoffel symbols. In parallel, a tangent-vector predictor $v_\phi$ outputs an initial velocity $v_0 = v_\phi(y_0)$, constrained to the unit ball via tanh-based retraction. Geodesic endpoints are obtained by integrating the following ODE system over $N = C \times H \times W$0: $N = C \times H \times W$1 yielding $N = C \times H \times W$2 and the geodesic edit $N = C \times H \times W$3. The model is trained to match example edits $N = C \times H \times W$4 using the objective

$N = C \times H \times W$5

This Riemannian approach preserves the semantics of edits by respecting the intrinsic structure of the latent distribution.

2. Dual-SLERP Blending for Edit and Identity Control

RemEdit employs a dual-SLERP approach, hierarchically interpolating between the original and edited states in both feature and noise spaces. Given two unit vectors $N = C \times H \times W$6 and $N = C \times H \times W$7 with angle $N = C \times H \times W$8, the SLERP is defined as

$N = C \times H \times W$9

The inner SLERP applies this to $g_{ij}(h) = \langle \partial_i, \partial_j \rangle$0 (original latent) and $g_{ij}(h) = \langle \partial_i, \partial_j \rangle$1 (geodesic endpoint), facilitating continuous modulation of edit strength: $g_{ij}(h) = \langle \partial_i, \partial_j \rangle$2 The outer SLERP operates in noise space after U-Net forward passes. It separates semantic change (identity-orthogonal component) from fidelity, fusing predictions as: $g_{ij}(h) = \langle \partial_i, \partial_j \rangle$3 This dual mechanism enables explicit, fine-grained control over semantic transformation and identity retention.

3. Goal-Aware Semantic Prompt Enrichment

Textual edit prompts are typically under-specified for targeted manipulations (e.g., "face → face with makeup"). RemEdit enriches prompts by extracting a fine-grained caption $g_{ij}(h) = \langle \partial_i, \partial_j \rangle$4 from the input image $g_{ij}(h) = \langle \partial_i, \partial_j \rangle$5 using a pretrained Vision-LLM (Qwen2-VL): $g_{ij}(h) = \langle \partial_i, \partial_j \rangle$6 Semantic edit direction $g_{ij}(h) = \langle \partial_i, \partial_j \rangle$7 is then computed in text-embedding space (e.g., via CLIP): $g_{ij}(h) = \langle \partial_i, \partial_j \rangle$8 This approach grounds edits in actual image content, enhancing consistency and specificity without the need for additional training—prompt enrichment is a single forward pass through the VLM.

4. Task-Specific Attention Pruning

RemEdit introduces a novel attention pruning mechanism to accelerate inference while maintaining semantic fidelity. Each self-attention block's feature map $g_{ij}(h) = \langle \partial_i, \partial_j \rangle$9 is reshaped into tokens $T_h\mathcal M$0, with $T_h\mathcal M$1. An MLP pruner $T_h\mathcal M$2 processes $T_h\mathcal M$3 to produce soft importance scores $T_h\mathcal M$4. During inference, a pruning rate $T_h\mathcal M$5 determines the top $T_h\mathcal M$6 tokens to retain; attention is computed only over these, yielding: $T_h\mathcal M$7 The pruner is trained to optimize

$T_h\mathcal M$8

enabling effective acceleration without degrading semantic edit quality.

5. Empirical Performance and Benchmarks

RemEdit demonstrates state-of-the-art results on several benchmark datasets. Quantitative metrics on CelebA-HQ ($T_h\mathcal M$9) include directionality score ($\nabla$0), segmentation consistency, Fréchet Inception Distance (FID), and runtime:

Method	$\nabla$1	Seg. Consistency $\nabla$2	FID$\nabla$3	Time (s) $\nabla$4
Asyrp (h-space)	0.1900	87.9%	24.3	28.9
LEdits++	0.1820	89.7%	22.5	20.1
RemEdit (full)	0.1982	92.4%	19.8	2.8

($\nabla$5 FID evaluated on 250 edited samples vs. held-out real images.)

On LSUN-Church, RemEdit similarly outperforms Diffusion-CLIP and BoundaryDiffusion in both $\nabla$6 and segmentation consistency. Ablation experiments (CelebA-HQ, "smiling" edit) further show that geodesic navigation and dual-SLERP deliver superior semantic accuracy, and that up to 50% attention pruning ($\nabla$7) preserves most fidelity while reducing runtime to approximately 2.31 s.

6. End-to-End Algorithmic Structure

RemEdit’s computational workflow integrates the above components into a coherent editing pipeline:

$\Gamma^k_{ij}(h)$0

This architecture demonstrates the synthesis of geometric, linguistic, and computational efficiency innovations, each contributing distinctly to controllable, high-speed image editing.

7. Significance and Implications

RemEdit's unified approach—learned Riemannian manifold navigation, dual-stage SLERP blending, prompt enrichment via VLM, and semantic attention pruning—addresses core limitations of prior diffusion editors. It sets new standards in edit fidelity ($\nabla$8 up to 0.198 on CelebA-HQ) and runtime ($\nabla$9 s at 50% pruning), evidencing not only superior semantic accuracy but also practical deployment feasibility. A plausible implication is that future generative editing frameworks may increasingly rely on data-driven geometric structures and task-aware resource allocation to reconcile interpretability, controllability, and efficiency (Adhikarla et al., 25 Jan 2026).