UnReflectAnything Framework

• UnReflectAnything Framework is a unified approach for removing reflections and highlights from single images using diffusion priors, transformer-based models, and minimal user guidance.
• It leverages innovative data synthesis, multi-stage training, and reflection-invariant losses to robustly separate entangled transmission and artifact components without dense supervision.
• Interactive components, such as contrastive mask guidance, enable precise, state-of-the-art removal performance in both natural scenes and controlled environments.

The UnReflectAnything framework encompasses a class of high-performance methodologies for single-image reflection and highlight removal, unified by their ability to generalize across diverse scenes and handle entangled image artifacts without reliance on specialized hardware or dense ground truth data. These frameworks leverage advanced data generation strategies, diffusion or transformer-based learned priors, and, in some variants, flexible user guidance to address dereflection and highlight suppression in both natural and controlled domains (Hu et al., 21 Mar 2025, Rota et al., 10 Dec 2025, Chen et al., 2024).

1. Problem Definition and Motivation

Single-image reflection and highlight removal address two fundamental degradations:

• Reflection removal: Decomposition of a blended image $M = T + R$ into its latent transmission $T$ (the desired scene) and reflection $R$ (spurious component).
• Specular highlight removal: Elimination of view-dependent, often saturated glints that violate the Lambertian model, obscuring both texture and geometry.

The challenges stem from strong entanglement of transmission and reflection/highlight terms, broad variability of glass/lighting, and the absence of ground-truth supervision at scale. Reflections and specularities impair a wide range of downstream tasks, including geometric correspondence (stereo, optical flow) and medical imaging, necessitating robust, generalizable solutions.

2. Core Components and Datasets

2.1 Diverse Reflection Removal (DRR) Dataset

The DRR dataset underpins UnReflectAnything’s generalization. It comprises 257 real-world scenes captured at 4K resolution, systematically varying glass angle ($\theta \sim U(0^\circ,180^\circ)$) and thickness ($d \in \{3\text{ mm},8\text{ mm}\}$), controlling reflection properties via Fresnel reflectance. For each configuration, aligned image pairs $(M_i,T)$ are computed using SIFT+RANSAC. Synthetic pairs employ a physically-motivated compositing formula:

$$M = \gamma_1 T + \gamma_2 R - \gamma_1 \gamma_2 (T \odot R)$$

with $\gamma_1 \sim U[0.8,1]$, $\gamma_2 \sim U[0.4,1]$, and filtering based on CLIP-derived realism scores. The dataset offers over 23,000 training pairs and 400 test pairs (DRR-S: standard; DRR-C: challenging) (Hu et al., 21 Mar 2025).

2.2 Virtual Highlight Synthesis for Supervision

For highlight removal (RGB-only), synthetic specularities are rendered over arbitrary RGB images, using monocular geometry and Fresnel-aware Blinn–Phong shading:

• Depth and normals are recovered via monocular estimators.
• Specular intensity is parameterized by incident/view directions and material parameters.
• The composited output provides explicit ground-truth highlight maps for training (Rota et al., 10 Dec 2025).

3. Model Architectures

3.1 One-Step Diffusion Prior for Dereflection

The diffusion-prior variant comprises:

• U-Net $\epsilon_\theta(\cdot,t)$: Trains as a diffusion denoiser in latent space. Forward diffusion follows a Gaussian Markov chain with decreasing variance ($\alpha_t$ schedule).
• ControlNet $f_\phi$: Encodes mixed input $M$ via $E(M)$. Structural cues are fused via cross-latent decoder $\mathcal{D}$ incorporating zero-conv skip-connections to recover high-frequency details lost during fast denoising.
• One-step denoising: Implements closed-form inversion at $t=0$, bypassing iterative refinement:

$$\hat{x}_0 = \frac{1}{\sqrt{\bar{\alpha}_t}}\left(x_t - \sqrt{1-\bar{\alpha}_t}\,\epsilon_\theta(x_t, t)\right)$$

• Achieves deterministic, sub-second inference for $768{\times}768$ images on a single GPU (Hu et al., 21 Mar 2025).

3.2 RGB-only Highlight Removal with Vision Transformers

The specular highlight variant comprises:

• Frozen ViT encoder (DINOv3-Large): Extracts multi-scale patch features.
• Highlight predictor $H$: DPT-style lightweight decoder outputs soft highlight mask $M_\text{highlight}$, downsampled and thresholded for patchwise masking.
• Token-level inpainting $T$: Restores features in “corrupted” patches using a learnable [MASK], local mean priors, and 2D positional encodings; several transformer layers inpaint at token level, yielding $F_\text{comp}$.
• RGB decoder $D$: Produces diffuse reconstruction from inpainted features.
• No paired clean data required; supervision is via the virtual highlight synthesis pipeline (Rota et al., 10 Dec 2025).

3.3 Interactive Reflection Removal with Contrastive Mask Guidance

FIRM adapts UnReflectAnything to interactive scenarios:

• User Guidance Conversion (UGC): Converts points, boxes, strokes, or text input into unified binary “contrastive” masks using foundation models (e.g., SAM).
• Contrastive Mask–Guided Reflection Removal Network (CMGR-Net): U-Net backbone with repeated Contrastive Guidance Interaction Blocks (CGIB), using channel-wise cross-attention to suppress reflection channels within the transmission features.
• Minimal guidance (e.g., 3–4 clicks) suffices for strong results (Chen et al., 2024).

4. Training Strategies and Losses

4.1 Progressive Multi-Stage Training

The diffusion-prior approach employs four training stages:

1. Foundation: Synthetic reflections, optimizing $L_\text{diff}$ over mixed data.
2. Fine-Tuning: Hard DRR pairs, especially grazing angles with high Fresnel reflectance.
3. Reflection-Invariant Tuning: Enforces consistent transmission recovery across DRR pairs sharing $T$ but differing in $R$. The contrastive loss:

$$L_\text{con} = \mathbb{E}_{M_1,M_2}[\|\mu_{\theta,\phi}(M_1) - \mu_{\theta,\phi}(M_2)\|^2]$$

encourages invariance with respect to reflections.

1. Decoder Training: Freezes earlier components; $L_\text{rec}$ sums $L_1$, SSIM, and LPIPS on real pairs for sharper textures (Hu et al., 21 Mar 2025).

4.2 Loss Formulations for Highlight Removal

Supervision aggregates:

• Dice, $L_1$, and TV losses on highlight maps
• Masked token-level inpainting and cosine similarity
• Autoencoder, seam, specularity, and standard RGB reconstruction losses

Supervision is strictly on synthetic/factually annotated regions to avoid ambiguity, as excluding dataset highlights in loss computation is shown critical (MSE doubles if not masked) (Rota et al., 10 Dec 2025).

4.3 Losses for Mask-Guided Interactive Removal

Multi-term objectives combine $L_\text{rec}$ (pixel-wise), $L_\text{grad}$ (edge), $L_\text{perc}$ (perceptual/VGG), $L_\text{excl}$ (decorrelation between $T$ and $R$), and $L_\text{res}$ (residual rectification) (Chen et al., 2024).

5. Inference Algorithms and Computational Characteristics

The diffusion-prior model’s inference proceeds via deterministic one-step denoising given a corrupted latent and encoded structural features. For highlight removal, inference consists of forward pass through a frozen ViT, highlight localization, inpainting at the token level, then RGB decoding. FIRM’s inference adds only a negligible interaction cost due to prompt-to-mask conversion.

Inference is fast (≈1 s for $768{\times}768$), deterministic, and scalable. This determinism is enabled by condensing the diffusion process into a single denoising step at $t=0$ with closed-form inversion (Hu et al., 21 Mar 2025).

6. Experimental Evaluation and Results

6.1 Quantitative Metrics

UnReflectAnything achieves state-of-the-art (SOTA) results on multiple reflection/highlight datasets:

Benchmark	Nature(20)	Real(20)	SIR²(500)	DRR-S(200)	DRR-C(200)
PSNR / SSIM (diffusion-prior)	26.81 / 0.843	25.21 / 0.841	27.19 / 0.930	27.25 / 0.902	23.77 / 0.843

Across highlight and downstream geometric benchmarks, UnReflectAnything consistently achieves lowest masked MSE (MSEₘ), top SSIM, and improvements in geometric inlier ratio and reduced epipolar error (Rota et al., 10 Dec 2025).

6.2 Qualitative and Ablation Results

Key outcomes include sharper removal of ghosting edges, robustness across diverse materials (water, plastics, displays), and strong geometry preservation in both natural and surgical imagery. Ablations demonstrate that token inpainting (in highlight removal) and reflection-invariant loss (in dereflection) are essential for top performance.

Failure cases emerge mainly when transmission and reflection (or background and specularity) intensities are nearly identical; this can cause over- or under-removal.

7. Open Challenges, Limitations, and Future Directions

While UnReflectAnything generalizes well due to data diversity and principled priors, several open challenges remain:

• Ambiguity Resolution: When reflection/transmission (or highlight/diffuse) intensities closely match, the model may fail. Potential remedies include user guidance, semantic context, or multi-view data.
• User Guidance and Segmentation: Interactive methods rely on foundation segmentation models for mask creation; segmentation quality fundamentally limits final performance (Chen et al., 2024).
• Extension to Videos: Current methods focus on single images; extending to time-consistent video dereflection or highlight removal is an open direction.
• Annotation and Supervision: Scaling to more modalities of human input (strokes, referring expressions), and joint training with segmentation models, have been proposed for future work.

UnReflectAnything collectively represents a set of frameworks characterized by large, diverse data, diffusion or attention-based priors, and, where applicable, flexible interaction, delivering robust, generalizable, and efficient solutions to reflection and highlight removal (Hu et al., 21 Mar 2025, Rota et al., 10 Dec 2025, Chen et al., 2024).