ReflectRM: Self-Supervised Reflection Removal

• ReflectRM is a self-supervised framework that uses dual-branch denoising diffusion models to decompose a single image into transmission and reflection layers without heavy paired data reliance.
• It integrates a nonlinear attention-based synthesis network with cycle consistency to fuse the decomposed layers, enhancing texture preservation and reducing artifacts.
• Optimized with adversarial regularization and diffusion priors, ReflectRM outperforms existing methods on benchmarks like SIR², FRR, and MRR in metrics such as PSNR, SSIM, LPIPS, and RAM.

ReflectRM is a self-supervised framework for single-image reflection removal that combines a dual-branch denoising diffusion probabilistic model for layer decomposition with a learned nonlinear attention-based synthesis network under cycle consistency. It is designed to separate a single observed image containing glass reflections into a transmission layer and a reflection layer without requiring large amounts of paired real-world training data, while still being able to exploit limited paired samples for pretraining before self-supervised adaptation (Lu et al., 2024).

1. Problem setting and image formation

ReflectRM addresses the problem of recovering a transmission image $T$ and a reflection image $R$ from a single observed camera image $x$ acquired through a transparent surface. In the formulation used by the method, the observation domain is denoted by $X$, the transmission-image domain by $S$, and the reflection-image domain by $R$. The decomposition mapping is defined as $G: X \to S \times R$, and the synthesis mapping is defined as $F: S \times R \to X$, with the cycle constraint $x \approx F(G(x))$ for all $x \in X$ (Lu et al., 2024).

A central design choice is that ReflectRM does not commit to a fixed linear superposition model such as $R$0. Instead, it learns the forward composition $R$1 through an attention-based nonlinear synthesis network. The stated rationale is twofold: first, real reflections often deviate from simple additivity because of ghosting, blur, attenuation, and nonlinearity; second, the learned synthesis process provides a cycle-consistent training signal when ground-truth pairs are scarce (Lu et al., 2024).

This formulation places ReflectRM within the class of single-image inverse problems that treat image formation as partially learnable rather than fully prescribed. A plausible implication is that the method is especially targeted at settings where physical simplifications fail, such as museum display glass, layered reflections, and nonuniform attenuation.

2. Decomposition and synthesis modules

The framework is organized around three components: a Reflective Removal Network, a Reflective Synthesis Network, and a Transmission Discriminator. The Reflective Removal Network (RRN) is the decomposition module. It takes a single input image $R$2 and predicts a transmission image $R$3 and a reflection image $R$4 by modeling two coupled reverse diffusion processes. The Reflective Synthesis Network (RSN) recombines the separated components to reconstruct the input image through an adaptive nonlinear fusion mechanism. The Transmission Discriminator (TD) acts as an adversarial regularizer on recovered transmission images, encouraging realism and mitigating collapse during self-supervised training (Lu et al., 2024).

The RSN is the component that closes the cycle. Given estimated layers, it synthesizes $R$5 so that

$R$6

This reconstruction constraint is used for both paired and unpaired data (Lu et al., 2024).

Its fusion mechanism is attention-based. RSN computes $R$7 spatial attention maps from concatenated transmission and reflection features, modulates each layer, and synthesizes the final image with a small CNN:

$R$8

$R$9

$x$0

Here $x$1 and $x$2 are small CNNs, and $x$3 denotes element-wise multiplication (Lu et al., 2024).

This decomposition–synthesis pairing is the structural core of ReflectRM. The RRN supplies candidate latent layers; the RSN evaluates whether those layers can plausibly regenerate the original observation. This suggests that the framework treats decomposition quality not only as a denoising problem but also as a consistency problem over a learned forward model.

3. Diffusion formulation and optimization

The decomposition model is built from dual denoising diffusion probabilistic model branches, one for transmission and one for reflection. The forward noising process follows a Markov diffusion with Gaussian noise:

$x$4

ReflectRM instantiates separate chains

$x$5

$x$6

with

$x$7

where $x$8 and $x$9 (Lu et al., 2024).

Both reverse processes share a denoiser $X$0 with a U-Net backbone. The denoising losses are defined independently for transmission and reflection and then summed. When paired targets are available, ReflectRM also uses a layer reconstruction loss

$X$1

The adversarial component is applied through TD, and the unpaired regime updates RRN and RSN jointly to minimize the cycle-consistency loss plus adversarial regularization (Lu et al., 2024).

The full training design is therefore hybrid rather than purely generative. Diffusion provides the prior over plausible decompositions, adversarial regularization constrains the transmission layer, and the RSN-derived cycle serves as the self-supervised signal. This suggests that ReflectRM is best understood as a coupled generative–reconstructive system rather than as a standalone DDPM.

4. Architecture and training paradigm

The RRN denoiser $X$2 uses a modified U-Net with 8 downsampling and 8 upsampling layers, symmetric skip connections, and attention blocks at $X$3 and $X$4 resolutions. The diffusion process uses $X$5 timesteps with a linear $X$6 schedule from $X$7 to $X$8. The RSN uses 4 downsampling and 4 upsampling stages, dual-stream fusion of $X$9 and $S$0, and a single-layer $S$1 convolution attention module with softmax normalization channel-wise. The Transmission Discriminator is a standard CNN discriminator operating on transmission outputs (Lu et al., 2024).

Training is implemented in PyTorch with Adam, learning rate $S$2, and batch size 8. The reported loss weights are $S$3, $S$4, and $S$5. Training proceeds in two stages. Stage 1 uses 721 paired samples from the MRR dataset for paired pretraining, alternating updates of RRN and RSN while TD is trained adversarially with RRN. Stage 2 uses 1,621 unpaired samples for self-supervised fine-tuning, updating RRN and RSN jointly under cycle-consistency and adversarial regularization. End-to-end training on NVIDIA RTX 4090 GPUs takes approximately 48 hours (Lu et al., 2024).

The paper’s practical guidance recommends $S$6 diffusion steps, linear $S$7 from $S$8 to $S$9, batch size 8, and $R$0 input resolution for a quality–speed balance. At inference time, only a single forward pass through RRN is used; RSN and TD are not used at test time, and no iterative optimization is required. Simple normalization of the transmission output is reported as sufficient post-processing (Lu et al., 2024).

5. Datasets, metrics, and empirical results

Evaluation is conducted on three datasets: SIR$R$1, consisting of 500 real-world pairs through glass; FRR, with 157 real-world flash/no-flash scenes and 1964 synthetic scenes; and the Museum Reflection Removal (MRR) dataset, which contains 1,621 unpaired and 721 paired images collected from museums and exhibitions across geological, botanical, animal, anthropological, and art/historic/technological/archaeological categories (Lu et al., 2024).

Performance is measured with PSNR, SSIM, LPIPS, and a new Reflection Artifact Measure (RAM):

$R$2

RAM is reported to correlate better with human judgments on reflection artifacts, with Pearson 0.85, than PSNR at 0.76 or SSIM at 0.79 (Lu et al., 2024).

Dataset	Metrics
SIR$R$3	PSNR 28.41, SSIM 0.912, LPIPS 0.087, RAM 0.0549
FRR	PSNR 24.76, SSIM 0.841, LPIPS 0.142, RAM 0.0437
MRR	PSNR 24.15, SSIM 0.835, LPIPS 0.148, RAM 0.0498

Across SIR$R$4, FRR, and MRR, ReflectRM is reported to outperform CEILNet, PercepNet, BDN, ERRNet, DSRN, and SDN, with gains of 0.50–3.84 dB in PSNR, 0.005–0.074 in SSIM, 0.003–0.057 in LPIPS, and 0.0050–0.0561 in RAM. Scenario-wise, the method shows large improvements on complex museum artifacts, with $R$5 dB PSNR over ERRNet, and on textured surfaces, with $R$6 dB over SDN. Gains are modest, at $R$7 dB over DSRN, in cases of extremely strong reflections or motion blur (Lu et al., 2024).

The ablation results on SIR$R$8 attribute measurable roles to each component. Removing TD degrades PSNR by 1.26 dB and increases RAM by 0.0334. Removing attention drops PSNR by 2.09 dB and increases RAM by 0.0676. Removing RSN and cycle consistency produces the largest degradation, at $R$9 dB PSNR and $G: X \to S \times R$0 RAM. The paper interprets these results as evidence that diffusion, attention, and cycle consistency are all essential contributors (Lu et al., 2024).

6. Complexity, robustness, and relation to prior work

ReflectRM has 89.7M parameters, compared with 85.3M for SDN and 82.1M for DSRN. The added capacity is attributed to dual-branch diffusion and RSN attention. Inference complexity is stated as $G: X \to S \times R$1 per image. On an RTX 4090, reported runtimes are 0.061 s for $G: X \to S \times R$2, 0.235 s for $G: X \to S \times R$3, and 0.892 s for $G: X \to S \times R$4 images (Lu et al., 2024).

The framework’s stated strengths are its ability to handle multilayer ghosting, blur, and nonlinearity; preserve textures and structures through attention-guided fusion; and generalize from limited paired data through cycle-consistent self-supervision. Typical failure cases include subtle low-contrast reflections that mimic background texture, overlapped reflections at varying intensities, and severe motion blur. These can induce over-removal or misclassification between transmission and reflection. In cultural heritage digitization, reflections near frames or glass edges may be ambiguous (Lu et al., 2024).

Relative to earlier approaches, ReflectRM is positioned against three families of methods. Unlike GAN-based layer separation methods such as CRRN, dual-attention GANs, and cycle GAN variants, it uses DDPMs for modeling the multimodal distributions of $G: X \to S \times R$5 and $G: X \to S \times R$6, with the stated advantage of stability and data efficiency. Compared with supervised CNN-based methods trained on synthetic pairs, such as CEILNet, ERRNet, and BDN, it removes the need for strict adherence to a fixed linear image formation model because synthesis is learned through $G: X \to S \times R$7 and attention maps. Compared with multi-image methods based on polarization, flash/no-flash, or multi-focus capture, it operates on a single image while remaining competitive on FRR and real-world SIR$G: X \to S \times R$8 (Lu et al., 2024).

A plausible source of terminological confusion is that the string “ReflectRM” was later used in an unrelated RLHF context for a generative reward model, but the computer-vision usage discussed here refers specifically to a self-supervised reflection-removal framework built from dual-branch diffusion, nonlinear synthesis, and cycle consistency (Qin et al., 8 Apr 2026).