DeflareMamba: SSM-Based Lens Flare Removal

• DeflareMamba is a U-shaped vision architecture that uses state-space models to effectively remove lens flares while maintaining local and global context.
• It employs dual-branch processing with local-enhanced scanning and hierarchical selective sampling to capture fine details and long-range dependencies in linear time.
• Quantitative evaluations demonstrate improved PSNR, SSIM, and downstream task performance compared to traditional CNN and transformer-based methods.

DeflareMamba is a state-space–model-based, U-shaped vision architecture for lens flare removal that is designed to preserve contextual consistency: the semantic coherence between local regions around light sources and the global scene, including the long-range relationships between reflective flares and their originating sources. It targets scattering flares, described as diffuse streaks produced by microscopic lens imperfections, and reflective flares, described as secondary bright shapes due to internal lens reflections such as rings or polygons. The method introduces Mamba/SSM-based sequence modeling to flare removal and couples hierarchical selective sampling with local-enhanced scanning so that local-global dependencies are modeled with linear-time complexity in sequence length (Huang et al., 4 Aug 2025).

1. Problem setting and restoration objective

Lens flare removal is presented as an information confusion problem between the underlying image background and optical flares, arising from complex optical interactions between light sources and the camera lens. In this formulation, the principal difficulty is not only separating flare corruption from image content, but doing so without producing incomplete removal, color or texture artifacts, or semantic inconsistency. DeflareMamba is explicitly framed around the need to maintain contextual consistency, particularly in scenes where reflective artifacts may be spatially far from their originating sources and where scattering streaks locally corrupt texture near bright illumination (Huang et al., 4 Aug 2025).

The architecture is motivated by limitations attributed to existing model classes. CNN-based methods are described as struggling to model long-range dependencies because of local receptive fields. Transformer-based methods capture global dependencies but incur quadratic complexity, which constrains high-resolution processing or motivates windowed attention that degrades global context. Prior SSM-based image restoration, including MambaIR, is described as promising but insufficient for flare removal for two stated reasons: conventional selective scan emphasizes broad patterns at the expense of local pixel relationships, and the long-term decay plus causal structure of standard SSMs weakens correlations with sequence distance and hinders bidirectional context.

Within this problem setting, DeflareMamba advances three specific claims. First, it is presented as the first work to introduce state space models to the flare removal task. Second, it establishes long-range pixel correlations through varied stride sampling patterns that convert distant pixels in the original two-dimensional grid into nearby tokens in one-dimensional SSM space. Third, it introduces a Local-enhanced Selective Scan, abbreviated in the paper as L-SSM, to preserve neighborhood relationships through windowed, multi-directional scanning.

2. Network architecture and dataflow

DeflareMamba adopts a U-shaped architecture, with the U-shaped design serving two roles: enlarging effective receptive fields via downsampling and reducing sequence length to mitigate SSM long-term decay. An input image $I \in \mathbb{R}^{H \times W \times 3}$ is first projected into $C$ channels and then processed by an encoding stage and a decoding stage (Huang et al., 4 Aug 2025).

The encoding stage is built from Local-enhanced Residual State Space Groups (L-RSSG). Each L-RSSG contains $l$ Local-enhanced Residual State Space Blocks (L-RSSB), followed by a convolution and residual summation for stabilization of deep propagation. These encoder-side blocks emphasize fine-grained local detail. The decoding stage mirrors this structure using Hierarchical Residual State Space Groups (H-RSSG), but the last block in each such group is a Hierarchical RSSB (H-RSSB), which applies the hierarchical selective scan to capture multi-scale, long-range dependencies through varied stride sampling.

The network follows the Flare7K++ pipeline and outputs $O \in \mathbb{R}^{H \times W \times 6}$. The first three channels, $O_{1:3}$, constitute the flare-free image, and the latter three, $O_{4:6}$, constitute the predicted flare component. This six-channel design provides auxiliary supervision for both the clean image and the flare signal, and it supports a reconstruction consistency term obtained by recombining the two predictions to match the input.

At the block level, each L/H-RSSB follows a dual-branch organization described as reminiscent of the Transformer pattern Norm $\rightarrow$ Attention $\rightarrow$ Norm $\rightarrow$ MLP, with attention replaced by a 2D selective SSM. Given input $X \in \mathbb{R}^{H \times W \times C}$, the dataflow is

$C$0

$C$1

$C$2

$C$3

Here, $C$4 is a depth-wise convolution for local feature extraction, $C$5 is SiLU, and $C$6 is either the Local-Enhanced SSM in the encoder or the Hierarchical SSM in the decoder. The main branch therefore mixes local convolutional processing with SSM-based sequence modeling, while the skip branch stabilizes propagation through a projected nonlinear path. Groups terminate in a convolution and residual addition to the group input.

This organization supports a specific division of labor. The local stream, realized by $C$7 and L-SSM, preserves local detail. The global stream, realized by hierarchical scan in H-SSM, establishes long-range correlations. Their fusion is intended to maintain contextual consistency between corrupted light-adjacent regions and the broader scene.

3. State-space formulation and selective scanning mechanisms

DeflareMamba is grounded in continuous-time linear state space models and their discrete-time forms. The continuous formulation is

$C$8

where $C$9 is the hidden state, $l$0 is the input, $l$1 is the output, and $l$2 are learnable. Under zero-order hold discretization with step size $l$3, the update becomes

$l$4

with

$l$5

This admits a convolutional form,

$l$6

which underlies the claim of $l$7 sequence complexity, in contrast to the $l$8 complexity associated with pairwise attention interactions (Huang et al., 4 Aug 2025).

A central technical issue is long-term decay. The contribution of token $l$9 to token $O \in \mathbb{R}^{H \times W \times 6}$0 for $O \in \mathbb{R}^{H \times W \times 6}$1 is written as

$O \in \mathbb{R}^{H \times W \times 6}$2

with

$O \in \mathbb{R}^{H \times W \times 6}$3

Because learned $O \in \mathbb{R}^{H \times W \times 6}$4 is typically negative, the exponential term decays rapidly as sequence distance increases, which weakens long-range dependencies. Causality also prevents later tokens from informing earlier ones. DeflareMamba addresses these difficulties through two scanning mechanisms and through the U-shaped shortening of sequences.

The Local-enhanced SS2D mechanism partitions the feature map into non-overlapping windows of size $O \in \mathbb{R}^{H \times W \times 6}$5, flattens each window in raster order, and concatenates the resulting subsequences while respecting window layout. To preserve two-dimensional geometry more faithfully, four directional variants are processed in parallel: the original feature map $O \in \mathbb{R}^{H \times W \times 6}$6, its transpose $O \in \mathbb{R}^{H \times W \times 6}$7, its reversed version $O \in \mathbb{R}^{H \times W \times 6}$8, and the reversed transpose $O \in \mathbb{R}^{H \times W \times 6}$9. Each directional sequence is processed by an SSM with distinct parameters, reshaped back into windows, inverse-transformed, and summed. The stated purpose is to ensure strong local consistency and reduce the probability that adjacent pixels become distant sequence positions.

The Hierarchical selective scan addresses global dependency modeling. For feature map $O_{1:3}$0, the decoder creates multiple sub-images at level $O_{1:3}$1 using stride $O_{1:3}$2. The mapped coordinates are

$O_{1:3}$3

with $O_{1:3}$4 enumerating the top-left offsets of the $O_{1:3}$5-th sub-image at level $O_{1:3}$6. Each sub-image is zero-padded to the largest size for batched SSM processing and then passed through a level-specific local-enhanced SSM:

$O_{1:3}$7

Reverse mapping yields level-wise features,

$O_{1:3}$8

and multi-level fusion produces

$O_{1:3}$9

The critical distinction from ordinary hierarchical processing is explicit in the description: it performs divide-and-conquer sampling on the original resolution rather than downsampling the feature grid. This suggests that the method seeks to improve long-range correlation modeling without sacrificing original-resolution structure.

4. Supervision, datasets, and optimization

The training objective supervises both the flare-free image and the flare component. Given input $O_{4:6}$0 with ground-truth clean image $O_{4:6}$1 and flare $O_{4:6}$2, the network prediction is

$O_{4:6}$3

A compound image-level loss is defined by

$O_{4:6}$4

where $O_{4:6}$5 is pixel-wise L1 reconstruction and $O_{4:6}$6 is a VGG-based perceptual loss. Reconstruction consistency is enforced through

$O_{4:6}$7

The total loss is

$O_{4:6}$8

with empirical weights $O_{4:6}$9 (Huang et al., 4 Aug 2025).

Training uses Flare7K++ as the primary dataset, with on-the-fly synthetic pairing following the Flare7K++ protocol and backgrounds sampled from the 24K Flickr dataset. Data augmentation includes random rotation, translation, shearing, scaling, blurring, and color jitter, all reported as consistent with Flare7K++. Images are center-cropped to $\rightarrow$0. Optimization uses Adam with learning rate $\rightarrow$1, batch size $\rightarrow$2, and $\rightarrow$3 iterations. No specialized SSM initialization, including HIPPO-style initialization, is reported.

The repository is publicly available and is described as providing implementation and pretrained checkpoints consistent with the Flare7K++ protocol. For inference, flare-corrupted images, commonly at $\rightarrow$4, are passed through the network, which returns both the flare-free image in the first three output channels and the predicted flare component in the last three channels. These can be saved independently or recombined for consistency checks.

5. Quantitative evaluation and downstream behavior

On Flare7K++, DeflareMamba is reported to achieve the best PSNR and SSIM among the compared methods. The comparison provided in the paper is as follows (Huang et al., 4 Aug 2025):

Method	PSNR	SSIM
Input	22.56	0.857
U-Net	27.19	0.894
HINet	27.55	0.892
MPRNet*	27.04	0.893
Restormer*	27.60	0.897
Uformer	27.63	0.894
Difflare	26.06	0.898
Kopt.el	27.66	0.897
DeflareMamba	27.78	0.899

Ablation analysis attributes the final performance to the combined effect of the local-enhanced and hierarchical components. A U-shaped MambaIR baseline without local or hierarchical enhancements attains PSNR $\rightarrow$5 and SSIM $\rightarrow$6. Adding Local-enhanced SS2D raises performance to $\rightarrow$7 and $\rightarrow$8, an increase of approximately $\rightarrow$9 dB in PSNR. Adding Hierarchical Selective Scan further raises performance to $\rightarrow$0 and $\rightarrow$1, an additional gain of approximately $\rightarrow$2 dB and a total increase of approximately $\rightarrow$3 dB over the baseline. The reported convergence curve indicates that the U-shaped design both accelerates training and improves final PSNR relative to non-U-shaped MambaIR.

Qualitatively, the method is described as better preserving details near strong light sources, removing diffuse streaks without over-smoothing textures, and maintaining color fidelity in non-flare regions. Reflective artifacts are reported to be removed while respecting their spatial ties to the light sources, and this behavior is presented as evidence of contextual consistency.

The paper also reports downstream gains on COCO validation images corrupted by Flare7K++ flares. For object detection, the reported mAP values improve from flare-corrupted inputs to restored images. With flare, Faster R-CNN, Deformable DETR, and YOLO8x score $\rightarrow$4, $\rightarrow$5, and $\rightarrow$6; after Uformer restoration these become $\rightarrow$7, $\rightarrow$8, and $\rightarrow$9; after DeflareMamba restoration they become $\rightarrow$0, $\rightarrow$1, and $\rightarrow$2. For vision-language alignment, the reported scores are: with flare, CLIP CS $\rightarrow$3, BLIP CS $\rightarrow$4, ITM $\rightarrow$5, TR@1 $\rightarrow$6, TR@5 $\rightarrow$7; with Uformer, CLIP $\rightarrow$8, BLIP CS $\rightarrow$9, ITM $X \in \mathbb{R}^{H \times W \times C}$0, TR@1 $X \in \mathbb{R}^{H \times W \times C}$1, TR@5 $X \in \mathbb{R}^{H \times W \times C}$2; with DeflareMamba, CLIP $X \in \mathbb{R}^{H \times W \times C}$3, BLIP CS $X \in \mathbb{R}^{H \times W \times C}$4, ITM $X \in \mathbb{R}^{H \times W \times C}$5, TR@1 $X \in \mathbb{R}^{H \times W \times C}$6, TR@5 $X \in \mathbb{R}^{H \times W \times C}$7. The paper interprets these improvements as reflecting cleaner inputs, better object visibility, and reduced semantic noise for cross-modal models.

6. Efficiency, limitations, and subsequent development

DeflareMamba’s efficiency claim rests on the linear-time $X \in \mathbb{R}^{H \times W \times C}$8 selective SSM used for sequence processing, contrasted with the $X \in \mathbb{R}^{H \times W \times C}$9 cost of attention in Transformer models. The U-shaped design reduces sequence lengths at lower resolutions, which is said to strengthen correlations while reducing runtime and memory pressure. Window partitioning in the Local-enhanced module enables efficient batched SSM operations, and hierarchical stride sampling operates at original resolution via structured subsampling rather than explicit feature-grid downsampling. At the same time, the paper does not report parameter counts, FLOPs, or detailed runtime and memory metrics, and it does not claim real-time capability (Huang et al., 4 Aug 2025).

The stated limitations are concentrated in severe flare conditions. Completely removing long streak artifacts from severe scattering flares remains challenging. Extremely intense overexposures or complex, multi-bounce reflective patterns may still leave residual artifacts or attenuated textures. The hierarchical feature processing is concentrated in a single block, and the authors suggest that distributing hierarchical operations across multiple blocks while keeping the total block count fixed could reduce complexity and improve robustness. Proposed future directions include stronger long-range retention strategies for SSMs, richer physics priors linking light sources to reflective artifacts, and more advanced augmentation for extreme nighttime conditions.

A later system, DeflareMambav2, is explicitly presented as advancing DeflareMamba by introducing a Flare Prior Network, radial serialization, and prior-guided selective state-space modeling with dual-level adaptive restoration (Fu et al., 8 May 2026). In the comparison given there, the original DeflareMamba is characterized as a hierarchical Mamba with varied-stride scanning that still relied on spatially uniform serialization and lacked explicit separation of sources from flare artifacts. DeflareMambav2 adds three priors, distance-based radial serialization, a Heterogeneous Branch for identity routing of light-source regions, and a Radial State-Space Equation for token-wise parameter modulation. The same source reports an improvement on FlareX from Clean-PSNR $C$00 and Clean-SSIM $C$01 for DeflareMamba to $C$02 and $C$03 for DeflareMambav2, while reducing parameters from approximately $C$04M to approximately $C$05M. This suggests that DeflareMamba established the initial SSM-based formulation for contextually consistent flare removal, and that later work recast some of its remaining weaknesses as a problem of spatial heterogeneity and prior-guided restoration rather than sequence modeling alone.