SAIGFormer: Illumination-Guided Transformer

• The paper introduces an innovative low-light enhancement framework that adaptively estimates spatially varying illumination and injects it into Transformer attention to improve non-uniform scene restoration.
• The methodology combines a dynamic integral image-based illumination estimator (SAI²E) with an illumination-guided multi-head self-attention mechanism (IG-MSA) within a U-Net-like Transformer architecture.
• Empirical results demonstrate that SAIGFormer outperforms prior methods on benchmarks by improving both PSNR and SSIM while more faithfully preserving spatial details.

SAIGFormer, abbreviated from Spatially-Adaptive Illumination-Guided Transformer, is a low-light image enhancement framework designed for scenes with spatially non-uniform illumination, including backlit foregrounds, shadowed regions, and mixed bright/dark content. Its central premise is that accurate enhancement requires both spatially adaptive illumination estimation and illumination-guided feature modeling inside a hierarchical Transformer. The framework combines a dynamic integral-image-based illumination estimator, termed $\text{SAI}^2\text{E}$, with an illumination-guided attention mechanism, IG-MSA, inside a U-Net-like encoder–decoder Transformer. In the reported formulation, the network predicts a residual image $I_{res}$ and produces the final result through $\hat{I}_{out} = I + I_{res}$, rather than relying on uniform brightening or a static global illumination prior (Li et al., 21 Jul 2025).

1. Problem setting and conceptual scope

Low-light image enhancement (LLIE) seeks to transform a poorly lit image into a visually pleasing and faithful one, but SAIGFormer is specifically motivated by the failure modes of recent Transformer-based LLIE methods under non-uniform illumination. The problem setting emphasized for SAIGFormer includes backlit regions that remain underexposed, shadow regions that are poorly restored, and already well-lit regions that become overexposed when enhancement is applied uniformly. The restoration task is further entangled with noise, artifacts, and color distortion, which makes global or approximately uniform illumination correction insufficient (Li et al., 21 Jul 2025).

The framework is positioned against two broad tendencies in prior work. One tendency is to use a global or approximate illumination prior, such as mean or max RGB statistics. The other is to adopt Retinex-style decomposition with complex architectures and multiple losses. SAIGFormer is presented as a response to both limitations: it does not rely on a static, globally applied illumination prior, and it does not only guide the network through coarse illumination features after a preliminary enhancement stage. Instead, it estimates illumination early from the original image, makes the illumination map spatially adaptive, and injects illumination directly into attention.

This positioning is closely related to the motivation behind Retinexformer, which argued that low-light restoration benefits from illumination-guided non-local interaction rather than uniform processing across pixels. Retinexformer formalized a one-stage Retinex-based framework and introduced an Illumination-Guided Transformer that uses illumination representations to direct non-local interactions of regions with different lighting conditions (Cai et al., 2023). SAIGFormer extends this general line of thought toward explicitly spatially varying illumination estimation for non-uniform scenes.

2. Architectural organization

SAIGFormer is a U-Net-like hierarchical Transformer with four spatial scales. It contains two major components. The first is $\text{SAI}^2\text{E}$, the Spatially-Adaptive Integral Illumination Estimator, which estimates a spatially varying illumination map directly from the original image. The second is SAIGT, the Spatially-Adaptive Illumination Guided Transformer block, whose core attention module is IG-MSA. The overall organization is therefore an encoder–decoder Transformer in which illumination is estimated once from the input image and then propagated across scales to condition feature processing (Li et al., 21 Jul 2025).

Given a low-light input image

$I \in \mathbb{R}^{H \times W \times 3},$

the framework first extracts a shallow feature embedding

$F_0 \in \mathbb{R}^{H \times W \times C},$

and estimates illumination

$I_{L_0} \in \mathbb{R}^{H \times W \times 3}.$

The image features then pass through the U-shaped encoder–decoder Transformer, each stage uses SAIGT blocks, illumination maps are downsampled across scales and shared between encoder and decoder, and the network outputs a residual image

$I_{res} \in \mathbb{R}^{H \times W \times 3}.$

The final prediction is

$\hat{I}_{out} = I + I_{res}.$

This residual formulation places SAIGFormer among direct image-restoration architectures rather than explicit decomposition-and-reconstruction pipelines. Compared with methods such as Uformer, Restormer, Retinexformer, SNR-Net, and CIDNet, the framework is distinguished by early illumination estimation from the original image, spatial adaptivity in the illumination map itself, and direct illumination injection into attention. The intended consequence is more faithful treatment of uneven lighting: well-lit regions are less prone to over-brightening, while shadowed or backlit regions receive targeted restoration.

3. Dynamic integral image representation and $\text{SAI}^2\text{E}$

The illumination estimator is based on the assumption that illumination mainly resides in the low-frequency component of the image. SAIGFormer argues that fixed-window low-pass filtering is too rigid for real scenes because well-lit and dark regions require different smoothing scales. $I_{res}$0 therefore replaces a fixed receptive field with a dynamic integral image representation that adapts the integration region per pixel (Li et al., 21 Jul 2025).

The estimator begins with the integral image, or summed-area table,

$I_{res}$1

This permits constant-time computation of sums over rectangular regions once the integral image has been constructed. Two small subnetworks then parameterize the local integration region. Offset-Net predicts

$I_{res}$2

and Modulation-Net predicts

$I_{res}$3

Their definitions are given as

$I_{res}$4

$I_{res}$5

The four channels of $I_{res}$6 correspond to top, left, bottom, and right offsets, denoted $I_{res}$7.

Because training uses random cropping, the offsets are scaled according to

$I_{res}$8

where $I_{res}$9 denote the training crop size and $\hat{I}_{out} = I + I_{res}$0 the original image dimensions. The four corners of the dynamic rectangular region centered at $\hat{I}_{out} = I + I_{res}$1 are then

$\hat{I}_{out} = I + I_{res}$2

Using these corners, the dynamic integral image response at $\hat{I}_{out} = I + I_{res}$3 is

$\hat{I}_{out} = I + I_{res}$4

The corresponding area is

$\hat{I}_{out} = I + I_{res}$5

which yields the estimated illumination

$\hat{I}_{out} = I + I_{res}$6

Finally, modulation is applied:

$\hat{I}_{out} = I + I_{res}$7

In functional terms, $\hat{I}_{out} = I + I_{res}$8 is a learned, spatially adaptive low-pass estimator based on integral images, with $\hat{I}_{out} = I + I_{res}$9 computation per pixel after the integral image is built. The model discussion states that bright, well-exposed regions benefit from small integration windows, whereas dark, noisy regions benefit from larger windows. The paper further emphasizes that this is the first use of dynamic integral image representation in deep learning for LLIE. A plausible implication is that SAIGFormer treats illumination estimation not merely as a side prior but as a locally conditioned signal whose spatial support varies with scene content.

4. Illumination-guided attention and Dual-Gated Feed-Forward refinement

The transformer block used throughout the hierarchy is SAIGT. Each block contains Pre-LayerNorm, IG-MSA, and a Dual-Gated Feed-Forward Network (DG-FFN). Its update equations are

$\text{SAI}^2\text{E}$0

$\text{SAI}^2\text{E}$1

This makes illumination conditioning a first-class component of the feature transformation, rather than a post hoc correction (Li et al., 21 Jul 2025).

For a feature map $\text{SAI}^2\text{E}$2, the model first applies a $\text{SAI}^2\text{E}$3 convolution for channel mixing and a $\text{SAI}^2\text{E}$4 depthwise separable convolution for local encoding, and then splits the result into query, key, and value tensors:

$\text{SAI}^2\text{E}$5

where $\text{SAI}^2\text{E}$6 denotes depthwise separable convolution and $\text{SAI}^2\text{E}$7 denotes pointwise convolution. Illumination is incorporated by downsampling the illumination map with a $\text{SAI}^2\text{E}$8 depthwise separable convolution, channel-aligning it with a $\text{SAI}^2\text{E}$9 convolution, and concatenating it with the query:

$I \in \mathbb{R}^{H \times W \times 3},$0

$I \in \mathbb{R}^{H \times W \times 3},$1

The resulting query representation satisfies

$I \in \mathbb{R}^{H \times W \times 3},$2

The attention computation is written as

$I \in \mathbb{R}^{H \times W \times 3},$3

and the output projection is

$I \in \mathbb{R}^{H \times W \times 3},$4

where $I \in \mathbb{R}^{H \times W \times 3},$5 is a learnable scaling parameter. In the description accompanying the formulation, illumination influences the affinity computation, emphasizes features useful for restoring dark or unevenly lit areas, and suppresses over-brightening in already bright regions.

The DG-FFN further refines the illumination-guided features through a dual gating structure:

$I \in \mathbb{R}^{H \times W \times 3},$6

This module expands features by two $I \in \mathbb{R}^{H \times W \times 3},$7 convolutions and recombines them through two gating paths to preserve and refine illumination-sensitive information.

A useful comparative distinction emerges when SAIGFormer is set beside Retinexformer. Retinexformer’s IG-MSA uses an illumination-aware multiplicative gate on the value path, with illumination features modulating $I \in \mathbb{R}^{H \times W \times 3},$8 inside attention, whereas SAIGFormer injects illumination into the query path to calibrate affinity computation itself (Cai et al., 2023). This suggests two related but not identical interpretations of illumination guidance: one centered on value gating after affinity formation, and the other centered on illumination-conditioned affinity construction.

5. Optimization protocol and benchmark configuration

The reported network embeds the input image into a 32-channel feature map, and the numbers of SAIGT blocks across stages are

$I \in \mathbb{R}^{H \times W \times 3},$9

Training uses 300k iterations, the Adam optimizer with $F_0 \in \mathbb{R}^{H \times W \times C},$0 and $F_0 \in \mathbb{R}^{H \times W \times C},$1, batch size 8, and a learning rate of $F_0 \in \mathbb{R}^{H \times W \times C},$2 with cosine annealing to $F_0 \in \mathbb{R}^{H \times W \times C},$3. Data augmentation consists of random flipping, rotations by $F_0 \in \mathbb{R}^{H \times W \times C},$4, $F_0 \in \mathbb{R}^{H \times W \times C},$5, and $F_0 \in \mathbb{R}^{H \times W \times C},$6, and random cropping to $F_0 \in \mathbb{R}^{H \times W \times C},$7. The implementation is reported as trained on a single NVIDIA RTX 4090 GPU (Li et al., 21 Jul 2025).

The loss function combines L1 loss and SSIM loss. The SSIM component is explicitly given by

$F_0 \in \mathbb{R}^{H \times W \times C},$8

This choice reflects an optimization target that balances pixel-level fidelity with structural similarity.

Evaluation is conducted on six benchmarks: LOL-v1, LOL-v2-Real, LOL-v2-Syn, SID, SMID, and LOL-Blur. The dataset details are specified as follows. LOL-v1 contains 500 paired images with 485 for training and 15 for testing. LOL-v2-Real uses 689 training and 100 testing images. LOL-v2-Syn uses 900 training and 100 testing images. SID uses 2,099 training and 598 testing images, with RAW converted to sRGB. SMID uses 15,763 training images and the remaining samples for testing, again with RAW converted to sRGB. LOL-Blur contains 12,000 paired images split 17:3. The evaluation metrics are PSNR and SSIM, and comparison results for other methods are taken from public code, pretrained weights, or reported papers.

6. Empirical performance, ablation evidence, and relation to adjacent methods

The quantitative results reported for SAIGFormer cover paired LLIE benchmarks and a cross-domain blur-corrupted benchmark. The following values are explicitly stated for the final model (Li et al., 21 Jul 2025):

Dataset	PSNR	SSIM
LOL-v1	24.94	0.863
LOL-v2-Real	23.84	0.873
LOL-v2-Syn	26.60	0.946
SID	23.50	0.687
SMID	29.67	0.831
LOL-Blur	27.41	0.908

These results are contextualized by several pairwise comparisons. On LOL-v1, SAIGFormer is better than SNR-Net by $F_0 \in \mathbb{R}^{H \times W \times C},$9 dB and better than MIRNet by $I_{L_0} \in \mathbb{R}^{H \times W \times 3}.$0 dB. On LOL-v2-Real, it is second best, surpasses URWKV by $I_{L_0} \in \mathbb{R}^{H \times W \times 3}.$1 dB, and is only $I_{L_0} \in \mathbb{R}^{H \times W \times 3}.$2 dB below CIDNet. On LOL-v2-Syn, it improves over URWKV by $I_{L_0} \in \mathbb{R}^{H \times W \times 3}.$3 dB. On SID, it obtains the best SSIM, the second-best PSNR, is the best among methods from the past two years, and outperforms URWKV by $I_{L_0} \in \mathbb{R}^{H \times W \times 3}.$4 dB. On SMID, it is best overall, beating URWKV by $I_{L_0} \in \mathbb{R}^{H \times W \times 3}.$5 dB and Retinexformer by $I_{L_0} \in \mathbb{R}^{H \times W \times 3}.$6 dB. On LOL-Blur, it gives the best result, beating URWKV by $I_{L_0} \in \mathbb{R}^{H \times W \times 3}.$7 dB and PDHAT by $I_{L_0} \in \mathbb{R}^{H \times W \times 3}.$8 dB.

The qualitative characterization is equally specific. LLFlow and GSAD can overexpose images; CIDNet may underexpose dark areas; Retinexformer and SNR-Net can introduce artifacts and noise amplification; URWKV can still produce global artifacts in extreme cases. SAIGFormer is described as better preserving spatially varying illumination, avoiding over-brightening of well-lit regions, restoring shadow and backlit regions more faithfully, and improving color fidelity and detail recovery. On SID and SMID, the reported behavior is especially strong in backlit areas, shadowed regions, and noise-contaminated content. On LOL-Blur, the method also restores details in extremely dark and blurred regions, which suggests robustness across mixed degradations.

The ablation evidence isolates the contribution of the two principal modules. A baseline model achieves 23.01 PSNR and 0.867 SSIM; adding IG-MSA raises performance to 23.22 and 0.871; adding both IG-MSA and $I_{L_0} \in \mathbb{R}^{H \times W \times 3}.$9 yields 23.84 and 0.873. A separate design comparison for the estimator reports 23.01/0.867 for the baseline, 22.83/0.870 for avgpool $I_{res} \in \mathbb{R}^{H \times W \times 3}.$0, 22.95/0.870 without the modulation map, and 23.84/0.873 for full $I_{res} \in \mathbb{R}^{H \times W \times 3}.$1. The stated conclusion is that fixed-size pooling is inferior to adaptive integral estimation, the modulation map is important, and spatial adaptivity is key for proper illumination estimation.

The reported model size is 12.35M parameters. The paper describes this as moderate relative to many strong Transformer-based LLIE models and connects it to practical use. Practical application domains named for the method include smartphone photography in poor lighting, surveillance footage, driving or night-time vision, RAW-to-sRGB enhancement, and joint enhancement/deblurring under dark conditions. In relation to adjacent illumination-guided transformers, SAIGFormer can therefore be understood as a system in which illumination is estimated early, represented spatially adaptively, and used to calibrate attention for non-uniform enhancement; Retinexformer provides an important conceptual precursor by demonstrating that illumination-guided non-local interaction can be embedded within a one-stage restoration framework (Cai et al., 2023).