Low-Light Video Enhancement (SCDA)

• Low-light video enhancement is a process to restore visual quality in dark videos, addressing noise, flicker, and exposure issues using advanced attention mechanisms.
• SCDA employs self-, cross-, and dilated attention blocks to integrate information across frames, enhancing temporal consistency and reducing artifacts without explicit motion estimation.
• Recent approaches combine spatio-temporal decomposition, transformer models, and event-based methods to achieve higher PSNR/SSIM and real-world robustness in dynamic low-light scenarios.

Low-light video enhancement (LLVE) seeks to restore visual quality and semantic utility in sequences severely degraded by insufficient illumination, noise, and exposure artifacts. This area has advanced rapidly due to both hardware progress (e.g., event cameras) and algorithmic innovations that leverage spatio-temporal models, decomposition, attention, and domain adaptation. Several state-of-the-art approaches, including those employing Self-Cross Dilated Attention (SCDA), have been proposed to address longstanding challenges of noise suppression, color correction, temporal flicker, and data scarcity. The following sections systematically organize the main problem formulations, algorithmic principles, model architectures—including SCDA-based approaches—theoretical and empirical results, and existing limitations in this domain.

1. Problem Formulation and Core Losses

The LLVE problem is generally posed as follows. Let $$X = \{x_1, \ldots, x_T\},\ x_t \in \mathbb{R}^{H\times W\times C}$$ be a low-light video with $T$ frames, and $\hat Y = \{\hat y_1, \ldots, \hat y_T\}$ the enhanced sequence. Enhancement is learned via a deep network $f_\theta$ such that $\hat y_t = f_\theta(x_1, \ldots, x_T)_t$ for $t=1,\ldots,T$ (Zheng et al., 2022). Training objective minimally requires frame-wise fidelity (e.g., $$\mathcal{L}_{\mathrm{rec}}(\hat y_t, y_t^{\mathrm{gt}})$$) plus a temporal regularizer, most commonly realized as flow-warping-based penalty: $$\mathcal{L}(\theta) = \sum_{t=1}^T \mathcal{L}_{\mathrm{rec}}(\hat y_t, y_t^{\mathrm{gt}}) + \lambda \sum_{t=2}^T \mathcal{L}_{\mathrm{temp}}(\hat y_t, \hat y_{t-1})$$ where

$$\mathcal{L}_{\mathrm{temp}} = \sum_{t=2}^{T} \left\| \mathcal{W}(\hat y_t, F_{t\to t-1}) - \hat y_{t-1} \right\|_1$$

with $F_{t\to t-1}$ the optical flow and $T$0 a differentiable warping operator.

Classic methods also incorporate Retinex-based or illumination-reflectance decompositions, additional adversarial or perceptual (VGG/LPIPS) losses, and temporal stability evaluated by warping-error or brightness consistency metrics (Xu et al., 2024, Azizi et al., 2022).

2. Self-Cross Dilated Attention (SCDA) and Related Attention Mechanisms

The SCDA block, as proposed in "Low Light Video Enhancement by Learning on Static Videos with Cross-Frame Attention," is a general-purpose module to address the exploitation of inter-frame information even under large dynamics or without access to paired dynamic training videos (Chhirolya et al., 2022). Its operation is as follows:

• For each encoder/bottleneck scale, three consecutive frames ($T$1) are split into non-overlapping $T$2 spatial blocks.
• Blocked multi-head self-attention is applied to each frame individually.
• Cross-attention maps are computed between the current frame and its temporal neighbors, allowing information exchange.
• To handle large inter-frame motion, a dilated-block strategy is used: for neighbor frames, blocks are sampled at double size and stride, substantially increasing the cross-frame receptive field without explicit motion estimation.
• All self-, cross-, and dilated-attention outputs are per-pixel softmax-fused and passed through a Residual Channel Attention Block (RCAB).
• The module requires no explicit optical flow estimation, yielding robustness in both static and dynamic test scenarios.

The SCDA-based model demonstrates strong performance in settings where only static (tripod or fixed-camera) low-light videos are available for training but dynamic (camera/object motion) sequences are seen at test time. Cross-frame matching via SCDA allows for learning temporal relationships absent in single-frame networks (Chhirolya et al., 2022).

3. Spatio-Temporal Decomposition and Dual-Structure Architectures

Recent approaches extend Retinex or intrinsic-image decompositions into spatial-temporal models. In "Low-Light Video Enhancement via Spatial-Temporal Consistent Decomposition" (Xu et al., 2024) and its more recent generalization VLLVE++ (Xu et al., 9 Feb 2026), each frame is factorized into: $T$3 where $T$4 is the per-frame illumination (view-dependent), $T$5 the intrinsic, view-independent reflectance, and $T$6 (in VLLVE++) a learned residual handling scene-adaptive degradations. Enhancement is driven by three superposed losses:

• Frame reconstruction: $T$7
• Cross-frame reflectance consistency, via sparse dense matches (e.g., DKM, LoFTR): penalize dissimilar $T$8 and $T$9 at corresponding locations
• Smoothness regularizer on $\hat Y = \{\hat y_1, \ldots, \hat y_T\}$0 and $\hat Y = \{\hat y_1, \ldots, \hat y_T\}$1

A Cross-Frame Interaction Module (CFIM) is placed at the U-Net bottleneck, providing both self- and cross-attention between features of neighboring frames, then fused and decoded for each output. Supervision is applied simultaneously to frame pairs, further promoting temporal coherence and robust decomposition (Xu et al., 2024, Xu et al., 9 Feb 2026). VLLVE++ introduces an additional bidirectional refinement network to adapt and filter correspondence maps, and a residual branch for hard-to-model degradations.

4. Diverse Modeling Paradigms: Transformers, GANs, and Unrolled Optimization

Several dominant network paradigms for LLVE include:

• Transformer-based architectures: STA-SUNet employs a Swin Transformer backbone with pyramid-cascade deformable alignment for spatio-temporal feature fusion, yielding superior PSNR/SSIM across BVI, DRV, and SDSD datasets (Lin et al., 2024). The VJT model uses a multi-tier transformer to jointly learn denoising, low-light enhancement, and deblurring, with illumination-boosted intermediate targets and adaptive tier-to-tier feature fusion (Hui et al., 2024).
• GAN/cycle-consistency-based approaches: SIDGAN synthesizes paired dynamic short-long RAW/RGB video data by chaining two CycleGANs: one between wild RGB and sensor-specific long-exposure RGB, and one (semi-supervised) between RGB and RAW. By training RAW-to-RGB translation on a mix of synthetic and real data, the resulting video models reach over 39 dB PSNR and high temporal consistency with minimal real paired footage (Triantafyllidou et al., 2020).
• Unrolled optimization/unpaired learning: UDU-Net frames enhancement as an unpaired MAP estimation problem and unrolls ADMM-style iterations with explicit intra-frame (expert prior/statistics, human feedback) and inter-frame (flow-guided) modules, enabling progressive spatial and temporal restoration without paired ground truth (Zhu et al., 2024).
• Hybrid/self-supervised regimes: SALVE bootstraps enhancement by self-supervised patch-wise ridge regression on a few Retinex-enhanced keyframes and applies the mapping to all subsequent frames, achieving high temporal consistency and low computational cost (Azizi et al., 2022).

5. Evaluation Benchmarks and Quantitative Results

Recent studies report results on a range of public and custom datasets:

• Night Wenzhou: Large-scale, real-world, unpaired night video for unsupervised/zero-shot evaluation, capturing severe real-world degradations (Zheng et al., 2022).
• SDSD, SMID, DRV: Mechatronically aligned multi-illumination and dynamic/static benchmarks (Xu et al., 2024, Xu et al., 9 Feb 2026).
• MLBN: Multi-degradation (blur, noise, low-light), per-scene low-light video set for joint evaluation (Hui et al., 2024).
• BVI-RLV, DID: Frame-registered indoor scenes for paired enhancement (Li et al., 12 Nov 2025, Lin et al., 10 Oct 2025).

SCDA-based and decomposition-based models consistently outperform single-frame and simple temporal models. For example, VLLVE++ leads with up to 31.06 dB PSNR / 0.95 SSIM on dynamic DID, a 0.5–1.5 dB gain over previous best (Xu et al., 9 Feb 2026). UDU-Net, in unpaired/unsupervised settings, matches or exceeds supervised baselines (SDSDNet) in both PSNR/SSIM and temporal warping error (Zhu et al., 2024). SCDA-attention models deliver sharp detail preservation and minimal flicker on synthetic and real dynamic video (Chhirolya et al., 2022).

Method	Dataset	PSNR (dB)	SSIM	Temporal Coherence
VLLVE++	DID	31.06	0.95	state-of-the-art
UDU-Net	SDSD-out	23.94	0.745	best warp error
STA-SUNet	BVI (10%)	27.32	0.847	high stability
SCDA	DAVIS	29.02	0.82	ST-RRED=241
DWTA-Net	DID	24.27	0.857

All values reported exactly as in the respective cited sources.

6. Real-World Integration and Event-Based Methods

Recent LLVE pipelines recognize that enhancements must operate under hardware, temporal, and application constraints:

• Event-based fusion, as demonstrated in EvLight++, combines event streams with frames using SNR-guided multimodal fusion and temporal recurrence (convGRU), yielding large gains over both image- and video-based baselines (+1.37 dB over best single-image, +3.71 dB over best video) (Chen et al., 2024). Downstream metrics (e.g., mIoU for segmentation) also substantially improve.
• Edge/cloud pipelines coordinate video compression quality (QP) and enhancement network scale for joint tradeoff of communication, computational resource, and inference accuracy, with controllers optimizing throughput and quality under real-world constraints (He et al., 2023).

7. Limitations, Open Challenges, and Future Directions

Despite rapid progress, LLVE faces persistent challenges:

• Uneven exposure and compound impairments (e.g., co-occurring haze, noise, extreme lighting) remain unresolved (Zheng et al., 2022).
• Most SCDA and attention-based models operate at significant computational cost; real-time or mobile settings demand lightweight architectures (Lin et al., 2024).
• Decomposition models’ reliance on dense correspondence or flow estimation may degrade under extreme motion or illumination shift (Xu et al., 9 Feb 2026).
• Generalization to fully unsupervised, unpaired, and cross-domain video remains an open direction. Hybridization of decomposition, attention, and domain adaptation, as well as joint optimization with downstream tasks (segmentation, tracking), are under active investigation (Hui et al., 2024, Chen et al., 2024).

Emerging work foregrounds the integration of self-, cross-, and dilated attention (SCDA and variants), decomposition with view/task-aware priors, and transformer-based long-range modeling as principal algorithmic axes for the next generation of LLVE systems.