Bilateral Spectrum Aligner (Bi-SAB)

• Bi-SAB is a decoder block in DRWKV that fuses brightness and color features using cross-attentive alignment to mitigate brightness–color misalignment and artifact amplification.
• It combines spectral enhancement, edge-guided refinement, and feature difference adjustment to ensure restored images have natural brightness, faithful color, and crisp edges.
• Empirical tests demonstrate that integrating Bi-SAB improves PSNR and SSIM metrics while reducing computational cost in low-light image enhancement tasks.

Searching arXiv for the specified paper to ground the article in the primary source. Search query: (Bai et al., 24 Jul 2025) OR "DRWKV: Focusing on Object Edges for Low-Light Image Enhancement" The Bilateral Spectrum Aligner (Bi-SAB) is a decoder block introduced in DRWKV for low-light image enhancement, where it serves as the module responsible for jointly refining luminance and chrominance features so that enhanced outputs exhibit natural brightness, faithful color, and minimal artifacts. Within DRWKV, which also integrates Global Edge Retinex (GER) theory to decouple illumination and edge structures and Evolving WKV Attention to model spatial edge continuity, Bi-SAB functions as the principal brightness–color fusion mechanism in the decoder and is trained end-to-end under the tailored MS$^2$-Loss regime (Bai et al., 24 Jul 2025).

1. Functional role within DRWKV

Bi-SAB is inserted in the decoder of DRWKV at each resolution level where a cross-scale skip connection joins the encoder and decoder. Its inputs and output share the same tensor shape, with low-level feature $F_1 \in \mathbb{R}^{B \times C \times H \times W}$ carrying fine details and brightness cues, higher-level feature $F_1' \in \mathbb{R}^{B \times C \times H \times W}$ carrying richer color and contextual information, and output feature $$F_{\text{out}} \in \mathbb{R}^{B \times C \times H \times W}$$ (Bai et al., 24 Jul 2025).

In this placement, Bi-SAB is not a stand-alone enhancement network but a decoder-stage fusion unit. A common simplification is to treat it as only an attention block; in DRWKV it is instead a composite module whose operation combines cross-attentive alignment, spectral enhancement, and edge-guided refinement. This architectural role is central to the paper’s broader claim that low-light enhancement quality depends not only on illumination correction, but also on controlled coordination between brightness restoration, color consistency, and edge preservation.

2. Motivation: spectral alignment for brightness–color fusion

The paper motivates Bi-SAB by identifying two failure modes that arise under extreme low-light conditions when illumination and color are enhanced separately. The first is Brightness–Color Misalignment: a luminance-only branch may over-brighten regions without simultaneously correcting color statistics, producing hue shifts or patchy color reproduction, while a chrominance branch that depends on deeper features may lose edge fidelity when its color adjustments are applied after brightening. The second is Artifact Amplification: if noise is amplified in the brightness path and then recolored by chrominance corrections, ringing, halos, and color speckles become more visible (Bai et al., 24 Jul 2025).

Bi-SAB is designed specifically to address those two modes. The paper describes three design principles. Cross-Attentive Alignment allows the brightness feature to query the color feature through a lightweight cross-attention mechanism so that color statistics are drawn from the appropriate spatial locations. Spectral Enhancement uses a Spectral Alignment Enhancer (SAE) to adaptively fuse corrected feature spectra in order to suppress noise and reinforce weak edges. Edge-Guided Refinement adds a Scharr Edge Enhancement (SEE) path to preserve and sharpen fine structural detail. Together, these principles are intended to make the fusion both luminance-coherent and chrominance-faithful. A plausible implication is that Bi-SAB operationalizes “spectral alignment” not as a frequency-domain transform in isolation, but as a coordinated feature-space alignment between brightness-related and color-related representations.

3. Internal structure and data flow

Bi-SAB comprises four sub-modules arranged after initial query–key–value projection. The projections are

$$F_Q = \mathrm{Conv}_{3 \times 3}(F_1), \qquad F_K = \mathrm{Conv}_{3 \times 3}(F_1'), \qquad F_V = \mathrm{Conv}_{3 \times 3}(F_1'),$$

with each convolution using $C$ input and $C$ output channels (Bai et al., 24 Jul 2025).

Component	Core operation	Declared purpose
Cross-Attention (CA)	Brightness queries color through lightweight cross-attention	Spatially aligned color statistics
Feature Difference Adjustment (FDA)	Combines attended feature, key, value, and query with fixed $\lambda$	Corrective fusion before spectral enhancement
Scharr Edge Enhancement (SEE)	Fixed Scharr gradients plus convolutional restoration	Preserve and sharpen structural detail
Spectral Alignment Enhancer (SAE)	Layer-normalized, split-stream depth-wise processing and multiplicative fusion	Suppress noise and reinforce weak edges

The Cross-Attention (CA) module produces

$$F_{\text{attn}} = \mathrm{CA}(F_Q, F_K) \in \mathbb{R}^{B \times C \times H \times W}.$$

According to the paper, CA uses $1 \times 1$ and then $F_1 \in \mathbb{R}^{B \times C \times H \times W}$0 depth-wise convolutions to generate $F_1 \in \mathbb{R}^{B \times C \times H \times W}$1, $F_1 \in \mathbb{R}^{B \times C \times H \times W}$2, and $F_1 \in \mathbb{R}^{B \times C \times H \times W}$3, splits channels into $F_1 \in \mathbb{R}^{B \times C \times H \times W}$4 heads, performs per-head $F_1 \in \mathbb{R}^{B \times C \times H \times W}$5 normalization, computes softmaxed cross-covariance attention, and re-projects the result to $F_1 \in \mathbb{R}^{B \times C \times H \times W}$6 channels. This makes the color/context feature $F_1 \in \mathbb{R}^{B \times C \times H \times W}$7 the source of both key and value information while the low-level detail feature $F_1 \in \mathbb{R}^{B \times C \times H \times W}$8 provides the query signal.

The Feature Difference Adjustment (FDA) stage then computes

$F_1 \in \mathbb{R}^{B \times C \times H \times W}$9

$F_1' \in \mathbb{R}^{B \times C \times H \times W}$0

$F_1' \in \mathbb{R}^{B \times C \times H \times W}$1

The fixed coefficient $F_1' \in \mathbb{R}^{B \times C \times H \times W}$2 constrains the scale of the attention-modulated correction.

The Scharr Edge Enhancement (SEE) branch computes $F_1' \in \mathbb{R}^{B \times C \times H \times W}$3, where fixed Scharr kernels $F_1' \in \mathbb{R}^{B \times C \times H \times W}$4 and $F_1' \in \mathbb{R}^{B \times C \times H \times W}$5 extract gradients, the gradients are fused, and the result is restored through a $F_1' \in \mathbb{R}^{B \times C \times H \times W}$6 convolutional path with SiLU. In parallel, the Spectral Alignment Enhancer (SAE) computes

$F_1' \in \mathbb{R}^{B \times C \times H \times W}$7

where the normalized feature is expanded by $F_1' \in \mathbb{R}^{B \times C \times H \times W}$8, split into two halves, each half is processed by $F_1' \in \mathbb{R}^{B \times C \times H \times W}$9 residual, the two streams are multiplied element-wise, and a final $$F_{\text{out}} \in \mathbb{R}^{B \times C \times H \times W}$$0 reduces the channels back to $$F_{\text{out}} \in \mathbb{R}^{B \times C \times H \times W}$$1. The final output is

$$F_{\text{out}} \in \mathbb{R}^{B \times C \times H \times W}$$2

This decomposition makes clear that Bi-SAB is not a purely attentional decoder block. Its output is explicitly the sum of a spectrum-aligned branch and an edge-preserving branch, which is consistent with the paper’s emphasis on simultaneously maintaining color naturalness and edge continuity.

4. Exact mathematical formulation

The paper presents the complete Bi-SAB formulation for $$F_{\text{out}} \in \mathbb{R}^{B \times C \times H \times W}$$3 as follows (Bai et al., 24 Jul 2025).

For the projection stage,

$$F_{\text{out}} \in \mathbb{R}^{B \times C \times H \times W}$$4

where each $$F_{\text{out}} \in \mathbb{R}^{B \times C \times H \times W}$$5 denotes a $$F_{\text{out}} \in \mathbb{R}^{B \times C \times H \times W}$$6 convolution.

For cross-attention, the channels are split into heads:

$$F_{\text{out}} \in \mathbb{R}^{B \times C \times H \times W}$$7

with analogous definitions for $$F_{\text{out}} \in \mathbb{R}^{B \times C \times H \times W}$$8 and $$F_{\text{out}} \in \mathbb{R}^{B \times C \times H \times W}$$9. Per-head normalization is

$$F_Q = \mathrm{Conv}_{3 \times 3}(F_1), \qquad F_K = \mathrm{Conv}_{3 \times 3}(F_1'), \qquad F_V = \mathrm{Conv}_{3 \times 3}(F_1'),$$0

The cross-covariance attention matrix is

$$F_Q = \mathrm{Conv}_{3 \times 3}(F_1), \qquad F_K = \mathrm{Conv}_{3 \times 3}(F_1'), \qquad F_V = \mathrm{Conv}_{3 \times 3}(F_1'),$$1

and the aggregated values are

$$F_Q = \mathrm{Conv}_{3 \times 3}(F_1), \qquad F_K = \mathrm{Conv}_{3 \times 3}(F_1'), \qquad F_V = \mathrm{Conv}_{3 \times 3}(F_1'),$$2

After rearrangement and projection,

$$F_Q = \mathrm{Conv}_{3 \times 3}(F_1), \qquad F_K = \mathrm{Conv}_{3 \times 3}(F_1'), \qquad F_V = \mathrm{Conv}_{3 \times 3}(F_1'),$$3

For Feature Difference Adjustment,

$$F_Q = \mathrm{Conv}_{3 \times 3}(F_1), \qquad F_K = \mathrm{Conv}_{3 \times 3}(F_1'), \qquad F_V = \mathrm{Conv}_{3 \times 3}(F_1'),$$4

$$F_Q = \mathrm{Conv}_{3 \times 3}(F_1), \qquad F_K = \mathrm{Conv}_{3 \times 3}(F_1'), \qquad F_V = \mathrm{Conv}_{3 \times 3}(F_1'),$$5

$$F_Q = \mathrm{Conv}_{3 \times 3}(F_1), \qquad F_K = \mathrm{Conv}_{3 \times 3}(F_1'), \qquad F_V = \mathrm{Conv}_{3 \times 3}(F_1'),$$6

For Scharr Edge Enhancement,

$$F_Q = \mathrm{Conv}_{3 \times 3}(F_1), \qquad F_K = \mathrm{Conv}_{3 \times 3}(F_1'), \qquad F_V = \mathrm{Conv}_{3 \times 3}(F_1'),$$7

$$F_Q = \mathrm{Conv}_{3 \times 3}(F_1), \qquad F_K = \mathrm{Conv}_{3 \times 3}(F_1'), \qquad F_V = \mathrm{Conv}_{3 \times 3}(F_1'),$$8

$$F_Q = \mathrm{Conv}_{3 \times 3}(F_1), \qquad F_K = \mathrm{Conv}_{3 \times 3}(F_1'), \qquad F_V = \mathrm{Conv}_{3 \times 3}(F_1'),$$9

$C$0

For the Spectral Alignment Enhancer,

$C$1

$C$2

$C$3

$C$4

The block concludes with

$C$5

Formally, this structure couples channel-wise cross-covariance attention with multiplicative spectral fusion and explicit gradient-based structural injection. This suggests that the block’s behavior is governed by two complementary priors: cross-feature correspondence across luminance and chrominance pathways, and local edge stability under restoration.

5. Optimization under MS$C$6-Loss and implementation choices

The entire DRWKV network, including Bi-SAB, is trained end-to-end using the multi-term MS$C$7-Loss, which enforces five objectives: pixel-wise reconstruction $C$8, edge sparsity $C$9, illumination smoothness $C$0, artifact suppression $C$1, and parameter regularization $C$2 (Bai et al., 24 Jul 2025).

The loss terms are defined as

$C$3

$C$4

$C$5

$C$6

$C$7

The total loss is

$C$8

The paper attributes distinct roles to Bi-SAB within this loss regime. Its cross-attention and spectral enhancer actively reduce $C$9 by improving alignment of brightness and color, and reduce $\lambda$0 by suppressing noise–color misregistration, while the SEE branch supports $\lambda$1 by preserving true edges. This suggests that Bi-SAB is supervised indirectly through a coupled restoration objective rather than by an isolated color-alignment criterion.

Implementation details are specified with unusual precision. At each Bi-SAB resolution level, the channel count is $\lambda$2. All convolutions use kernel sizes $\lambda$3 or $\lambda$4, and the depth-wise convolutions in CA and SAE have one group per channel. Cross-attention uses $\lambda$5 heads with learnable temperature $\lambda$6 initialized to $\lambda$7, and the FDA weight $\lambda$8 is fixed. DRWKV is trained with Adam using $\lambda$9 and $$F_{\text{attn}} = \mathrm{CA}(F_Q, F_K) \in \mathbb{R}^{B \times C \times H \times W}.$$0, initial learning rate $$F_{\text{attn}} = \mathrm{CA}(F_Q, F_K) \in \mathbb{R}^{B \times C \times H \times W}.$$1, cosine annealing to $$F_{\text{attn}} = \mathrm{CA}(F_Q, F_K) \in \mathbb{R}^{B \times C \times H \times W}.$$2 over 500 epochs, and batch size tuned to fill an A800-80 GB GPU. The loss weights are $$F_{\text{attn}} = \mathrm{CA}(F_Q, F_K) \in \mathbb{R}^{B \times C \times H \times W}.$$3, $$F_{\text{attn}} = \mathrm{CA}(F_Q, F_K) \in \mathbb{R}^{B \times C \times H \times W}.$$4, $$F_{\text{attn}} = \mathrm{CA}(F_Q, F_K) \in \mathbb{R}^{B \times C \times H \times W}.$$5, $$F_{\text{attn}} = \mathrm{CA}(F_Q, F_K) \in \mathbb{R}^{B \times C \times H \times W}.$$6, and $$F_{\text{attn}} = \mathrm{CA}(F_Q, F_K) \in \mathbb{R}^{B \times C \times H \times W}.$$7. The input crop size is $$F_{\text{attn}} = \mathrm{CA}(F_Q, F_K) \in \mathbb{R}^{B \times C \times H \times W}.$$8, and up/down-sampling in other DRWKV stages is performed via wavelet transforms.

6. Quantitative contribution, visual effects, and interpretation

On the LOLv2-Real test set, the paper reports a progressive ablation isolating Bi-SAB’s effect. A baseline with ES-RWKV + SIR and no Bi-SAB obtains $$F_{\text{attn}} = \mathrm{CA}(F_Q, F_K) \in \mathbb{R}^{B \times C \times H \times W}.$$9 dB and $1 \times 1$0. Adding Bi-SAB while still using SSIM + $1 \times 1$1 loss raises performance to $1 \times 1$2 dB and $1 \times 1$3, corresponding to gains of $1 \times 1$4 dB and $1 \times 1$5, respectively. Replacing the $1 \times 1$6 loss with MS$1 \times 1$7-Loss further increases performance to $1 \times 1$8 dB and $1 \times 1$9 (Bai et al., 24 Jul 2025).

The same ablation is used to support an efficiency claim: spectral alignment alone yields a substantial improvement in both distortion and perceptual metrics while halving the model’s GFLOPS relative to a naïve UNet-style integration. Within the scope of the paper, this identifies Bi-SAB as a decoder mechanism intended to improve fidelity without disproportionate computational growth.

The paper also attributes several qualitative effects to Bi-SAB’s aligned outputs: elimination of color bleeding and hue shifts around bright edges, cleaner shadow regions without colored noise speckles, and crisp edge continuity consistent with human perception. It further states that these effects are validated by a user-study MOS increase over the nearest competitor. Because no separate MOS value is provided in the supplied data, the result is best interpreted qualitatively rather than as a quantified benchmark comparison.

Several clarifications follow from the ablation. First, Bi-SAB should not be understood as contributing only chromatic correction; the reported gains include both PSNR and SSIM improvement, indicating impact on distortion-oriented and structural quality measures. Second, its effect is not reducible to attention alone, because the block’s declared contribution depends on the combination of CA, SAE, and SEE. Third, its role is specifically decoder-centric: it aligns skip-connected low-level and high-level features during reconstruction, rather than replacing the paper’s separate mechanisms for illumination–edge decoupling or spatial continuity modeling.