---
title: Bilateral Spectrum Aligner (Bi-SAB)
url: https://www.emergentmind.com/topics/bilateral-spectrum-aligner-bi-sab
type: topic
---

# Bilateral Spectrum Aligner (Bi-SAB)

Searching arXiv for the specified paper to ground the article in the primary source.
Search query: arXiv:2507.18594 OR "DRWKV: Focusing on Object Edges for Low-Light Image Enhancement"
The **Bilateral Spectrum Aligner (Bi-SAB)** is a decoder block introduced in DRWKV for low-light image enhancement, where it serves as the module responsible for jointly refining luminance and chrominance features so that enhanced outputs exhibit natural brightness, faithful color, and minimal artifacts. Within DRWKV, which also integrates Global Edge Retinex (GER) theory to decouple illumination and edge structures and Evolving WKV Attention to model spatial edge continuity, Bi-SAB functions as the principal brightness–color fusion mechanism in the decoder and is trained end-to-end under the tailored MS$^2$-Loss regime [2507.18594].

## 1. Functional role within DRWKV

Bi-SAB is inserted in the decoder of DRWKV at each resolution level where a cross-scale skip connection joins the encoder and decoder. Its inputs and output share the same tensor shape, with low-level feature $F_1 \in \mathbb{R}^{B \times C \times H \times W}$ carrying fine details and brightness cues, higher-level feature $F_1' \in \mathbb{R}^{B \times C \times H \times W}$ carrying richer color and contextual information, and output feature $F_{\text{out}} \in \mathbb{R}^{B \times C \times H \times W}$ [2507.18594].

In this placement, Bi-SAB is not a stand-alone enhancement network but a decoder-stage fusion unit. A common simplification is to treat it as only an attention block; in DRWKV it is instead a composite module whose operation combines cross-attentive alignment, spectral enhancement, and edge-guided refinement. This architectural role is central to the paper’s broader claim that low-light enhancement quality depends not only on illumination correction, but also on controlled coordination between brightness restoration, color consistency, and edge preservation.

## 2. Motivation: spectral alignment for brightness–color fusion

The paper motivates Bi-SAB by identifying two failure modes that arise under extreme low-light conditions when illumination and color are enhanced separately. The first is **Brightness–Color Misalignment**: a luminance-only branch may over-brighten regions without simultaneously correcting color statistics, producing hue shifts or patchy color reproduction, while a chrominance branch that depends on deeper features may lose edge fidelity when its color adjustments are applied after brightening. The second is **Artifact Amplification**: if noise is amplified in the brightness path and then recolored by chrominance corrections, ringing, halos, and color speckles become more visible [2507.18594].

Bi-SAB is designed specifically to address those two modes. The paper describes three design principles. **Cross-Attentive Alignment** allows the brightness feature to query the color feature through a lightweight cross-attention mechanism so that color statistics are drawn from the appropriate spatial locations. **Spectral Enhancement** uses a Spectral Alignment Enhancer (SAE) to adaptively fuse corrected feature spectra in order to suppress noise and reinforce weak edges. **Edge-Guided Refinement** adds a Scharr Edge Enhancement (SEE) path to preserve and sharpen fine structural detail. Together, these principles are intended to make the fusion both luminance-coherent and chrominance-faithful. A plausible implication is that Bi-SAB operationalizes “spectral alignment” not as a frequency-domain transform in isolation, but as a coordinated feature-space alignment between brightness-related and color-related representations.

## 3. Internal structure and data flow

Bi-SAB comprises four sub-modules arranged after initial query–key–value projection. The projections are
$$
F_Q = \mathrm{Conv}_{3 \times 3}(F_1), \qquad
F_K = \mathrm{Conv}_{3 \times 3}(F_1'), \qquad
F_V = \mathrm{Conv}_{3 \times 3}(F_1'),
$$
with each convolution using $C$ input and $C$ output channels [2507.18594].

| Component | Core operation | Declared purpose |
|---|---|---|
| Cross-Attention (CA) | Brightness queries color through lightweight cross-attention | Spatially aligned color statistics |
| Feature Difference Adjustment (FDA) | Combines attended feature, key, value, and query with fixed $\lambda$ | Corrective fusion before spectral enhancement |
| Scharr Edge Enhancement (SEE) | Fixed Scharr gradients plus convolutional restoration | Preserve and sharpen structural detail |
| Spectral Alignment Enhancer (SAE) | Layer-normalized, split-stream depth-wise processing and multiplicative fusion | Suppress noise and reinforce weak edges |

The **Cross-Attention (CA)** module produces
$$
F_{\text{attn}} = \mathrm{CA}(F_Q, F_K) \in \mathbb{R}^{B \times C \times H \times W}.
$$
According to the paper, CA uses $1 \times 1$ and then $3 \times 3$ depth-wise convolutions to generate $Q$, $K$, and $V$, splits channels into $n_h$ heads, performs per-head $L_2$ normalization, computes softmaxed cross-covariance attention, and re-projects the result to $C$ channels. This makes the color/context feature $F_1'$ the source of both key and value information while the low-level detail feature $F_1$ provides the query signal.

The **Feature Difference Adjustment (FDA)** stage then computes
$$
\mathrm{Att}_{\text{out}} = F_{\text{attn}} \odot F_K,
$$
$$
F_{\text{DA}} = \lambda \cdot \mathrm{Att}_{\text{out}} + F_V, \qquad \lambda = 0.2,
$$
$$
\mathrm{DA}_{\text{out}} = F_{\text{attn}} \odot F_Q + F_{\text{DA}}.
$$
The fixed coefficient $\lambda = 0.2$ constrains the scale of the attention-modulated correction.

The **Scharr Edge Enhancement (SEE)** branch computes $F_{\text{SEE}} = \mathrm{SEE}(F_1)$, where fixed Scharr kernels $W_x$ and $W_y$ extract gradients, the gradients are fused, and the result is restored through a $1 \times 1 \rightarrow 3 \times 3$ convolutional path with SiLU. In parallel, the **Spectral Alignment Enhancer (SAE)** computes
$$
F_{\text{SAE}} = \mathrm{SAE}(\mathrm{LN}(\mathrm{DA}_{\text{out}})),
$$
where the normalized feature is expanded by $\mathrm{Conv}_{1 \times 1}$, split into two halves, each half is processed by $\mathrm{DepthConv}_{3 \times 3} + \tanh +$ residual, the two streams are multiplied element-wise, and a final $\mathrm{Conv}_{1 \times 1}$ reduces the channels back to $C$. The final output is
$$
F_{\text{out}} = F_{\text{SAE}} + F_{\text{SEE}}.
$$

This decomposition makes clear that Bi-SAB is not a purely attentional decoder block. Its output is explicitly the sum of a spectrum-aligned branch and an edge-preserving branch, which is consistent with the paper’s emphasis on simultaneously maintaining color naturalness and edge continuity.

## 4. Exact mathematical formulation

The paper presents the complete Bi-SAB formulation for $F_1, F_1' \in \mathbb{R}^{B \times C \times H \times W}$ as follows [2507.18594].

For the projection stage,
$$
F_Q = W_Q * F_1, \qquad
F_K = W_K * F_1', \qquad
F_V = W_V * F_1',
$$
where each $*$ denotes a $3 \times 3$ convolution.

For cross-attention, the channels are split into heads:
$$
Q_h \in \mathbb{R}^{n_h \times c_h \times (H \cdot W)}, \qquad c_h = C / n_h,
$$
with analogous definitions for $K_h$ and $V_h$. Per-head normalization is
$$
\hat Q_h = Q_h / \|Q_h\|_2, \qquad
\hat K_h = K_h / \|K_h\|_2.
$$
The cross-covariance attention matrix is
$$
A_h = \mathrm{softmax}\left( \frac{\hat Q_h \cdot \hat K_h^{\top}}{\tau} \right) \in \mathbb{R}^{n_h \times c_h \times c_h},
$$
and the aggregated values are
$$
O_h = A_h \cdot V_h \in \mathbb{R}^{n_h \times c_h \times (H \cdot W)}.
$$
After rearrangement and projection,
$$
F_{\text{attn}} = W_O \cdot \mathrm{rearrange}(\{O_h\}).
$$

For Feature Difference Adjustment,
$$
\mathrm{Att}_{\text{out}} = F_{\text{attn}} \odot F_K,
$$
$$
F_{\text{DA}} = \lambda \cdot \mathrm{Att}_{\text{out}} + F_V,
$$
$$
\mathrm{DA}_{\text{out}} = F_{\text{attn}} \odot F_Q + F_{\text{DA}}.
$$

For Scharr Edge Enhancement,
$$
E_x = \mathrm{Conv}_{3 \times 3}(F_1; W_x), \qquad
E_y = \mathrm{Conv}_{3 \times 3}(F_1; W_y),
$$
$$
E_{\text{scharr}} = \|E_x\|_1 + \|E_y\|_1,
$$
$$
E_{\text{fuse}} = F_1 + \mathrm{BN}(E_{\text{scharr}}),
$$
$$
F_{\text{SEE}} = \mathrm{Conv}_{3 \times 3}(\mathrm{SiLU}(\mathrm{Conv}_{1 \times 1}(E_{\text{fuse}}))).
$$

For the Spectral Alignment Enhancer,
$$
[F_1', F_2'] = \mathrm{Split}(\mathrm{DepthConv}_{3 \times 3}(\mathrm{Conv}_{1 \times 1}(\mathrm{LN}(\mathrm{DA}_{\text{out}})))),
$$
$$
F_1'' = F_1' + \tanh(\mathrm{DepthConv}_{3 \times 3}(F_1')),
$$
$$
F_2'' = F_2' + \tanh(\mathrm{DepthConv}_{3 \times 3}(F_2')),
$$
$$
F_{\text{SAE}} = \mathrm{Conv}_{1 \times 1}(F_1'' \odot F_2'').
$$

The block concludes with
$$
F_{\text{out}} = F_{\text{SAE}} + F_{\text{SEE}}.
$$

Formally, this structure couples channel-wise cross-covariance attention with multiplicative spectral fusion and explicit gradient-based structural injection. This suggests that the block’s behavior is governed by two complementary priors: cross-feature correspondence across luminance and chrominance pathways, and local edge stability under restoration.

## 5. Optimization under MS$^2$-Loss and implementation choices

The entire DRWKV network, including Bi-SAB, is trained end-to-end using the multi-term MS$^2$-Loss, which enforces five objectives: pixel-wise reconstruction $(L_{\text{recon}})$, edge sparsity $(L_{\text{sparse}})$, illumination smoothness $(L_{\text{smooth}})$, artifact suppression $(L_{\text{artifact}})$, and parameter regularization $(L_{\text{reg}})$ [2507.18594].

The loss terms are defined as
$$
L_{\text{recon}} = \|I - \hat I\|_1,
$$
$$
L_{\text{sparse}} = \|E\|_1,
$$
$$
L_{\text{smooth}} = |\nabla L| \exp(-\lambda |\nabla I|),
$$
$$
L_{\text{artifact}} = \|S\|_1 + \delta \, TV(S),
$$
$$
L_{\text{reg}} = \alpha^2 + \beta^2 + \gamma^2.
$$
The total loss is
$$
\mathcal{L}_{\mathrm{MS^2}} =
\lambda_1 L_{\text{recon}} +
\lambda_2 L_{\text{sparse}} +
\lambda_3 L_{\text{smooth}} +
\lambda_4 L_{\text{artifact}} +
\lambda_5 L_{\text{reg}}.
$$

The paper attributes distinct roles to Bi-SAB within this loss regime. Its cross-attention and spectral enhancer actively reduce $L_{\text{recon}}$ by improving alignment of brightness and color, and reduce $L_{\text{artifact}}$ by suppressing noise–color misregistration, while the SEE branch supports $L_{\text{sparse}}$ by preserving true edges. This suggests that Bi-SAB is supervised indirectly through a coupled restoration objective rather than by an isolated color-alignment criterion.

Implementation details are specified with unusual precision. At each Bi-SAB resolution level, the channel count is $C = 16$. All convolutions use kernel sizes $1 \times 1$ or $3 \times 3$, and the depth-wise convolutions in CA and SAE have one group per channel. Cross-attention uses $n_h = 4$ heads with learnable temperature $\tau$ initialized to $1 / \sqrt{c_h}$, and the FDA weight $\lambda = 0.2$ is fixed. DRWKV is trained with Adam using $\beta_1 = 0.9$ and $\beta_2 = 0.999$, initial learning rate $2 \times 10^{-4}$, cosine annealing to $1 \times 10^{-6}$ over 500 epochs, and batch size tuned to fill an A800-80 GB GPU. The loss weights are $\lambda_1 = 1$, $\lambda_2 = 0.01$, $\lambda_3 = 0.1$, $\lambda_4 = 0.05$, and $\lambda_5 = 10^{-4}$. The input crop size is $128 \times 128$, and up/down-sampling in other DRWKV stages is performed via wavelet transforms.

## 6. Quantitative contribution, visual effects, and interpretation

On the LOLv2-Real test set, the paper reports a progressive ablation isolating Bi-SAB’s effect. A baseline with ES-RWKV + SIR and no Bi-SAB obtains $\mathrm{PSNR} = 22.67$ dB and $\mathrm{SSIM} = 0.748$. Adding Bi-SAB while still using SSIM + $L_1$ loss raises performance to $\mathrm{PSNR} = 23.47$ dB and $\mathrm{SSIM} = 0.776$, corresponding to gains of $+0.80$ dB and $+0.028$, respectively. Replacing the $L_1 + \mathrm{SSIM}$ loss with MS$^2$-Loss further increases performance to $\mathrm{PSNR} = 24.12$ dB and $\mathrm{SSIM} = 0.832$ [2507.18594].

The same ablation is used to support an efficiency claim: spectral alignment alone yields a substantial improvement in both distortion and perceptual metrics while halving the model’s GFLOPS relative to a naïve UNet-style integration. Within the scope of the paper, this identifies Bi-SAB as a decoder mechanism intended to improve fidelity without disproportionate computational growth.

The paper also attributes several qualitative effects to Bi-SAB’s aligned outputs: elimination of color bleeding and hue shifts around bright edges, cleaner shadow regions without colored noise speckles, and crisp edge continuity consistent with human perception. It further states that these effects are validated by a user-study MOS increase over the nearest competitor. Because no separate MOS value is provided in the supplied data, the result is best interpreted qualitatively rather than as a quantified benchmark comparison.

Several clarifications follow from the ablation. First, Bi-SAB should not be understood as contributing only chromatic correction; the reported gains include both PSNR and SSIM improvement, indicating impact on distortion-oriented and structural quality measures. Second, its effect is not reducible to attention alone, because the block’s declared contribution depends on the combination of CA, SAE, and SEE. Third, its role is specifically decoder-centric: it aligns skip-connected low-level and high-level features during reconstruction, rather than replacing the paper’s separate mechanisms for illumination–edge decoupling or spatial continuity modeling.

Source: https://www.emergentmind.com/topics/bilateral-spectrum-aligner-bi-sab