Degradation-Aware Channel Attention Module (DCAM)

• DCAM is a neural module that adaptively modulates feature channels by explicitly modeling image degradations such as haze, rain, and snow using semantic priors.
• It leverages architectures like DHFormer and MdaIF, combining convolutional, transformer-based, and vision-language techniques to enhance image restoration and multi-modal fusion.
• Empirical evaluations show that integrating DCAM improves metrics like PSNR and SSIM by refining feature representations and mitigating degradation effects.

The Degradation-Aware Channel Attention Module (DCAM) is a class of neural network modules designed to reweight feature channels based on explicit modeling of image degradation. DCAMs enable neural networks to integrate scene- and weather-aware priors into channel attention, facilitating adaptive processing of inputs suffering from diverse degradation phenomena. They have been deployed both as standalone components and as integrated attention heads within larger architectures for image restoration and multi-modal fusion, notably in recent vision transformer-based and vision-language-driven frameworks (Wasi et al., 2023, Li et al., 16 Nov 2025).

1. Conceptual Basis and Motivation

DCAM modules operate on the principle that the significance of each feature channel in a neural representation should be adaptively modulated according to the type, degree, and context of the image degradation present. Traditional channel attention mechanisms lack direct awareness of physical degradations (e.g., haze, rain, snow) or the latent cues available from high-level scene or environmental information. DCAMs overcome this by injecting explicit degradation or scene priors into the channel weighting process.

The motivation for DCAM formulations in both dehazing and fusion pipelines is twofold: (1) to disentangle and emphasize channel features critical for restoring image fidelity under adverse input conditions, and (2) to ensure that downstream modules (e.g., mixture-of-experts or image reconstruction heads) operate on features that have already been adaptively recentered toward degradation-relevant semantics (Wasi et al., 2023, Li et al., 16 Nov 2025).

2. Architectural Realizations

Two representative DCAM implementations are exemplified by DHFormer for image dehazing and by MdaIF for multi-degradation fusion.

DHFormer-Style DCAM:

• Inputs originate from residual-learned degradation maps, constructed through residual learning based on the atmospheric scattering model.
• The input feature map passes through parallel convolutional layers (e.g., 3×3, 5×5 kernels), whose concatenated outputs define the DCAM's feature input.
• A transformer-based encoder stack, composed of patch embedding, local and global self-attention, and MLP sub-blocks, produces channel and spatial attention maps.
• Channel attention is extracted by pooling transformer output tokens, optionally processed through an MLP or sigmoid activation, resulting in per-channel weights.
• Spatial attention is computed via CBAM-like operations (channel pooling plus convolutions), yielding a spatial map fused with the channel-refined features.
• The two attention outputs are fused, projected back to image space, and the processed residual subtracted from the input according to the physical model (Wasi et al., 2023).

MdaIF-Style DCAM:

• Receives as input both multi-modal feature maps (concatenated visible/infrared encodings) and a semantic prior extracted from a frozen vision-LLM (e.g., BLIP-2 with OPT 2.7B).
• The semantic prior is average pooled, projected by a small MLP and LayerNorm, and then undergoes a sigmoid nonlinearity to form a vector of degradation-prototype activations.
• A trainable set of K “degradation prototypes” (learned vectors in feature space) is weighted and summed (via semantic-prior activations), and after a sigmoid, yields channel-wise attention coefficients.
• Input features, layer-normalized, are channel-wise rescaled by these coefficients and fused with a residual connection.
• This process channels semantic knowledge about scene and weather conditions directly into the feature modulation, prior to downstream mixture-of-experts fusion (Li et al., 16 Nov 2025).

3. Mathematical Formulations

The core mathematical techniques in DCAMs are as follows:

Prototype-Based Channel Modulation (MdaIF):

$$s_K = \sigma\left(\mathcal{N}_{\text{layer}}\left(\Phi_{m}^{II}(\text{AvgPool}(S_{\text{prior}}))\right)\right), \quad s_K\in\mathbb{R}^K$$

$$W_{\text{proto}} = \begin{bmatrix} k_1^\top \ k_2^\top \ \vdots \ k_K^\top \end{bmatrix} \in\mathbb{R}^{K \times C}$$

$$w = \sigma(s_K W_{\text{proto}}), \quad w \in \mathbb{R}^C$$

$$F_{\text{dcam}} = \mathcal{N}_{\text{layer}}(F_{\text{in}}) \odot w + F_{\text{in}}$$

where $S_{\text{prior}}$ denotes the semantic prior, $k_i$ are prototype vectors, and $\odot$ is channel-wise multiplication.

Transformer-Based Channel and Spatial Attention (DHFormer):

$$z = \frac{1}{N} \sum_{i=1}^{N} T_N[i, \cdot], \quad M_c = \sigma(W_2\,\text{ReLU}(W_1 z)), \quad M_c \in \mathbb{R}^D$$

$$A = \text{AvgPool}_{\text{channel}}(F_c),\qquad M = \text{MaxPool}_{\text{channel}}(F_c)$$

$$M_s = \sigma\left(\text{Conv}_{7\times7}([A, M])\right)$$

$F' = M_s \odot F_c$

(Wasi et al., 2023).

4. Integration with Upstream Semantic Priors

A distinguishing feature of recent DCAMs is the explicit use of semantic priors for channel attention, particularly in fusion tasks under unknown degradations. For example, in MdaIF, the semantic prior is sourced from a vision-LLM (BLIP-2), summarizing global scene context and specific degradation cues detected by language-driven inference. This vector of semantic tokens is projected and used to weigh degradation prototypes that map directly onto feature channels. The integration with such high-level priors enables the system to adapt attention dynamically for arbitrary combinations or transitions between weather phenomena, as well as diverse scene content (Li et al., 16 Nov 2025).

5. Downstream Effects, Performance, and Ablation

The effect of DCAM modules can be isolated via ablation studies. For instance, in the MSRS fusion benchmark, introducing DCAM alone (without the downstream mixture-of-experts mechanism) results in quantifiable gains in PSNR (~0.74 dB), SSIM (~0.05 improvement), and reductions in NABF, relative to a backbone without DCAM. When DCAM is combined with a mixture-of-experts, these improvements compound further, evidencing DCAM's role in enhancing both fidelity and structural preservation in the presence of degradation (Li et al., 16 Nov 2025).

System Config	PSNR	SSIM	NABF	MI
Baseline (no DCAM)	16.265	1.144	0.0142	2.006
+ DCAM only	17.009	1.194	0.0135	2.101
+ DMoE only	15.350	1.083	0.0148	2.159
DCAM + DMoE (Full)	17.977	1.269	0.0131	2.390

6. Training Objectives and Loss Formulations

DCAM modules are generally trained indirectly under the overall task objective, without stand-alone loss terms. For example, in MdaIF, the fusion loss $L_{\text{fusion}}$ is composed of an intensity preservation loss $L_{\text{inte}}$ and a chrominance alignment loss $L_{\text{color}}$ (in YCbCr space). In DHFormer, only the residual $\ell_2$ loss on the predicted atmospheric term is enforced. Optional losses, such as perceptual (VGG-based) metrics, can be added to the pipeline but are not part of the original module definitions (Wasi et al., 2023, Li et al., 16 Nov 2025).

7. Design Considerations and Hyperparameters

Key implementation factors for DCAM include:

• The number of degradation prototypes $K$, which is often set slightly above the number of observed degradation types for representation flexibility.
• Dimensionality $C$ of feature channels, inherited from the backbone architecture.
• Orthogonal initialization for prototype vectors $k_i$ to maximize diversity.
• In transformer-based variants, the balance between local and global context in self-attention layers is crucial for capturing depth or non-local degradation cues.
• The presence and architecture of the module responsible for semantic prior extraction (e.g., which vision-LLM, the dimensionality of the output, self-attention within the semantic token sequence).

A plausible implication is that increased $K$ can yield more nuanced channel reweighting at additional computational cost, and that richer semantic priors may make DCAMs more robust under unseen or mixed degradations.

---

DCAMs provide an explicit interface between high-level degradation understanding and low-level channel feature modulation, making them a key technology for robust image restoration, fusion, and potentially other tasks as multi-modal and degradation-aware vision systems continue to advance (Wasi et al., 2023, Li et al., 16 Nov 2025).