Defocus-Aware Aggregation Module

• Defocus-Aware Aggregation Module is a method that adaptively fuses per-pixel blur maps with image features to effectively handle spatially variant defocus blur.
• It employs techniques like channel-attention fusion and mask-guided multi-branch operations to process distinct blur levels based on localized blur estimations.
• Integrating supervised blur map predictions with unsupervised geometric consistency has been shown to improve PSNR and perceptual quality in advanced deblurring tasks.

A Defocus-Aware Aggregation Module is a feature fusion mechanism that adaptively combines information guided by estimated spatially-varying defocus blur maps. It is central to modern deep learning architectures for defocus deblurring, enabling networks to treat spatially variant blur within images by explicitly leveraging per-pixel or per-region blur predictions. Such modules are instrumental in exploiting the non-uniform, non-stationary blur patterns present in real-world defocus images, and form the computational core of several state-of-the-art frameworks for both single- and dual-pixel input modalities (Ren et al., 26 Sep 2024, Liang et al., 2021, Vo, 2021).

1. Design Principles and Technical Motivation

Defocus blur in images is rarely globally stationary; distinct spatial regions often manifest variable degrees and shapes of blur. Conventional CNN-based deblurring models with homogenous convolutional processing are inherently ill-suited to this scenario, as they apply the same processing kernel everywhere, thus failing to respect local blur statistics (Liang et al., 2021). Defocus-aware aggregation addresses this by conditioning the processing pipeline on either explicit blur maps or implicitly estimated per-pixel blur characteristics, thus enabling spatially adapted restoration.

The core principles underlying these modules are:

• Local adaptation: Fusion rules or aggregation routines are controlled by locally inferred or predicted blur metrics (e.g., defocus maps, circle of confusion estimates).
• Multi-path or attention-based fusion: Parallel expert sub-networks or channel-attention mechanisms allow selective emphasis depending on local blur magnitude.
• End-to-end learnability: All fusion and estimation stages are trained jointly with the primary deblurring objective, often aided by auxiliary geometric or perceptual losses.

2. Architectures and Module Implementations

Channel-Attention Fusion in Reblurring-Guided Deblurring

In the reblurring-guided JDRL architecture (Ren et al., 26 Sep 2024), the Defocus-Aware Aggregation Module is realized as a lightweight channel-attention block that injects the estimated blur map $\hat B$ at every encoder and decoder stage:

• $\hat B\in\mathbb{R}^{H\times W\times M}$ (defocus map estimator output) is transformed via two 1×1 convolutions and ReLU into a gating tensor.
• A channelwise sigmoid produces an attention map $A\in[0,1]^{H\times W\times C}$.
• Aggregation of feature $F$ occurs via residual attention:

$F_\mathrm{fused} = F \odot A + F$

where $\odot$ is broadcasted multiplication. This design enables per-channel, per-location modulation of feature representations as a function of local defocus, with all channel-attention weights learned end-to-end by backpropagation.

Mask-Guided Multi-branch Fusion in BaMBNet

BaMBNet (Liang et al., 2021) proposes a discrete aggregation approach:

• A circle-of-confusion (COC) map is inferred unsupervisedly for each pixel.
• Pixels are assigned to $M$ discrete blur levels through meta-learned thresholds, producing binary masks $D_i(x)$ per level.
• A multi-branch network processes the encoded features $F_\mathrm{enc}$ in parallel branches $\phi_i$ of varying capacity (depth), each tuned to a specific blur severity.
• Aggregation occurs by mask-weighted summation:

$$F_\mathrm{comb} = F_\mathrm{enc} + \sum_{i=1}^M D_i \odot \phi_i(F_\mathrm{enc})$$

This partitioned fusion ensures that fine details in sharp areas are preserved by shallow branches, while information loss in severely blurred regions is recovered by deeper, more expressive sub-networks.

Dual-Source Defocus-Aware Aggregation in ATTSF

In ATTSF (Vo, 2021), defocus-aware aggregation is instituted at the bottleneck by joint fusion of features from dual-pixel views:

• Each view is processed by a dual-attention encoder (channel and position).
• At the bottleneck, features are aggregated via parallel triple-local and global-local modules, whose outputs are concatenated and fused by a 1×1 convolution.
• The aggregation adaptively weighs local and non-local context, leveraging spatial and inter-view information to maximize restoration quality.

3. Blur Map Prediction and Supervision Mechanisms

All recent instantiations rely on a mechanism to predict or estimate a defocus blur map:

• In JDRL, a convolutional sub-network $\mathcal{F}_B$ predicts $\hat B$ from the input blurry image. Pseudo-supervision for this map is provided by a differentiably learned reblurring operator, which reconstructs local isotropic kernels from learned seeds and spatial weights (Ren et al., 26 Sep 2024).
• In BaMBNet, a dedicated encoder-decoder network unsupervisedly estimates the COC map by enforcing geometric consistency between left/right DP image patches, leveraging PSF flip relations under the thin lens model (Liang et al., 2021). These maps are not only used for aggregation, but also as supervisory signals through dedicated loss terms (e.g., pseudo-map loss, geometric loss).

4. Training Objectives and Loss Formulations

Defocus-Aware Aggregation Modules are trained under comprehensive loss objectives:

• In JDRL, three principal terms are used: deblurring loss $\mathcal{L}_D$ (using robust $\ell_1$+flow compensation), reblurring loss $\mathcal{L}_R$, and pseudo-map loss $\mathcal{L}_P$; the total objective is

$$\mathcal{L} = \mathcal{L}_D + \alpha \mathcal{L}_R + \beta \mathcal{L}_P$$

with $\alpha=\beta=0.5$ (Ren et al., 26 Sep 2024).

• BaMBNet adopts a two-stage training regime: initially the COC estimator is trained with geometric and smoothness losses, followed by end-to-end optimization of the deblurring network with $L_1$ pixelwise loss (Liang et al., 2021).

Empirical ablations demonstrate that removal or simplification of the aggregation module leads to statistically significant drops in restoration quality: in BaMBNet, omitting the binary mask gating lowers PSNR from 26.40 dB to 26.22 dB and increases LPIPS (Liang et al., 2021); in ATTSF, eliminating component branches of the aggregation causes PSNR drops up to 0.4 dB (Vo, 2021).

5. Comparative Table: Aggregation Strategies

Paper	Aggregation Mechanism	Blur Map/Mask Source
JDRL (Ren et al., 26 Sep 2024)	Channel-attention fusion via sigmoid-gated 1×1 conv	Single-image learned map $\hat B$
BaMBNet (Liang et al., 2021)	Mask-gated multi-branch fusion	Unsupervised DP COC map, meta-learned masks
ATTSF (Vo, 2021)	Bottleneck fusion of triple-local and non-local blocks	Dual-pixel dual-attention, no explicit blur map

While implementations vary, all leverage learned, spatially variant information to direct feature integration, and all demonstrate improved performance over uniform fusion baselines.

6. Empirical Results and Quantitative Impact

Defocus-Aware Aggregation Modules underpin performance gains across several competitive benchmarks. For instance, BaMBNet achieves higher PSNR and lower perceptual error on DPD-Blur compared to single-branch models. ATTSF reports a PSNR of 25.98 dB and SSIM 81.15% on NTIRE 2021, outperforming prior work (Vo, 2021).

Critically, the largest quality gains appear in regions with severe or spatially complex blur, validating the central hypothesis that spatially aware fusion is a prerequisite for state-of-the-art defocus deblurring (Liang et al., 2021, Ren et al., 26 Sep 2024).

7. Significance, Limitations, and Future Directions

The Defocus-Aware Aggregation Module formalizes the concept of adaptively fusing restoration hypotheses or feature paths under spatially varying blur, and is now foundational in the field. A key limitation remains the reliance on accurately predicted blur maps or masks, whose quality strongly affects performance. A plausible implication is that future work may focus on more robust or semantically informed blur estimation, scaling aggregation strategies to more challenging settings (e.g., out-of-focus scenes in the wild), and integrating prior geometric knowledge directly into fusion rules. Broader generalization beyond defocus deblurring (e.g., spatially variant artifact removal) is also suggested by the architecture's modularity (Ren et al., 26 Sep 2024, Liang et al., 2021, Vo, 2021).