---
title: 'M2AENet: Mask & Motion Deblurring'
url: https://www.emergentmind.com/topics/m2aenet
type: topic
---

# M2AENet: Mask & Motion Deblurring

M²AENet, short for **Mask- and Motion-Aware Efficient Network**, is a CNN-based architecture for **local motion deblurring** introduced in "Motion-Aware Adaptive Pixel Pruning for Efficient Local Motion Deblurring" [2507.07708]. It addresses the setting in which only part of an image is blurred because dynamic objects move relative to a static imaging system during exposure, producing **spatially varying** blur while other regions remain sharp. The method combines two coupled mechanisms: a **mask-aware adaptive pixel pruning** scheme that allocates computation only to blurred pixels, and a **motion-aware deformable restoration** scheme that uses estimated intra-frame motion trajectories to guide deblurring. The resulting system is built on a U-shaped **NAFNet** backbone, but modifies selected blocks into **M²AS blocks** that jointly exploit blur localization and motion structure [2507.07708].

## 1. Problem setting and design objective

M²AENet is formulated for **local motion blur**, not for uniformly blurred images. In this regime, blur is spatially varying: different regions can exhibit different blur strengths and directions, and many pixels are already sharp. This differs from **global motion blur**, where blur is approximately uniform across the whole image, for example under camera shake. The central premise of M²AENet is that local blur restoration should not process every pixel equally, because this wastes computation and can distort regions that do not require correction [2507.07708].

The method is motivated by three limitations identified in prior work. First, **global deblurring models**, including CNN- and Transformer-based architectures designed for global blur, do not explicitly focus on blurred regions and may distort sharp areas while spending substantial computation on them. Second, **patch- or window-based local deblurring methods**, including LBAG, LMD‑ViT, and AdaRevD, allocate resources only coarsely, because a patch or window may mix blurred and sharp pixels; this still leaves many sharp pixels unnecessarily processed. Third, existing local blur methods largely use blur masks for gating or pruning but do not explicitly model the **physical cause** of blur, namely the motion trajectory during exposure. M²AENet is designed to move from patch-level to **pixel-level** allocation while adding explicit **intra-frame motion trajectory** modeling [2507.07708].

This design leads to a two-part objective. One part is **efficiency**: only blurred pixels should go through expensive computation. The other part is **restoration fidelity**: the receptive field used for deblurring should follow estimated motion trajectories rather than relying solely on generic spatial aggregation. A plausible implication is that the architecture is intended not merely as a pruning scheme, but as a joint blur-localization and motion-conditioned reconstruction system.

## 2. Backbone organization and M²AS blocks

M²AENet adopts a **U-Net-like CNN built on NAFNet**. The network consists of an in-projection, **4 encoder stages**, a bottleneck, and **4 decoder stages**, with skip connections between encoder and decoder stages. The in-projection is a \(3\times 3\) convolution mapping RGB input to **base channels 32**. The stage configuration is reported as **[1, 1, 1, 28]** blocks for the encoder, **1** block for the bottleneck, and **[1, 1, 1, 1]** for the decoder; channels double after each downsampling. For efficiency and training stability, the **mask predictor** and **intra-motion analyzer** are placed mainly at the **fourth encoder stage**, i.e., the highest semantic and lowest spatial-resolution stage [2507.07708].

At selected scales, original NAFNet blocks are replaced by **M²AS blocks** (**Mask- and Motion-Aware Simple blocks**). The first half of the block performs **mask-aware convolution**:

$$
\begin{aligned}
\mathbf{f}_{1} &= \mathtt{LN}(\mathbf{f}_{in}), \\
\mathbf{f}_2 &= \mathtt{Conv}(\mathbf{f}_{1})\odot \mathbf{m} + \mathtt{Rep}(\mathbf{f}_{1})\odot (\mathbf{1} - \mathbf{m}), \\
\mathbf{f}_3 &= \mathtt{SCA}(\mathtt{SG}(\mathbf{f}_2)), \\
\mathbf{f}_{mid} &= \mathtt{Conv}_{1\times 1}(\mathbf{f}_3) + \mathbf{f}_{in},
\end{aligned}
$$

where \( \mathtt{LN} \) is layer normalization, \( \mathtt{Conv} \) is one \(1\times1\) convolution followed by one \(3\times3\) depth-wise convolution, \( \mathbf{m} \) is the blur mask, \( \mathtt{Rep} \) denotes channel repetition, \( \mathtt{SG} \) is the Simple Gate from NAFNet, and \( \mathtt{SCA} \) is simplified channel attention. On **blurred pixels** (\(m=1\)), the block applies full convolutional processing; on **sharp pixels** (\(m=0\)), it uses a repeated identity-like path [2507.07708].

The second half performs **motion-aware deformable convolution**:

$$
\begin{aligned}
\mathbf{f}_{4} &= \mathtt{Conv}_{1\times 1}(\mathtt{LN}(\mathbf{f}_{mid})), \\
\mathbf{f}_5 &= \mathtt{SG}(\mathtt{DConv}(\mathbf{f}_{4}, \mathbf{d})), \\
\mathbf{f}_{out} &= \mathtt{Conv}_{1\times 1}(\mathbf{f}_5) + \mathbf{f}_{mid},
\end{aligned}
$$

where \( \mathtt{DConv} \) is a **motion-aware deformable convolution** and the offsets \( \mathbf{d} \) are not generic learned offsets but are supplied directly by the intra-motion analyzer. This preserves the residual and lightweight style of NAFNet while coupling it to explicit motion estimation [2507.07708].

The placement of mask-aware computation inside the block was also ablated. Three possible internal positions were considered, and the reported best tradeoff was to apply mask-aware convolution **only at position (1)**. Applying it to more positions reduced FLOPs further but harmed restoration quality. This suggests that the architecture treats selective sparsification as beneficial only when applied at the most consequential convolution rather than indiscriminately throughout the block.

## 3. Blur-mask prediction and adaptive pixel pruning

The **mask predictor** operates on a feature map \( \mathbf{f}_{in} \in \mathbb{R}^{H \times W \times C} \). The feature map is flattened into \(D=H\times W\) per-pixel tokens, transformed by an MLP, pooled into a global feature by channel-wise averaging, concatenated back to each pixel representation, and passed through another MLP with softmax to yield per-pixel blur probabilities:

$$
\mathbf{p} = \mathtt{Softmax}\big(\mathtt{MLP}(\mathtt{Concat}(\mathbf{f}'_{in}, \mathtt{Avg}(\mathbf{f}'_{in})))\big), \quad \mathbf{p} \in \mathbb{R}^{D \times 2}.
$$

Here \( \mathbf{p}_{[*,0]} \) denotes the probability that a pixel is blurred. During training, a differentiable binary mask is produced with **Gumbel-Softmax**,

$$
\mathbf{m} = {\mathtt{Gumbel\text{-}Softmax}(\mathbf{p})}_{[*,0]}, \quad \mathbf{m}\in \mathbb{R}^{D \times 1},
$$

whereas at test time the network uses thresholding,

$$
\mathbf{m} =
\left\{
\begin{array}{ll}
1, & \text{if } \mathbf{p}_{[*,0]} > \epsilon, \\
0, & \text{otherwise},
\end{array}
\right.
\quad \epsilon = 0.5.
$$

This makes the blur mask both trainable and operational as an inference-time pruning signal [2507.07708].

The supervision strategy depends on the dataset. On **ReLoBlur**, the method uses refined blur masks from Li et al. (LMD‑ViT) as ground truth \( \mathbf{m}^{gt} \). On **GoPro**, all pixels are treated as blurred and the mask is set to all ones. The mask loss is

$$
\mathcal{L}_{mask} = \sum_{i=1}^S \mathtt{Cross\text{-}Entropy}(\mathbf{m}_i, \mathbf{m}^{gt}_i),
$$

where \(S\) is the number of scales with mask predictors. The same masks are also used during training to suppress processing in sharp regions and to reduce ambiguity in motion-offset estimation [2507.07708].

A distinctive component of M²AENet is **structural reparameterization** for inference-time pruning. A \(3\times 3\) convolution with weight tensor \( \mathbf{W} \in \mathbb{R}^{C_{out} \times C_{in} \times 3 \times 3} \) is reshaped to \( \mathbf{W}' \in \mathbb{R}^{C_{out} \times (C_{in}\times 9) \times 1 \times 1} \), while the feature map is unfolded so that the original spatial neighborhood becomes channel dimension. The \(3\times3\) convolution is thereby converted into an equivalent \(1\times1\) convolution over unfolded features. This matters because \(1\times1\) convolutions are compatible with **per-pixel selection** [2507.07708].

At inference, the network collects the indices of blurred pixels from \( \mathbf{m} \); if there are \(Q\) such pixels, only those \(Q\) feature vectors are sampled, convolved, and scattered back into the full map, while sharp pixels bypass heavy computation. The computational reduction scales with \( \frac{Q}{H\times W} \). When blur occupies only a small fraction of the image, the resulting FLOPs savings can be substantial, particularly for high-resolution inputs.

## 4. Intra-frame motion analyzer and trajectory-guided restoration

The **intra-frame motion analyzer** is designed to encode blur **magnitude** and **direction**, which the mask alone cannot provide. It is implemented as a **single convolutional layer** applied to the feature map \( \mathbf{f}_{in} \), and predicts per-pixel displacements from the midpoint of exposure to the start and end of exposure:

$$
\mathbf{o}^{(t_0)} \in \mathbb{R}^{H \times W \times 2}, \qquad
\mathbf{o}^{(t_{N-1})} \in \mathbb{R}^{H \times W \times 2}.
$$

The exposure interval is discretized into \(N\) time instants \(t_0,\dots,t_{N-1}\), and the ground-truth sharp image is treated as corresponding approximately to the midpoint [2507.07708].

Intermediate displacements are not predicted independently; they are generated by **quadratic interpolation**:

$$
\begin{aligned}
\mathbf{o}^{(t_n)} &=
\frac{\mathbf{o}^{(t_0)}+\mathbf{o}^{(t_{N-1})}}{2}\left(\frac{2 n}{N-1}-1\right)^2 \\
&\quad +\frac{\mathbf{o}^{(t_{N-1})}-\mathbf{o}^{(t_0)}}{2}\left(\frac{2 n}{N-1}-1\right), \quad n=0,\ldots,N-1.
\end{aligned}
$$

At \( n = \frac{N-1}{2} \), the displacement is zero. The resulting set \( \{\mathbf{o}^{(t_n)}\}_{n=0}^{N-1} \) defines a per-pixel motion trajectory over exposure time. The authors adopt quadratic interpolation because it performed better than linear interpolation and was nearly as effective as predicting all displacements independently, while requiring less complexity [2507.07708].

These trajectories are then converted into the relative displacements required by **deformable convolution**. For a \(3\times3\) kernel, 9 offsets are needed per pixel. Instead of letting the model learn arbitrary offsets, M²AENet constructs offsets that trace the estimated motion path during exposure. Conceptually, the deformable kernel samples trajectory-aligned neighbors, i.e., pixels that physically contributed to the blur at a given location. This constrains the receptive field to be **trajectory-aware** rather than only feature-adaptive [2507.07708].

The motion field is supervised indirectly through a **reblur consistency** objective and spatial smoothness. This is important because blur is caused by the integration of moving content over time. The reported limitation is that reblur loss alone can misinterpret **flat regions** as moving, since low-texture static areas may satisfy the reblur objective; mask supervision is introduced partly to mitigate this ambiguity.

## 5. Optimization, benchmarks, and empirical behavior

The total training objective is

$$
\mathcal{L} = \mathcal{L}_{recon} + \lambda \mathcal{L}_{mask} + \mathcal{L}_{offset},
$$

with \( \lambda = 0.01 \), and \( \gamma = 0.1 \) inside \( \mathcal{L}_{recon} \). The reconstruction term is

$$
\mathcal{L}_{recon} = \mathtt{L}_{1}(\mathbf{y}, \mathbf{y}^{gt}) + \mathtt{SSIM}(\mathbf{y}, \mathbf{y}^{gt}) + \gamma \mathtt{FFT}(\mathbf{y}, \mathbf{y}^{gt}),
$$

while the offset term is

$$
\mathcal{L}_{offset} = \sum_{i=1}^{S}\left(\mathtt{Reblur}(\{\mathbf{o}^{(t_n)}_i\}_{n=0}^{N-1}, \mathbf{x}_i, \mathbf{y}^{gt}_i) + \alpha\sum_{n=0}^{N-1}\mathtt{TV}(\mathbf{o}^{(t_n)}_i)\right).
$$

The reblur component synthesizes a blurred image by forward-warping the sharp ground-truth image with each displacement field and summing the warped frames; the implementation uses softmax splatting. The coefficient \( \alpha \) is not numerically specified in the main body. Training uses **AdamW**, momentum **(0.9, 0.9)**, weight decay \(1\times10^{-3}\), an initial learning rate \(1\times10^{-3}\) decayed to \(1\times10^{-6}\) with **cosine annealing**, batch size **8**, patch size **256×256**, and **blur-aware patch cropping**. The framework is **PyTorch**, and training is reported on an **NVIDIA RTX A6000 GPU** [2507.07708].

Experiments are conducted on **ReLoBlur** and **GoPro**, with a **1:1 sampling mixture** used during training. ReLoBlur is a real local motion blur dataset with **2010** training images and **395** test images at resolution **2152 × 1436**. GoPro is the standard global dynamic scene blur benchmark synthesized from high-frame-rate videos. Evaluation uses **PSNR** and **SSIM** on GoPro, and on ReLoBlur both full-image PSNR/SSIM and blur-weighted metrics \( \text{PSNR}_{w} \) and \( \text{SSIM}_{w} \) [2507.07708].

On **ReLoBlur**, the reported results are: NAFNet **34.584 / 0.9244 / 28.633 / 0.8751**, LBAG **34.655 / 0.9257 / 29.320 / 0.8810**, Restormer **34.670 / 0.9228 / 29.436 / 0.8793**, LMD‑ViT **35.394 / 0.9280 / 30.240 / 0.8929**, and **M²AENet 35.647 / 0.9302 / 31.036 / 0.8997** for \( \text{PSNR} / \text{SSIM} / \text{PSNR}_{w} / \text{SSIM}_{w} \). On **GoPro**, the reported values are NAFNet **32.402 / 0.9338**, Restormer **32.452 / 0.9365**, LMD‑ViT **32.500 / 0.9312**, and **M²AENet 33.055 / 0.9406**. The paper therefore reports that M²AENet outperforms the listed baselines on both local and global blur benchmarks [2507.07708].

Efficiency is quantified against NAFNet and LMD‑ViT. NAFNet is reported at **17.11M** parameters, **0.764T** FLOPs, **5.0GB** GPU memory, and **0.569s/image**. LMD‑ViT is reported at **54.55M**, **1.485T**, **28.0GB**, and **1.284s/image**. M²AENet is reported at **17.44M**, **0.759T**, **5.8GB**, and **0.790s/image**. Relative to LMD‑ViT, the paper states a **49% reduction in FLOPs**, about **38.5% faster** inference, and GPU memory usage at about **21%** of LMD‑ViT’s consumption [2507.07708].

Ablations separate the contributions of the two central mechanisms. The baseline NAFNet obtains **34.584** PSNR and **28.633** \( \text{PSNR}_{w} \) at **0.764T** FLOPs. Adding **motion-aware DConv only** yields **35.091 / 29.591 / 0.995T**. Adding **mask-aware Conv only** yields **35.021 / 30.081 / 0.528T**. The **full M²AENet** yields **35.647 / 31.036 / 0.759T**. This suggests that mask-aware convolution primarily improves **local deblurring metrics** and efficiency, whereas motion-aware deformable convolution primarily improves **global reconstruction quality**; the best overall behavior arises when both are used together. Additional ablations show that **quadratic interpolation** is preferred over linear interpolation, and that using \(N_2=9\) time instants in the reblur loss is sufficient because gains saturate beyond that point [2507.07708].

## 6. Nomenclature, relation to earlier work, and limitations

M²AENet should be distinguished from **M2A** (**Motion Aware Attention**), a 2021 **temporal module for video action recognition** introduced in "M2A: Motion Aware Attention for Accurate Video Action Recognition" [2111.09976]. That earlier work is a plug-in motion-aware attention mechanism for 2D and 3D CNN backbones in video classification, whereas **M²AENet** is a 2025 local motion deblurring network whose name expands to **Mask- and Motion-Aware Efficient Network**. The two works share the phrase “motion aware,” but they address different tasks, use different architectures, and are not variations of a single model family [2111.09976].

Within the deblurring literature, M²AENet is positioned against three categories of prior methods: **global deblurring models**, which may distort already sharp regions in local-blur scenarios; **patch/window-based local methods**, which allocate computation only coarsely; and methods that use **blur masks without explicit motion modeling**. The architecture responds to those limitations by combining a mask predictor with a motion analyzer. The paper explicitly notes that these two components might appear redundant, but the ablation study indicates that both are needed for the reported accuracy-efficiency tradeoff [2507.07708].

The reported limitations are concentrated in motion supervision. Reblur loss can generate **spurious offsets in flat regions with low texture**, because such regions may satisfy the image formation constraint even without true motion. Mask supervision alleviates this but does not eliminate the ambiguity. Future directions suggested in the paper include a more discriminative **reblur loss formulation** and better **motion constraints** that reduce reliance on explicit mask supervision. The authors also report that a larger architecture increased memory to **21GB** and FLOPs to **2.1T** without meaningful performance gain, which motivated retention of the original NAFNet block configuration. The reference implementation is provided at `https://github.com/shangwei5/M2AENet` [2507.07708].

Source: https://www.emergentmind.com/topics/m2aenet