M2AENet: Mask & Motion Deblurring

• M²AENet is a CNN-based architecture for local motion deblurring that selectively processes blurred pixels using mask-aware adaptive pruning.
• It combines mask-aware pixel pruning with motion-aware deformable convolutions to efficiently allocate computation and preserve sharp image regions.
• Built on a U-Net-like NAFNet backbone, it demonstrates improved restoration fidelity and efficiency over traditional global deblurring models on benchmarks like ReLoBlur and GoPro.

M²AENet, short for Mask- and Motion-Aware Efficient Network, is a CNN-based architecture for local motion deblurring introduced in "Motion-Aware Adaptive Pixel Pruning for Efficient Local Motion Deblurring" (Shang et al., 10 Jul 2025). It addresses the setting in which only part of an image is blurred because dynamic objects move relative to a static imaging system during exposure, producing spatially varying blur while other regions remain sharp. The method combines two coupled mechanisms: a mask-aware adaptive pixel pruning scheme that allocates computation only to blurred pixels, and a motion-aware deformable restoration scheme that uses estimated intra-frame motion trajectories to guide deblurring. The resulting system is built on a U-shaped NAFNet backbone, but modifies selected blocks into M²AS blocks that jointly exploit blur localization and motion structure (Shang et al., 10 Jul 2025).

1. Problem setting and design objective

M²AENet is formulated for local motion blur, not for uniformly blurred images. In this regime, blur is spatially varying: different regions can exhibit different blur strengths and directions, and many pixels are already sharp. This differs from global motion blur, where blur is approximately uniform across the whole image, for example under camera shake. The central premise of M²AENet is that local blur restoration should not process every pixel equally, because this wastes computation and can distort regions that do not require correction (Shang et al., 10 Jul 2025).

The method is motivated by three limitations identified in prior work. First, global deblurring models, including CNN- and Transformer-based architectures designed for global blur, do not explicitly focus on blurred regions and may distort sharp areas while spending substantial computation on them. Second, patch- or window-based local deblurring methods, including LBAG, LMD‑ViT, and AdaRevD, allocate resources only coarsely, because a patch or window may mix blurred and sharp pixels; this still leaves many sharp pixels unnecessarily processed. Third, existing local blur methods largely use blur masks for gating or pruning but do not explicitly model the physical cause of blur, namely the motion trajectory during exposure. M²AENet is designed to move from patch-level to pixel-level allocation while adding explicit intra-frame motion trajectory modeling (Shang et al., 10 Jul 2025).

This design leads to a two-part objective. One part is efficiency: only blurred pixels should go through expensive computation. The other part is restoration fidelity: the receptive field used for deblurring should follow estimated motion trajectories rather than relying solely on generic spatial aggregation. A plausible implication is that the architecture is intended not merely as a pruning scheme, but as a joint blur-localization and motion-conditioned reconstruction system.

2. Backbone organization and M²AS blocks

M²AENet adopts a U-Net-like CNN built on NAFNet. The network consists of an in-projection, 4 encoder stages, a bottleneck, and 4 decoder stages, with skip connections between encoder and decoder stages. The in-projection is a $3\times 3$ convolution mapping RGB input to base channels 32. The stage configuration is reported as [1, 1, 1, 28] blocks for the encoder, 1 block for the bottleneck, and [1, 1, 1, 1] for the decoder; channels double after each downsampling. For efficiency and training stability, the mask predictor and intra-motion analyzer are placed mainly at the fourth encoder stage, i.e., the highest semantic and lowest spatial-resolution stage (Shang et al., 10 Jul 2025).

At selected scales, original NAFNet blocks are replaced by M²AS blocks (Mask- and Motion-Aware Simple blocks). The first half of the block performs mask-aware convolution:

$$\begin{aligned} \mathbf{f}_{1} &= \mathtt{LN}(\mathbf{f}_{in}), \ \mathbf{f}_2 &= \mathtt{Conv}(\mathbf{f}_{1})\odot \mathbf{m} + \mathtt{Rep}(\mathbf{f}_{1})\odot (\mathbf{1} - \mathbf{m}), \ \mathbf{f}_3 &= \mathtt{SCA}(\mathtt{SG}(\mathbf{f}_2)), \ \mathbf{f}_{mid} &= \mathtt{Conv}_{1\times 1}(\mathbf{f}_3) + \mathbf{f}_{in}, \end{aligned}$$

where $\mathtt{LN}$ is layer normalization, $\mathtt{Conv}$ is one $1\times1$ convolution followed by one $3\times3$ depth-wise convolution, $\mathbf{m}$ is the blur mask, $\mathtt{Rep}$ denotes channel repetition, $\mathtt{SG}$ is the Simple Gate from NAFNet, and $\mathtt{SCA}$ is simplified channel attention. On blurred pixels ($$\begin{aligned} \mathbf{f}_{1} &= \mathtt{LN}(\mathbf{f}_{in}), \ \mathbf{f}_2 &= \mathtt{Conv}(\mathbf{f}_{1})\odot \mathbf{m} + \mathtt{Rep}(\mathbf{f}_{1})\odot (\mathbf{1} - \mathbf{m}), \ \mathbf{f}_3 &= \mathtt{SCA}(\mathtt{SG}(\mathbf{f}_2)), \ \mathbf{f}_{mid} &= \mathtt{Conv}_{1\times 1}(\mathbf{f}_3) + \mathbf{f}_{in}, \end{aligned}$$0), the block applies full convolutional processing; on sharp pixels ($$\begin{aligned} \mathbf{f}_{1} &= \mathtt{LN}(\mathbf{f}_{in}), \ \mathbf{f}_2 &= \mathtt{Conv}(\mathbf{f}_{1})\odot \mathbf{m} + \mathtt{Rep}(\mathbf{f}_{1})\odot (\mathbf{1} - \mathbf{m}), \ \mathbf{f}_3 &= \mathtt{SCA}(\mathtt{SG}(\mathbf{f}_2)), \ \mathbf{f}_{mid} &= \mathtt{Conv}_{1\times 1}(\mathbf{f}_3) + \mathbf{f}_{in}, \end{aligned}$$1), it uses a repeated identity-like path (Shang et al., 10 Jul 2025).

The second half performs motion-aware deformable convolution:

$$\begin{aligned} \mathbf{f}_{1} &= \mathtt{LN}(\mathbf{f}_{in}), \ \mathbf{f}_2 &= \mathtt{Conv}(\mathbf{f}_{1})\odot \mathbf{m} + \mathtt{Rep}(\mathbf{f}_{1})\odot (\mathbf{1} - \mathbf{m}), \ \mathbf{f}_3 &= \mathtt{SCA}(\mathtt{SG}(\mathbf{f}_2)), \ \mathbf{f}_{mid} &= \mathtt{Conv}_{1\times 1}(\mathbf{f}_3) + \mathbf{f}_{in}, \end{aligned}$$2

where $$\begin{aligned} \mathbf{f}_{1} &= \mathtt{LN}(\mathbf{f}_{in}), \ \mathbf{f}_2 &= \mathtt{Conv}(\mathbf{f}_{1})\odot \mathbf{m} + \mathtt{Rep}(\mathbf{f}_{1})\odot (\mathbf{1} - \mathbf{m}), \ \mathbf{f}_3 &= \mathtt{SCA}(\mathtt{SG}(\mathbf{f}_2)), \ \mathbf{f}_{mid} &= \mathtt{Conv}_{1\times 1}(\mathbf{f}_3) + \mathbf{f}_{in}, \end{aligned}$$3 is a motion-aware deformable convolution and the offsets $$\begin{aligned} \mathbf{f}_{1} &= \mathtt{LN}(\mathbf{f}_{in}), \ \mathbf{f}_2 &= \mathtt{Conv}(\mathbf{f}_{1})\odot \mathbf{m} + \mathtt{Rep}(\mathbf{f}_{1})\odot (\mathbf{1} - \mathbf{m}), \ \mathbf{f}_3 &= \mathtt{SCA}(\mathtt{SG}(\mathbf{f}_2)), \ \mathbf{f}_{mid} &= \mathtt{Conv}_{1\times 1}(\mathbf{f}_3) + \mathbf{f}_{in}, \end{aligned}$$4 are not generic learned offsets but are supplied directly by the intra-motion analyzer. This preserves the residual and lightweight style of NAFNet while coupling it to explicit motion estimation (Shang et al., 10 Jul 2025).

The placement of mask-aware computation inside the block was also ablated. Three possible internal positions were considered, and the reported best tradeoff was to apply mask-aware convolution only at position (1). Applying it to more positions reduced FLOPs further but harmed restoration quality. This suggests that the architecture treats selective sparsification as beneficial only when applied at the most consequential convolution rather than indiscriminately throughout the block.

3. Blur-mask prediction and adaptive pixel pruning

The mask predictor operates on a feature map $$\begin{aligned} \mathbf{f}_{1} &= \mathtt{LN}(\mathbf{f}_{in}), \ \mathbf{f}_2 &= \mathtt{Conv}(\mathbf{f}_{1})\odot \mathbf{m} + \mathtt{Rep}(\mathbf{f}_{1})\odot (\mathbf{1} - \mathbf{m}), \ \mathbf{f}_3 &= \mathtt{SCA}(\mathtt{SG}(\mathbf{f}_2)), \ \mathbf{f}_{mid} &= \mathtt{Conv}_{1\times 1}(\mathbf{f}_3) + \mathbf{f}_{in}, \end{aligned}$$5. The feature map is flattened into $$\begin{aligned} \mathbf{f}_{1} &= \mathtt{LN}(\mathbf{f}_{in}), \ \mathbf{f}_2 &= \mathtt{Conv}(\mathbf{f}_{1})\odot \mathbf{m} + \mathtt{Rep}(\mathbf{f}_{1})\odot (\mathbf{1} - \mathbf{m}), \ \mathbf{f}_3 &= \mathtt{SCA}(\mathtt{SG}(\mathbf{f}_2)), \ \mathbf{f}_{mid} &= \mathtt{Conv}_{1\times 1}(\mathbf{f}_3) + \mathbf{f}_{in}, \end{aligned}$$6 per-pixel tokens, transformed by an MLP, pooled into a global feature by channel-wise averaging, concatenated back to each pixel representation, and passed through another MLP with softmax to yield per-pixel blur probabilities:

$$\begin{aligned} \mathbf{f}_{1} &= \mathtt{LN}(\mathbf{f}_{in}), \ \mathbf{f}_2 &= \mathtt{Conv}(\mathbf{f}_{1})\odot \mathbf{m} + \mathtt{Rep}(\mathbf{f}_{1})\odot (\mathbf{1} - \mathbf{m}), \ \mathbf{f}_3 &= \mathtt{SCA}(\mathtt{SG}(\mathbf{f}_2)), \ \mathbf{f}_{mid} &= \mathtt{Conv}_{1\times 1}(\mathbf{f}_3) + \mathbf{f}_{in}, \end{aligned}$$7

Here $$\begin{aligned} \mathbf{f}_{1} &= \mathtt{LN}(\mathbf{f}_{in}), \ \mathbf{f}_2 &= \mathtt{Conv}(\mathbf{f}_{1})\odot \mathbf{m} + \mathtt{Rep}(\mathbf{f}_{1})\odot (\mathbf{1} - \mathbf{m}), \ \mathbf{f}_3 &= \mathtt{SCA}(\mathtt{SG}(\mathbf{f}_2)), \ \mathbf{f}_{mid} &= \mathtt{Conv}_{1\times 1}(\mathbf{f}_3) + \mathbf{f}_{in}, \end{aligned}$$8 denotes the probability that a pixel is blurred. During training, a differentiable binary mask is produced with Gumbel-Softmax,

$$\begin{aligned} \mathbf{f}_{1} &= \mathtt{LN}(\mathbf{f}_{in}), \ \mathbf{f}_2 &= \mathtt{Conv}(\mathbf{f}_{1})\odot \mathbf{m} + \mathtt{Rep}(\mathbf{f}_{1})\odot (\mathbf{1} - \mathbf{m}), \ \mathbf{f}_3 &= \mathtt{SCA}(\mathtt{SG}(\mathbf{f}_2)), \ \mathbf{f}_{mid} &= \mathtt{Conv}_{1\times 1}(\mathbf{f}_3) + \mathbf{f}_{in}, \end{aligned}$$9

whereas at test time the network uses thresholding,

$\mathtt{LN}$0

This makes the blur mask both trainable and operational as an inference-time pruning signal (Shang et al., 10 Jul 2025).

The supervision strategy depends on the dataset. On ReLoBlur, the method uses refined blur masks from Li et al. (LMD‑ViT) as ground truth $\mathtt{LN}$1. On GoPro, all pixels are treated as blurred and the mask is set to all ones. The mask loss is

$\mathtt{LN}$2

where $\mathtt{LN}$3 is the number of scales with mask predictors. The same masks are also used during training to suppress processing in sharp regions and to reduce ambiguity in motion-offset estimation (Shang et al., 10 Jul 2025).

A distinctive component of M²AENet is structural reparameterization for inference-time pruning. A $\mathtt{LN}$4 convolution with weight tensor $\mathtt{LN}$5 is reshaped to $\mathtt{LN}$6, while the feature map is unfolded so that the original spatial neighborhood becomes channel dimension. The $\mathtt{LN}$7 convolution is thereby converted into an equivalent $\mathtt{LN}$8 convolution over unfolded features. This matters because $\mathtt{LN}$9 convolutions are compatible with per-pixel selection (Shang et al., 10 Jul 2025).

At inference, the network collects the indices of blurred pixels from $\mathtt{Conv}$0; if there are $\mathtt{Conv}$1 such pixels, only those $\mathtt{Conv}$2 feature vectors are sampled, convolved, and scattered back into the full map, while sharp pixels bypass heavy computation. The computational reduction scales with $\mathtt{Conv}$3. When blur occupies only a small fraction of the image, the resulting FLOPs savings can be substantial, particularly for high-resolution inputs.

4. Intra-frame motion analyzer and trajectory-guided restoration

The intra-frame motion analyzer is designed to encode blur magnitude and direction, which the mask alone cannot provide. It is implemented as a single convolutional layer applied to the feature map $\mathtt{Conv}$4, and predicts per-pixel displacements from the midpoint of exposure to the start and end of exposure:

$\mathtt{Conv}$5

The exposure interval is discretized into $\mathtt{Conv}$6 time instants $\mathtt{Conv}$7, and the ground-truth sharp image is treated as corresponding approximately to the midpoint (Shang et al., 10 Jul 2025).

Intermediate displacements are not predicted independently; they are generated by quadratic interpolation:

$\mathtt{Conv}$8

At $\mathtt{Conv}$9, the displacement is zero. The resulting set $1\times1$0 defines a per-pixel motion trajectory over exposure time. The authors adopt quadratic interpolation because it performed better than linear interpolation and was nearly as effective as predicting all displacements independently, while requiring less complexity (Shang et al., 10 Jul 2025).

These trajectories are then converted into the relative displacements required by deformable convolution. For a $1\times1$1 kernel, 9 offsets are needed per pixel. Instead of letting the model learn arbitrary offsets, M²AENet constructs offsets that trace the estimated motion path during exposure. Conceptually, the deformable kernel samples trajectory-aligned neighbors, i.e., pixels that physically contributed to the blur at a given location. This constrains the receptive field to be trajectory-aware rather than only feature-adaptive (Shang et al., 10 Jul 2025).

The motion field is supervised indirectly through a reblur consistency objective and spatial smoothness. This is important because blur is caused by the integration of moving content over time. The reported limitation is that reblur loss alone can misinterpret flat regions as moving, since low-texture static areas may satisfy the reblur objective; mask supervision is introduced partly to mitigate this ambiguity.

5. Optimization, benchmarks, and empirical behavior

The total training objective is

$1\times1$2

with $1\times1$3, and $1\times1$4 inside $1\times1$5. The reconstruction term is

$1\times1$6

while the offset term is

$1\times1$7

The reblur component synthesizes a blurred image by forward-warping the sharp ground-truth image with each displacement field and summing the warped frames; the implementation uses softmax splatting. The coefficient $1\times1$8 is not numerically specified in the main body. Training uses AdamW, momentum (0.9, 0.9), weight decay $1\times1$9, an initial learning rate $3\times3$0 decayed to $3\times3$1 with cosine annealing, batch size 8, patch size 256×256, and blur-aware patch cropping. The framework is PyTorch, and training is reported on an NVIDIA RTX A6000 GPU (Shang et al., 10 Jul 2025).

Experiments are conducted on ReLoBlur and GoPro, with a 1:1 sampling mixture used during training. ReLoBlur is a real local motion blur dataset with 2010 training images and 395 test images at resolution 2152 × 1436. GoPro is the standard global dynamic scene blur benchmark synthesized from high-frame-rate videos. Evaluation uses PSNR and SSIM on GoPro, and on ReLoBlur both full-image PSNR/SSIM and blur-weighted metrics $3\times3$2 and $3\times3$3 (Shang et al., 10 Jul 2025).

On ReLoBlur, the reported results are: NAFNet 34.584 / 0.9244 / 28.633 / 0.8751, LBAG 34.655 / 0.9257 / 29.320 / 0.8810, Restormer 34.670 / 0.9228 / 29.436 / 0.8793, LMD‑ViT 35.394 / 0.9280 / 30.240 / 0.8929, and M²AENet 35.647 / 0.9302 / 31.036 / 0.8997 for $3\times3$4. On GoPro, the reported values are NAFNet 32.402 / 0.9338, Restormer 32.452 / 0.9365, LMD‑ViT 32.500 / 0.9312, and M²AENet 33.055 / 0.9406. The paper therefore reports that M²AENet outperforms the listed baselines on both local and global blur benchmarks (Shang et al., 10 Jul 2025).

Efficiency is quantified against NAFNet and LMD‑ViT. NAFNet is reported at 17.11M parameters, 0.764T FLOPs, 5.0GB GPU memory, and 0.569s/image. LMD‑ViT is reported at 54.55M, 1.485T, 28.0GB, and 1.284s/image. M²AENet is reported at 17.44M, 0.759T, 5.8GB, and 0.790s/image. Relative to LMD‑ViT, the paper states a 49% reduction in FLOPs, about 38.5% faster inference, and GPU memory usage at about 21% of LMD‑ViT’s consumption (Shang et al., 10 Jul 2025).

Ablations separate the contributions of the two central mechanisms. The baseline NAFNet obtains 34.584 PSNR and 28.633 $3\times3$5 at 0.764T FLOPs. Adding motion-aware DConv only yields 35.091 / 29.591 / 0.995T. Adding mask-aware Conv only yields 35.021 / 30.081 / 0.528T. The full M²AENet yields 35.647 / 31.036 / 0.759T. This suggests that mask-aware convolution primarily improves local deblurring metrics and efficiency, whereas motion-aware deformable convolution primarily improves global reconstruction quality; the best overall behavior arises when both are used together. Additional ablations show that quadratic interpolation is preferred over linear interpolation, and that using $3\times3$6 time instants in the reblur loss is sufficient because gains saturate beyond that point (Shang et al., 10 Jul 2025).

6. Nomenclature, relation to earlier work, and limitations

M²AENet should be distinguished from M2A (Motion Aware Attention), a 2021 temporal module for video action recognition introduced in "M2A: Motion Aware Attention for Accurate Video Action Recognition" (Gebotys et al., 2021). That earlier work is a plug-in motion-aware attention mechanism for 2D and 3D CNN backbones in video classification, whereas M²AENet is a 2025 local motion deblurring network whose name expands to Mask- and Motion-Aware Efficient Network. The two works share the phrase “motion aware,” but they address different tasks, use different architectures, and are not variations of a single model family (Gebotys et al., 2021).

Within the deblurring literature, M²AENet is positioned against three categories of prior methods: global deblurring models, which may distort already sharp regions in local-blur scenarios; patch/window-based local methods, which allocate computation only coarsely; and methods that use blur masks without explicit motion modeling. The architecture responds to those limitations by combining a mask predictor with a motion analyzer. The paper explicitly notes that these two components might appear redundant, but the ablation study indicates that both are needed for the reported accuracy-efficiency tradeoff (Shang et al., 10 Jul 2025).

The reported limitations are concentrated in motion supervision. Reblur loss can generate spurious offsets in flat regions with low texture, because such regions may satisfy the image formation constraint even without true motion. Mask supervision alleviates this but does not eliminate the ambiguity. Future directions suggested in the paper include a more discriminative reblur loss formulation and better motion constraints that reduce reliance on explicit mask supervision. The authors also report that a larger architecture increased memory to 21GB and FLOPs to 2.1T without meaningful performance gain, which motivated retention of the original NAFNet block configuration. The reference implementation is provided at https://github.com/shangwei5/M2AENet (Shang et al., 10 Jul 2025).