---
title: Adaptive Sparse Motion Aggregator (ASMA)
url: https://www.emergentmind.com/topics/adaptive-sparse-motion-aggregator-asma
type: topic
---

# Adaptive Sparse Motion Aggregator (ASMA)

Searching arXiv for the primary paper and closely related prior work cited in the provided data.
Adaptive Sparse Motion Aggregator (ASMA) is the learnable motion-aggregation module introduced within EDFFDNet, an unsupervised multi-grid image registration framework designed for scenes with depth disparities. In EDFFDNet, ASMA converts the latent motion feature produced by convolutional blocks into explicit motion parameters, namely 4-point homography motions in the global stage or free-form deformation (FFD) control-point displacements in local refinement stages. Its defining design choice is to replace the dense MLP motion aggregator used in prior work with grouped linear transformations followed by a final fusion layer, thereby transforming dense interactions into sparse ones while reducing parameters and improving accuracy [2509.07662].

## 1. Definition, role, and problem setting

Within the EDFFDNet architecture, ASMA appears at the end of the motion estimator. The pipeline is: correlation volumes, either global or local, are processed by several convolutional blocks to produce a latent motion feature; this feature is then passed to ASMA, which outputs motion parameters. In this sense, ASMA is not a standalone registration model but the motion aggregator embedded in the homography/FFD estimator [2509.07662].

The problem it addresses is explicitly tied to the limitations of previous motion aggregation schemes. The previous state-of-the-art method, UDIS++, uses MLPs for motion aggregation. The paper states that linear layers effectively achieve global motion aggregation, but that their dense interactions lead to large computation cost and high parameter counts, restricting deployment. ASMA is therefore designed to retain the benefit of global aggregation while replacing dense, fully connected interactions with sparse, grouped interactions.

This role is specific: ASMA operates on the flattened latent motion feature extracted from the convolutional blocks and regresses the final motion parameters. It is used both for global homography estimation and for local FFD refinement, so its function spans the full coarse-to-fine motion-estimation pipeline rather than a single stage.

## 2. Internal architecture and mathematical formulation

ASMA comprises two Group Linear Layers (GLL) and a single linear layer. The two GLLs implement sparse feature interactions, and the final linear layer adaptively fuses the sparse motion aggregation results to produce the motion parameters [2509.07662].

Let the flattened latent motion feature be
\[
\mathbf{F}_{\mathrm{c}} \in \mathbb{R}^{C_{\mathrm{c}}},
\]
where \(\mathbf{F}_{\mathrm{c}}\) is extracted from \(N_{\mathrm{c}}\) convolutional blocks. ASMA partitions \(\mathbf{F}_{\mathrm{c}}\) into \(N_{\mathrm{g}}\) groups:
\[
\mathbf{F}_{\mathrm{g},k}\in \mathbb{R}^{C_{\mathrm{g}}}, \quad k=1,\cdots,N_{\mathrm{g}},
\]
with
\[
C_{\mathrm{g}} = C_{\mathrm{c}} / N_{\mathrm{g}}
\]
assuming divisibility.

Each group is processed independently by its own linear transformation:
\[
\mathbf{F}'_{\mathrm{g},k}=\mathbf{W}_{k}(\mathbf{F}_{\mathrm{g},k})+\mathbf{b}_{k},
\]
where \(\mathbf{W}_{k}\) and \(\mathbf{b}_{k}\) are group-specific parameters. The transformed group features are concatenated and passed through a ReLU:
\[
\mathbf{F}'=\sigma(\text{Concat}(\mathbf{F}'_{\mathrm{g},1},\cdots,\mathbf{F}'_{\mathrm{g},N_{\mathrm{g}}})).
\]

The paper characterizes GLL as converting the dense connections of a traditional linear layer into sparse ones. In matrix terms, this corresponds to a block-diagonal structure within each grouped layer, with no cross-group connections inside the GLLs. After two GLLs, the feature is passed to a standard dense linear layer for global fusion:
\[
\mathbf{m} = \mathbf{W}_{\text{out}}\mathbf{F}^{(2)} + \mathbf{b}_{\text{out}},
\]
where \(\mathbf{m}\) denotes the motion parameters and \(\mathbf{F}^{(2)}\) is the output of the second GLL.

The sparsity here is structural rather than learned through masks or top-\(k\) selection. The paper explicitly distinguishes this from attention or dynamic masking: no such mechanisms are used. Instead, sparsity is imposed by fixed grouping in feature space.

The “adaptive” aspect has two components. First, each group has its own learned transformation \((\mathbf{W}_k,\mathbf{b}_k)\), enabling specialization across groups. Second, the final dense fusion layer can combine group outputs arbitrarily when generating motion parameters. There is no explicit sparsity-inducing loss; the grouped structure itself enforces sparsity, and all parameters are optimized end-to-end under the registration objective.

## 3. Relation to previous MLP motion aggregators

ASMA is introduced as a replacement for the MLP-based motion aggregator used in UDIS++. Previous work is described as using convolutional blocks and an MLP to construct the motion estimator. In that formulation, each output neuron is connected to all input neurons in each linear layer, producing dense all-to-all interactions. The paper notes that the parameter count and complexity of a fully connected layer scale as \(O(C_{\mathrm{in}}C_{\mathrm{out}})\) per layer, which becomes costly when the latent motion feature is high-dimensional [2509.07662].

The architectural analogy offered in the paper is to depthwise separable convolutions in MobileNet. ASMA plays, in the linear domain, a role analogous to sparse factorization in efficient convolutional design: grouped linear transformations capture local or modular interactions, while a final dense linear layer restores global coupling. The paper explicitly states that ASMA is inspired by the sparse feature processing in depthwise separable convolutions.

This comparison is central to understanding ASMA’s design rationale. The MLP aggregator preserves global aggregation but does so with dense connectivity throughout. ASMA instead performs most aggregation within groups and reserves dense interaction for the final fusion stage. This yields a different inductive bias: strong structured sparsity during intermediate processing, followed by lightweight global recombination.

The paper also states that ASMA is a drop-in replacement for dense MLP aggregation in the motion-estimation setting. A plausible implication is that the module’s utility is not limited to a single deformation family, since in EDFFDNet it serves both homography estimation and FFD control-point regression. The paper further notes that the grouped design is conceptually aligned with a “mixture-of-local-experts” perspective, although it does not formalize that interpretation theoretically.

## 4. Integration with EDFFDNet and coarse-to-fine motion estimation

ASMA is embedded in EDFFDNet’s broader registration pipeline, which combines feature extraction, correlation computation, motion estimation, and deformation composition. The input images are a target image \(\mathbf{I}_{\mathrm{t}}\) and a reference image \(\mathbf{I}_{\mathrm{r}}\). A multi-scale feature extractor produces feature maps \(\mathbf{F}_{\mathrm{t}}^{(d)}\) and \(\mathbf{F}_{\mathrm{r}}^{(d)}\) at downsample scales \(d \in \{4, 8, 16\}\) [2509.07662].

In the global homography stage, EDFFDNet uses global correlation. The patch-to-patch correlation is defined as
\[
\mathbf{C}^{\mathrm{g}}_{(x_r,y_r,x_t,y_t)} =
\sum_{i,j=-\lfloor{\frac{K}{2}}\rfloor}^{\lfloor{\frac{K}{2}}\rfloor}
\frac{ \left\langle \mathbf{F}_{\mathrm{r},(x_r+i, y_r+j)}^{(d)}, \mathbf{F}_{\mathrm{t},(x_t+i, y_t+j)}^{(d)} \right\rangle }
{\left\| \mathbf{F}_{\mathrm{r},(x_r+i, y_r+j)}^{(d)} \right\| \left\| \mathbf{F}_{\mathrm{t},(x_t+i, y_t+j)}^{(d)} \right\|}.
\]
This is transformed into a feature flow \(\mathbf{V}\) via scaling and softmax. In the local refinement stages, EDFFDNet switches to local correlation:
\[
\mathbf{C}^{\mathrm{l}}(\mathbf{p}, \mathbf{p}^{\prime}) = \mathbf{F}_{\mathrm{r}}^{(d)}(\mathbf{p})^{\top} \mathbf{F}_{\mathrm{t}}^{(d)}(\mathcal{A}(\mathbf{p}^{\prime}, r)),
\]
where \(\mathcal{A}(\mathbf{p}^{\prime}, r)\) denotes a local area.

These correlation results are processed by \(N_c\) 2D convolutional blocks, each consisting of \(3\times3\) convolution, ReLU, and max-pooling, to form the latent motion feature \(\mathbf{F}_{\mathrm{c}}\). ASMA then regresses either the 4-point homography motion or the FFD control-point displacements \(\Delta\mathbf{p}_{m,n}\).

At local refinement stage \(i\), the predicted control-point motions are used in the exponential-decay FFD:
\[
\mathbf{x}' = \mathbf{x} + \sum_{m=0}^{M_i} \sum_{n=0}^{N_i} \Delta\mathbf{p}_{m,n} \exp\left(-r_{m,n}/(\theta\eta)\right),
\]
with
\[
r_{m,n} = \| \mathbf{x}-\mathbf{p}_{m,n} \|_2.
\]
This yields residual displacement fields \(\Delta\mathbf{D}_i\), which are combined with the global homography \(\mathbf{H}\) for final warping.

ASMA therefore participates in every motion estimator in EDFFDNet. In the coarse global stage, it aggregates global correlation patterns into homography parameters. In the finer local stages, it aggregates local correlation patterns into control-point motions. Its contribution is thus coupled to the progressive correlation refinement strategy: the global-local correlation design changes the input statistics seen by ASMA, while ASMA provides a common aggregation mechanism across stages.

## 5. Empirical characteristics and efficiency–accuracy trade-off

The primary empirical evidence for ASMA comes from the ablation on motion aggregation. The paper reports the following results for the motion aggregation component while keeping the rest of the network fixed [2509.07662]:

| Motion aggregation | Parameters (M) | PSNR / SSIM |
|---|---:|---|
| MLP, reduction ratio 1 | 68.9 | 25.87 / 0.850 |
| ASMA, ratio 1, \(N_{\mathrm{g}}=4\) | 24.3 | 25.91 / 0.851 |
| ASMA, ratio 1, \(N_{\mathrm{g}}=8\) | 23.0 | 25.93 / 0.852 |
| ASMA, ratio 1, \(N_{\mathrm{g}}=16\) | 22.3 | 25.90 / 0.851 |
| MLP, reduction ratio 4 | 27.1 | 25.76 / 0.845 |
| ASMA, ratio 4, \(N_{\mathrm{g}}=8\) | 17.3 | 25.89 / 0.850 |

The paper’s stated conclusion is that ASMA outperforms MLP across all settings, achieving better accuracy while reducing parameters by \(66.6\%\) for \(N_{\mathrm{g}}=8\). This result is particularly notable because the gain is not framed merely as a compression trade-off. Even when the MLP hidden dimension is reduced by four times to bring its parameter count closer to ASMA’s, ASMA still maintains superior PSNR and SSIM.

In the full system, the difference remains substantial. UDIS++, which uses an MLP-based aggregator, is reported at 78.0M parameters and PSNR 25.43, while EDFFDNet is reported at 23.0M parameters and PSNR 25.93. The paper states that EDFFDNet reduces parameters, memory, and total runtime by \(70.5\%\), \(32.6\%\), and \(33.7\%\), respectively, while achieving a \(0.5\) dB PSNR gain over the state-of-the-art method. With an additional local refinement stage, EDFFDNet-2 further improves PSNR by 1.06 dB while maintaining lower computational costs.

The paper attributes a substantial portion of the parameter reduction to replacing the dense MLP with ASMA. It also reports strong cross-dataset generalization, outperforming previous deep learning methods. A plausible implication is that the structured sparsity introduced by grouped linear layers acts as a form of regularization, although the paper does not provide a formal theoretical analysis of that effect.

## 6. Design interpretation, limitations, and acronym ambiguity

The design logic of ASMA is closely aligned with the broader philosophy of EDFFDNet. EDFFD uses an exponential-decay basis function whose influence is local around each control point. ASMA, although operating in feature space rather than image space, also replaces globally dense intermediate interactions with structured local interactions inside groups and retains only a lightweight global fusion step. The paper summarizes this as replacing dense interactions with sparse ones for motion aggregation, reducing parameters and improving accuracy [2509.07662].

The paper does not provide a dedicated limitations section for ASMA, but it does allow several constrained observations. The sparsity pattern is static and manually chosen through the hyperparameter \(N_{\mathrm{g}}\). Grouping is performed in feature-index space rather than being explicitly aligned with spatial regions or control-point indices. There is no adaptive grouping mechanism and no data-driven sparsity schedule. The paper identifies these points only indirectly, so any extension beyond them remains speculative.

A common source of confusion is the acronym itself. In a different line of work, “ASMA” denotes “Adaptive Segmentation Mask Attack,” a targeted adversarial attack for semantic segmentation rather than a motion-aggregation module. That usage is explicitly distinct from Adaptive Sparse Motion Aggregator and appears in work on DeepLabV3 with MobileNetV3 and ResNet50 backbones on COCO 2017 [2208.01844]. The two usages share only the acronym; they refer to unrelated mechanisms, objectives, and application domains.

Within image registration, ASMA’s significance lies in being a narrowly targeted architectural modification with system-level consequences. It is neither an attention mechanism nor a learned masking method; its novelty is the use of Group Linear Layers plus a final dense fusion layer to perform motion aggregation more efficiently than dense MLPs. In EDFFDNet, that choice is integrated with exponential-decay FFD and progressive global-local correlation, yielding a motion-estimation module that is explicitly designed around locality, structured sparsity, and lightweight global coupling.

Source: https://www.emergentmind.com/topics/adaptive-sparse-motion-aggregator-asma