---
title: Dual-wing Adaptive Dynamic Mixer (DADM)
url: https://www.emergentmind.com/topics/dual-wing-adaptive-dynamic-mixer-dadm
type: topic
---

# Dual-wing Adaptive Dynamic Mixer (DADM)

Searching arXiv for the specified papers and closely related work to ground the article.
Dual-wing Adaptive Dynamic Mixer (DADM) is a CNN-based module for inertial odometry introduced within the IONext backbone as a Transformer-inspired replacement for self-attention in hierarchical sequence modeling [2507.17089]. It is designed to address a specific tension in IMU-based localization: pure Transformers model long-range dependencies effectively but have weak inductive biases and limited sensitivity to local, fine-grained motion variations, whereas conventional CNNs preserve local detail but have a restricted receptive field and static kernels [2507.17089]. DADM resolves this by splitting channels into two parallel “wings,” applying multi-scale depthwise temporal convolutions in each wing, and using input-dependent scalar weights to adaptively fuse the scales before recombination. In IONext, DADM serves as the principal spatial or spatio-channel mixer inside the Adaptive Dynamic Encoder (ADE), while the Spatio-Temporal Gating Unit (STGU) assumes the role of temporal gating [2507.17089]. The design is conceptually related to TransXNet’s Dual Dynamic Token Mixer, which also uses an evenly split dual-branch architecture to combine dynamic global and local processing, though DADM specializes this pattern for 1D IMU sequences rather than 2D vision features [2310.19380].

## 1. Origin and architectural role

DADM was proposed in "IONext: Unlocking the Next Era of Inertial Odometry" as the core spatial mixer of a purely convolutional inertial odometry backbone [2507.17089]. The underlying motivation is explicit. Transformer-based inertial odometry models are described as strong at long-range or global dependency modeling, but limited by weak inductive biases such as locality and translation equivariance, reduced sensitivity to local, fine-grained motion variations, and substantial computational cost on long sequences [2507.17089]. Standard CNNs exhibit the opposite profile: they preserve high-frequency local detail through small kernels but require very deep stacks or very large kernels to expand receptive field, and their convolution kernels remain static rather than input-adaptive [2507.17089].

Within IONext, DADM is embedded in the ADE block, where it replaces the self-attention sublayer of a Transformer encoder. The block equations are given as
\[
\begin{aligned}
X_i' &= X_i + \mathrm{DADM}(\mathrm{BN}(X_i)), \\
X_{i+1} &= X_i' + \mathrm{STGU}(\mathrm{BN}(X_i')),
\end{aligned}
\]
with batch normalization applied before each mixer [2507.17089]. In this arrangement, DADM performs the main spatial or spatio-channel mixing, while STGU replaces the MLP and carries out temporal gating [2507.17089].

IONext is a hierarchical CNN backbone described as conceptually similar to Swin-Transformer but purely convolutional [2507.17089]. Its four stages contain \(N_i=[2,2,6,2]\) ADE blocks with channel sizes \([96,192,384,768]\), so DADM appears in 12 encoder blocks across the network [2507.17089]. This repeated use is central to the model’s strategy for accumulating long-range temporal context via multi-scale convolutions and stage-wise downsampling rather than explicit self-attention.

## 2. Internal structure of the dual-wing mixer

The defining property of DADM is its “dual-wing” channel split. For an ADE input \(X_i \in \mathbb{R}^{C \times T}\), batch normalization produces \(\tilde X_i = \mathrm{BN}(X_i)\), after which channels are evenly divided into two parallel branches
\[
X_j \in \mathbb{R}^{\frac{C}{2} \times T}, \quad j \in \{0,1\}
\]
[2507.17089]. Each wing applies an identical multi-scale depthwise convolutional module followed by adaptive fusion, producing outputs \(F_0\) and \(F_1\), which are concatenated and linearly remixed [2507.17089].

The authors motivate the split by citing prior work such as InceptionNext, noting that important information is not uniformly distributed over channels and that naive random partitioning in depthwise structures can be harmful [2507.17089]. The even split is therefore intended to ensure full channel utilization while reducing compute and parameter cost relative to running full multi-scale stacks over all channels [2507.17089]. The paper further describes the two wings as analogous to multiple heads in attention, but implemented with convolutions instead of pairwise token interactions [2507.17089].

Inside each wing, DADM applies three parallel 1D depthwise convolutions along time:
\[
Y_{i,j} = \operatorname{DWConv}_i(\mathrm{BN}(X_j)), \quad i \in \{0,1,2\}
\]
with kernel sizes \(1\), \(k\), and \(3k+2\) [2507.17089]. In the concrete IONext configuration, ADE blocks use the settings
\[
(1,3,11; 1,5,17),
\]
corresponding to \((k,3k+2)=(3,11)\) or \((5,17)\) across stages [2507.17089]. The \(k=1\) branch acts as a purely local or per-time-step transform, the medium branch captures short-term motion segments, and the large branch expands temporal receptive field enough to approximate global phrase-like motion patterns over second-scale windows when stacked hierarchically [2507.17089].

After the per-wing fusion step, the two outputs are concatenated:
\[
F_{\text{concat}} = \mathrm{Concat}(F_0,F_1) \in \mathbb{R}^{C \times T}
\]
and passed through a final \(1 \times 1\) convolution
\[
F_{\text{out}} = W_2 \cdot F_{\text{concat}}
\]
for cross-channel mixing and projection back to \(C\) channels [2507.17089]. The result is injected through a residual connection,
\[
X_i' = X_i + F_{\text{out}},
\]
which preserves identity information and eases optimization [2507.17089].

## 3. Adaptive dynamic mixing mechanism

DADM’s adaptivity resides in its scale-selection mechanism rather than in full token-token attention. For each wing \(j\), the outputs of the three depthwise branches \(Y_{0,j}, Y_{1,j}, Y_{2,j}\) share the same shape \(\frac{C}{2}\times T\), and the model computes input-dependent scalar weights \(\omega_i\) over these scales [2507.17089].

The weight-generation process begins with adaptive global average pooling over time on \(X_j\), yielding a pooled descriptor \(s_j \in \mathbb{R}^{\frac{C}{2}}\) [2507.17089]. This descriptor is passed through a 1D convolution with kernel size 1, represented as \(W_1 \cdot s_j\), to produce three logits, one for each scale [2507.17089]. A softmax then normalizes them:
\[
\omega_i = \operatorname{softmax}\left(W_1 \cdot \mathrm{Ada}_{\text{mean}}(X_j)\right)_i
\]
or equivalently
\[
\omega_i = \frac{\exp(u_i)}{\sum_{l=0}^{2}\exp(u_l)}
\]
[2507.17089]. These \(\omega_i\) are scalar weights broadcast along channels and time, so the adaptivity is per wing and per sequence rather than per channel or per timestamp [2507.17089].

Fusion within a wing is then defined by
\[
F_j = \sum_{i=0}^{2} \omega_i \odot Y_{i,j}, \quad j \in \{0,1\}
\]
[2507.17089]. This mechanism makes DADM input-adaptive in a restricted but efficient sense. Rather than learning arbitrary pairwise affinities as in self-attention, it dynamically selects among receptive fields of sizes \((1,k,3k+2)\) according to the current motion pattern [2507.17089].

The paper explicitly compares this behavior to squeeze-and-excitation and dynamic convolution. Squeeze-and-excitation reweights channels using pooled global context, whereas DADM reweights scales using pooled global context [2507.17089]. The interpretation offered is that high-frequency sequences can induce upweighting of smaller kernels, while smoother motion can favor larger kernels [2507.17089]. Although the distribution of \(\omega_i\) is not plotted, the gains reported across heterogeneous scenarios such as pocket, handheld, cart, and indoor or outdoor motion suggest that the effective receptive field is being modulated according to motion regime and noise pattern [2507.17089]. This suggests that DADM functions as a structured low-dimensional alternative to attention: adaptive, but constrained to scale fusion rather than unconstrained token interaction.

## 4. Local detail, global context, and inductive bias

A central claim for DADM is that it jointly captures local fine-grained motion features and global motion patterns [2507.17089]. The \(k=1\) depthwise branch preserves the highest temporal resolution and is described as maximally sensitive to instantaneous changes such as sudden rotational spikes or quick accelerations [2507.17089]. The medium branch with kernel size 3 or 5 captures short-term temporal patterns such as short turns or segments of gait cycles [2507.17089]. These two branches together provide high-frequency modeling relevant to instantaneous velocity and heading estimation [2507.17089].

The large branch with kernel size 11 or 17 supplies expanded temporal context [2507.17089]. The paper notes that with IMU rates around 100–200 Hz and a 1 second window, a kernel length of 11–17 covers approximately \(0.055\text{–}0.17\) seconds, and when these kernels are stacked across 12 ADE blocks with downsampling between stages, the effective receptive field extends over the entire 1 second window and beyond in deeper stages [2507.17089]. The deeper blocks operate at coarser temporal resolution because each stage halves temporal length through stride-2 convolution, so their large kernels capture long-range dependencies over the entire motion snippet [2507.17089]. The intended result is Transformer-like global context without explicit attention.

This design introduces several inductive biases that the paper contrasts with pure Transformers: locality and translation equivariance through convolution, multi-scale temporal aggregation reminiscent of Inception or ResNet-style pyramids, and dynamic scale selection that is restricted across scales rather than across arbitrary token pairs [2507.17089]. The authors argue that these properties make DADM more robust to noisy IMU signals and limited data, while also avoiding the \(O(T^2)\) complexity and parameterization associated with multi-head self-attention [2507.17089].

A plausible implication is that DADM’s improvement is not merely due to larger receptive fields, but to the particular combination of structured temporal filtering and modest input adaptivity. In the paper’s framing, self-attention must learn useful temporal filters indirectly through similarity structure, whereas DADM imposes a prior over motion-relevant temporal filters and learns how to interpolate between them on a per-sequence basis [2507.17089].

## 5. Relationship to ADE, STGU, and related dual-dynamic mixers

DADM is not a standalone module in IONext; it is paired with the Spatio-Temporal Gating Unit inside ADE. The order is fixed:
\[
\begin{aligned}
X_i' &= X_i + \mathrm{DADM}(\mathrm{BN}(X_i)), \\
X_{i+1} &= X_i' + \mathrm{STGU}(\mathrm{BN}(X_i')).
\end{aligned}
\]
DADM therefore acts first, computing dynamic multi-scale features, while STGU follows by assigning time-step-specific weights to emphasize representative and task-relevant temporal segments [2507.17089].

STGU itself consists of a gating branch and a value branch. The gating branch computes
\[
\xi = \sigma\Big( W_3 \cdot \mathrm{Concat}(\mathrm{Ada}_{\text{mean}}(X_i'), \mathrm{Ada}_{\text{max}}(X_i')) \Big),
\]
while the value branch uses a depthwise convolution on \(X_i'\), and the output is
\[
Z = \xi \cdot \operatorname{DWConv}(X_i')
\]
[2507.17089]. The key distinction is that DADM adapts at the level of scale selection per sequence and per wing, whereas STGU performs temporal gating over time indices [2507.17089]. The paper summarizes this complementarity as DADM determining *what* to compute at each temporal scale and STGU determining *when* those features matter most [2507.17089].

The notion of a dual dynamic mixer has a clear precedent in TransXNet’s Dual Dynamic Token Mixer [2310.19380]. There, channels are evenly split between a global dynamics branch based on Overlapping Spatial Reduction Attention (OSRA) and a local dynamics branch based on input-dependent depthwise convolution (IDConv), after which the results are concatenated and refined [2310.19380]. The TransXNet paper explicitly frames this as simultaneous learning of global and local dynamics through input-dependent aggregation weights [2310.19380]. DADM shares the channel-split dual-wing principle and the emphasis on dynamic local processing, but replaces the vision-oriented global attention wing with a convolutional multi-scale mixer specialized for 1D temporal IMU signals [2507.17089; 2310.19380].

The IONext paper directly notes analogies to multiple heads in attention and cites InceptionNext for channel partitioning intuition [2507.17089]. It also situates DADM alongside large-kernel CNNs such as RepLKNet and SLaK, Inception-style architectures, ConvNeXt, SE blocks, MetaFormer-type token mixers, and TransXNet [2507.17089]. In this positioning, DADM is best understood not as a generic attention surrogate, but as a specialized dual dynamic temporal mixer that “attends across scales” using input-conditioned weights [2507.17089].

## 6. Empirical findings, implementation profile, and limitations

The empirical evidence for DADM is reported through the performance of IONext and through ablations on ADE and related module variants. On six public datasets, IONext is reported to consistently outperform state-of-the-art Transformer- and CNN-based methods [2507.17089]. On the RNIN dataset, the abstract states that IONext reduces average ATE by 10% and average RTE by 12% compared to iMOT [2507.17089]. In the detailed results section, Table 2 attributes larger reductions on RNIN relative to iMOT, namely \(\overline{ATE}\) by 45.5%, \(\overline{RTE}\) by 46.8%, and \(\overline{ALE}\) by 16.8% [2507.17089]. Because these aggregate gains arise from the full IONext design, the paper uses ablation to isolate DADM’s role.

Those ablations indicate that replacing RoNIN-ResNet residual blocks with ADE, deepening the network to \([2,2,6,2]\), widening channels to \([96,192,384,768]\), and modifying the stem each improve performance, with ADE identified as a major contributor [2507.17089]. Additional comparisons show that RoNIN-ResNet + DADM already shifts the CDF curves of ATE and RTE favorably on RNIN, and that removing STGU while keeping DADM still produces significant gains over RoNIN-ResNet [2507.17089]. Radar plots over six datasets further show RoNIN-ResNet being dominated by IONext without STGU, which the paper interprets as evidence of DADM’s impact [2507.17089].

From an implementation standpoint, DADM is part of a model that operates on 1 second windows of IMU data with \(C=6\) input channels corresponding to a 3D accelerometer and 3D gyroscope, \(X \in \mathbb{R}^{6\times T}\), where \(T\) depends on the sampling rate, such as 100 Hz or 200 Hz [2507.17089]. IONext begins with a 1D convolutional stem using kernel 4 and stride 4 to create tokens without explicit positional encoding [2507.17089]. Within each DADM, the depthwise branches have stride 1 and padding that preserves temporal length, and the final projection \(W_2\) is a \(1 \times 1\) convolution [2507.17089]. Downsampling between stages uses 1D non-overlapping convolution with kernel 2 and stride 2, and BatchNorm is reported to outperform LayerNorm in this setting [2507.17089].

The training objective does not include a DADM-specific auxiliary loss. The network is optimized end-to-end using mean squared error on predicted window-averaged velocity:
\[
\mathcal{L}_{\text{MSE}} = \frac{1}{N} \sum_{i=1}^{N} \left\|\hat{V}_i - V_i\right\|_2^2
\]
[2507.17089]. Training uses Adam with batch size 512, maximum 100 epochs, early stopping when the learning rate falls below \(10^{-6}\), and initial learning rate \(10^{-4}\) on RTX 3090 hardware [2507.17089].

Efficiency is a recurring theme. IONext with DADM and STGU is reported at approximately \(7.3\times 10^7\) FLOPs and \(1.1\times 10^7\) parameters, compared with a Swin-Transformer baseline at \(4.5\times 10^9\) FLOPs and \(2.83\times 10^7\) parameters [2507.17089]. This supports the paper’s claim that DADM contributes to a significantly more efficient alternative to attention-based inertial odometry backbones [2507.17089].

The paper also notes limitations. DADM itself does not explicitly handle device orientation changes, and the overall IONext model still struggles with random device rotations [2507.17089]. Suggested future directions include rotation-aware mechanisms such as equivariant convolutions or orientation-invariant encoding, as well as richer forms of adaptivity such as per-channel or per-time-step dynamic convolution or combinations with lightweight self-attention or state-space models [2507.17089]. This suggests that DADM’s current formulation is deliberately conservative: dynamic enough to recover some of attention’s context sensitivity, but structured enough to retain CNN priors and efficiency.

Source: https://www.emergentmind.com/topics/dual-wing-adaptive-dynamic-mixer-dadm