Multi-Scale Decomposable Mixing (MDM)

• Multi-Scale Decomposable Mixing (MDM) is a design pattern that decomposes temporal data into multiple resolutions using average pooling and re-mixes them through learnable residual MLP blocks.
• It is applied in time-series forecasting and sEEG speech decoding to extract hierarchical features that capture both coarse trends and fine-grained details.
• MDM integrates with end-to-end architectures through modules like Tent, AMS, and AMWG, yielding significant improvements in prediction accuracy and robustness.

Searching arXiv for the cited MDM papers and closely related work. arXiv.search query: id:(Wang et al., 29 Sep 2025) OR id:(Hu et al., 2024) OR id:(Gao et al., 13 May 2025) OR id:(Wang et al., 2024) Multi-Scale Decomposable Mixing (MDM) denotes a family of multiscale representation mechanisms for temporal data in which an input sequence is decomposed into multiple resolutions and then re-mixed through learnable mappings so that coarse and fine temporal structure can be used jointly. In recent arXiv literature, the term appears explicitly in time-series forecasting and sEEG-based speech decoding, and it is also closely aligned with decomposable multiscale mixing formulations such as TimeMixer’s Past-Decomposable-Mixing and Future-Multipredictor-Mixing blocks. Across these instantiations, the recurring idea is to construct a temporal pyramid by repeated average pooling or granularity-specific predictors, then propagate information across scales through residual MLP-style transformations, often before downstream dependency modeling or inference-time adaptation (Hu et al., 2024, Wang et al., 29 Sep 2025, Wang et al., 2024).

1. Scope of the concept and major instantiations

MDM is not presented as a single canonical block across the literature. In the AMD framework for time series forecasting, MDM sits at the entrance of the model and “peel[s] apart a raw multivariate time series into its constituent temporal patterns at different resolutions and then re-stitch[es] them into an enhanced residual representation” (Hu et al., 2024). In the sEEG speech-decoding framework, the Multi-Scale Decomposable Mixing module likewise sits “at the very front of the network” and extracts “a hierarchy of temporal features from the raw multi-channel sEEG time series” before tokenization and Transformer processing (Wang et al., 29 Sep 2025). TimeMixer describes a related decomposable multiscale mechanism in which seasonal and trend components are mixed in opposite directions across scales, and MDMixer extends the multiscale idea through dual-branch decomposition, parallel predictors, iterative mixing, and channel-adaptive weighting (Wang et al., 2024, Gao et al., 13 May 2025).

Setting	Scale construction	Distinctive mechanism
AMD (Hu et al., 2024)	Repeated average downsampling with factor $d$	Channel-wise top-down residual MLP mixing
sEEG speech decoding (Wang et al., 29 Sep 2025)	Three-level temporal pyramid via 1-D average pooling	Low-rank residual MLP hierarchy before Transformer
TimeMixer (Wang et al., 2024)	Iterated average-pooling by factor $2$	Seasonal fine$\rightarrow$coarse and trend coarse$\rightarrow$fine mixing
MDMixer (Gao et al., 13 May 2025)	Parallel multi-granularity forecasting heads	Iterative coarse$\rightarrow$fine fusion with channel-wise gating

This suggests that MDM is best understood as a design pattern centered on explicit multiscale temporal factorization and re-composition, rather than as a fixed, universally standardized operator.

2. Core computational pattern: temporal pyramids and residual remixing

A minimal MDM formulation begins from a temporal pyramid. In AMD, with input history $X \in \mathbb{R}^{C \times L}$, the model performs an $h$-level decomposition with downsampling factor $d$. Channel-wise, $\tau_1^{(c)} = x^{(c)} \in \mathbb{R}^{L}$ and, for $i = 2,\dots,h$,

$2$0

Top-down residual mixing then updates

$2$1

and the finest-scale output $2$2 is stacked into $2$3 (Hu et al., 2024).

The sEEG formulation is structurally similar but expressed in matrix form. For one trial, the preprocessed multi-channel signal is $2$4. A pyramid of $2$5 scales $2$6 is formed using 1-D average pooling along the time axis,

$2$7

with kernel size $2$8 and stride $2$9 by default. Mixed features $\rightarrow$0 are defined by

$\rightarrow$1

where

$\rightarrow$2

Here $\rightarrow$3 and $\rightarrow$4, so the MLP preserves the channel dimension at each time step; in the reported implementation, $\rightarrow$5 after an initial $\rightarrow$6 projection and $\rightarrow$7, giving inner dimension $\rightarrow$8 (Wang et al., 29 Sep 2025).

Several implementation choices recur in these formulations. Average pooling supplies the scale hierarchy; residual additions preserve fine-scale signals while injecting coarse-scale context; and the mixing blocks are intentionally lightweight. In AMD, each $\rightarrow$9 is a two-layer feedforward network with hidden dimension matching its input length. In the sEEG module, the MLP is low-rank, two-layer, uses GELU, has no dropout, employs no dilation, and includes no positional encoding inside MDM because position is handled afterward by the Transformer. The sEEG paper further states that no additional gating or explicit normalization is employed inside MDM beyond any batch-norm or layer-norm already configured inside the linear layers (Hu et al., 2024, Wang et al., 29 Sep 2025).

3. Extensions of MDM: decomposition, directional mixing, and adaptive fusion

Some MDM-family models move beyond a plain temporal pyramid and embed decomposition directly into the multiscale operator. TimeMixer first forms $\rightarrow$0 scales by iterated average pooling. At each scale $\rightarrow$1 and layer $\rightarrow$2, it decomposes the scale-specific feature $\rightarrow$3 into trend and seasonal components by a moving-average filter:

$\rightarrow$4

Its Past-Decomposable-Mixing block then performs seasonal mixing in the fine-to-coarse direction,

$\rightarrow$5

and trend mixing in the coarse-to-fine direction,

$\rightarrow$6

before residual fusion through a channel-wise feed-forward layer (Wang et al., 2024).

MDMixer introduces a different elaboration. It begins with reversible instance normalization (RevIN), then extracts trend and seasonal components by a moving-average filter,

$\rightarrow$7

and routes them into separate “Linear-based” and “MLP-based” subnetworks. It divides each branch into patches, embeds them, and uses a Multi-Granularity Parallel Predictor (MPP) to produce scale-specific forecasts of lengths $\rightarrow$8 with base granularity unit $\rightarrow$9. These are fused through a Multi-Granularity Iterative Mixer (MIM),

$\rightarrow$0

and then combined by an Adaptive Multi-Granularity Weighting Gate (AMWG) whose weights are normalized channel-wise across heads by softmax (Gao et al., 13 May 2025).

These variants clarify that “decomposable” in MDM can refer to at least two related operations. In AMD and the sEEG model, decomposition primarily means explicit multiresolution factorization followed by top-down remixing. In TimeMixer and MDMixer, decomposition is strengthened by separating trend and seasonal structure and, in MDMixer, by allowing channel-specific weighting of granularities. A plausible implication is that later MDM-family systems treat multiscale mixing not merely as hierarchical resampling, but as a controlled interaction among heterogeneous temporal components.

4. Role within end-to-end architectures

In AMD, MDM is the front-end module and produces the enriched feature map $\rightarrow$1. That output is passed to the Dual Dependency Interaction (DDI) block, which chops $\rightarrow$2 into patches along the time axis, applies temporal MLPs and optional cross-channel MLPs scaled by $\rightarrow$3, and re-assembles the result into $\rightarrow$4. Adaptive Multi-predictor Synthesis (AMS) then uses each channel’s $\rightarrow$5 to compute selector weights through a noisy-gating TopK+Softmax network and uses $\rightarrow$6 as inputs to an array of $\rightarrow$7 small predictors, yielding the final forecast $\rightarrow$8 (Hu et al., 2024).

In the sEEG speech-decoding framework, the finest-scale mixed feature $\rightarrow$9 is the direct output of MDM. The model then chops $X \in \mathbb{R}^{C \times L}$0 into fixed-length, non-overlapping temporal patches of length $X \in \mathbb{R}^{C \times L}$1 samples, projects each patch to a 512-dimensional token, adds a learnable class token, and feeds the sequence into an 8-layer Transformer encoder. All normalization layers across the network, including those inside $X \in \mathbb{R}^{C \times L}$2 if present, are subject to source-free online test-time adaptation via Tent. At inference time, only the affine parameters $X \in \mathbb{R}^{C \times L}$3 of each norm layer are updated to minimize prediction entropy,

$X \in \mathbb{R}^{C \times L}$4

so that MDM’s feature statistics can shift slightly to match the test distribution without modifying other weights (Wang et al., 29 Sep 2025).

TimeMixer places multiscale decomposition and remixing at the core of both past extraction and future prediction. After $X \in \mathbb{R}^{C \times L}$5 PDM layers operate on the multiscale set $X \in \mathbb{R}^{C \times L}$6, the model attaches one small predictor per scale,

$X \in \mathbb{R}^{C \times L}$7

and sums them to obtain the final forecast,

$X \in \mathbb{R}^{C \times L}$8

The Future-Multipredictor-Mixing block is therefore not separate from the multiscale philosophy; it is the forecasting-stage counterpart that aggregates complementary predictive capabilities from fine and coarse observations (Wang et al., 2024).

5. Objectives, supervision, and implementation regimes

MDM itself is commonly trained without an MDM-specific auxiliary loss. In the sEEG model, there is no separate $X \in \mathbb{R}^{C \times L}$9 term. The full network is trained end-to-end with standard cross-entropy for word classification,

$h$0

plus a layer-wise self-distillation regularizer from Zhang et al. 2021,

$h$1

The MDM-specific hyperparameters are fixed throughout the reported experiments: $h$2 scales, pooling window and stride both equal to $h$3, channel dimension $h$4 after the initial projection, bottleneck ratio $h$5, and no positional encoding inside MDM (Wang et al., 29 Sep 2025).

AMD likewise treats MDM as a purely feed-forward representational stage rather than as a separately supervised module. The paper states that layer normalization is applied downstream in DDI, and that inserting batch- or layer-norm inside MDM was found unnecessary. Its emphasis is architectural: MDM provides a multi-scale residual representation that downstream blocks exploit for dependency modeling and adaptive synthesis (Hu et al., 2024).

MDMixer adopts a more explicitly multi-output supervision scheme. With final prediction $h$6 and ground truth $h$7, the primary loss is

$h$8

Each intermediate scale is aligned with a downsampled target through

$h$9

and the total loss is

$d$0

with $d$1 by default. The training setup uses AdamW with learning rate $d$2 or $d$3, look-back $d$4, horizons $d$5, hidden size $d$6 or $d$7 on large datasets, patch length $d$8, stride $d$9, $\tau_1^{(c)} = x^{(c)} \in \mathbb{R}^{L}$0 heads, and reports the mean over three runs with different seeds (Gao et al., 13 May 2025).

TimeMixer uses an $\tau_1^{(c)} = x^{(c)} \in \mathbb{R}^{L}$1 loss on the final forecast, Adam optimization, learning rate approximately $\tau_1^{(c)} = x^{(c)} \in \mathbb{R}^{L}$2 for long-term forecasting or $\tau_1^{(c)} = x^{(c)} \in \mathbb{R}^{L}$3 for short-term forecasting, batch size $\tau_1^{(c)} = x^{(c)} \in \mathbb{R}^{L}$4–$\tau_1^{(c)} = x^{(c)} \in \mathbb{R}^{L}$5, and $\tau_1^{(c)} = x^{(c)} \in \mathbb{R}^{L}$6–$\tau_1^{(c)} = x^{(c)} \in \mathbb{R}^{L}$7 epochs. Its description states that there is no extra gating or normalization beyond standard weight decay, and inference is a single forward pass outputting $\tau_1^{(c)} = x^{(c)} \in \mathbb{R}^{L}$8 (Wang et al., 2024).

6. Empirical evidence, ablations, and interpretive issues

The strongest direct evidence for the utility of MDM comes from ablation studies that isolate or partially isolate the multiscale mixing component. In the sEEG benchmark, evaluated in a strict subject-dependent setting on the 12-subject DU-IN benchmark, the DU-IN baseline achieves a mean accuracy of $\tau_1^{(c)} = x^{(c)} \in \mathbb{R}^{L}$9 on subjects 06–12. Adding self-distillation raises this to $i = 2,\dots,h$0; adding MDM together with self-distillation yields $i = 2,\dots,h$1; and the full system with Tent reaches $i = 2,\dots,h$2. For subject 10, performance rises from $i = 2,\dots,h$3 for DU-IN to $i = 2,\dots,h$4 for the full MDM-Tent pipeline, with roughly $i = 2,\dots,h$5 absolute points ascribed to MDM alone in the $i = 2,\dots,h$6 ablation. The paper further states that, in the full 12-subject average, “MDM-Tent” outperforms “DU-IN” by $i = 2,\dots,h$7–$i = 2,\dots,h$8 absolute on nearly every subject (Wang et al., 29 Sep 2025).

AMD reports a sensitivity study on ETTm1 in which removing MDM by setting the number of downsampling layers $i = 2,\dots,h$9 yields MSE values $2$00 for forecast horizons $2$01. Introducing $2$02 lowers these to $2$03, a relative reduction of $2$04–$2$05. As a plugin study, adding MDM + AMS to DLinear and MTS-Mixers on Weather reduces average MSE by $2$06 for DLinear and by $2$07 for MTS-Mixers, while memory and per-iteration training time remain “virtually unchanged.” The complete AMD system is reported to train in $2$08 ms/iter and use $2$09 GB on Weather $2$10 (Hu et al., 2024).

MDMixer reports that, on eight standard LTSF benchmarks, it reduces MAE by $2$11 over TimeMixer and by $2$12 over iTransformer, with average MSE improvement of $2$13 and MAE improvement of $2$14 compared to TimeMixer. In a dual-branch ablation, grafting its MLP trend branch onto DLinear yields an extra approximately $2$15 MSE gain, while analogous grafts for RLinear and NLinear yield $2$16–$2$17 gains. An ablation of MPP, MIM, AMWG, and the alignment loss on ETTm1 and Weather shows that removing any component degrades MSE/MAE by $2$18–$2$19. Its interpretability analysis on Electricity visualizes a sparse $2$20 matrix in which most channels place heavy weight on head $2$21 and head $2$22, while different channels emphasize different intermediate heads (Gao et al., 13 May 2025).

TimeMixer reports average benchmark improvements including Solar-Energy MSE $2$23 with a $2$24 change versus PatchTST $2$25, Weather MSE $2$26 with a $2$27 change versus PatchTST $2$28, ETTm1 MSE $2$29 versus PatchTST $2$30, PEMS03 MAE $2$31 versus SCINet $2$32, and M4 weighted OWA $2$33 versus TimesNet $2$34. Its efficiency discussion states that, because each PDM mixing and Predictor is a small MLP of size $2$35 and $2$36, TimeMixer scales as $2$37 in time and memory; on ETTh1 with $2$38, it uses approximately $2$39 GiB GPU versus PatchTST approximately $2$40 GiB, and $2$41 s/it versus $2$42 s/it. Ablations report that removing FMM raises error by approximately $2$43–$2$44, removing BU or TD mixing also degrades performance by $2$45–$2$46, and reversing the mixing directions fails (Wang et al., 2024).

Two recurring interpretive points follow from these results. First, MDM is repeatedly associated with robustness to heterogeneous temporal structure, whether phrased as “stable multi-timescale representations” in sEEG decoding or as disentangling “slow, large-magnitude trends and faster, higher-frequency fluctuations” in forecasting (Wang et al., 29 Sep 2025, Hu et al., 2024). Second, the empirical record does not support the misconception that multiscale decomposition alone is sufficient. The highest-performing systems combine MDM-style mixing with additional mechanisms such as self-distillation and Tent in sEEG, DDI and AMS in AMD, FMM in TimeMixer, or AMWG and alignment loss in MDMixer. In that sense, the literature presents MDM as a foundational front-end or backbone principle whose effect is amplified by complementary modules rather than as an isolated universal solution.