DSAT-HD: Hybrid Decomposition Dual-Stream Transformer

• The paper presents DSAT-HD, an architecture that jointly integrates EMA and Fourier decompositions with adaptive sparse Transformer experts.
• It employs a multi-scale pathway and dual-stream residual learning, using CNN for seasonality and MLP for trends to enhance long-horizon predictions.
• Empirical evaluations across nine public datasets demonstrate its robust performance and effective ablations, confirming each component’s contribution.

The Hybrid Decomposition Dual-Stream Adaptive Transformer (DSAT-HD) is a multivariate time series forecasting architecture that combines three design motifs within a single pipeline: a hybrid decomposition mechanism built from EMA decomposition, Fourier decomposition, and normalization; a sparse multi-scale pathway that routes features to four parallel Transformer experts; and a dual-stream residual framework in which CNN and MLP branches specialize to seasonal and trend components, respectively (Wang et al., 29 Sep 2025). It was proposed to address two limitations attributed to prior Transformer forecasters: restricted modeling at fixed temporal scales and insufficient handling of complex, non-stationary seasonal-trend structure. In the paper’s own framing, DSAT-HD is not a transformer-only forecaster, but a compound system in which decomposition, sparse routing, local CNN operations, global attention, and residual branch specialization are jointly organized for long-horizon prediction (Wang et al., 29 Sep 2025).

1. Conceptual position and scope

DSAT-HD is formulated for multivariate forecasting with input history $x=(x_1,\dots,x_L)$, $x_t\in\mathbb{R}^M$, represented as $x\in\mathbb{R}^{M\times L}$, and target future sequence $$\hat{x}=(x_{L+1},\dots,x_{L+T})\in\mathbb{R}^{M\times T}$$ (Wang et al., 29 Sep 2025). Its central claim is that real time series often contain simultaneously local fluctuations, medium-range cycles, and long-range drift, so a single fixed-scale attention pattern and a single decomposition scheme are structurally inadequate.

The model’s name is descriptive. “Hybrid decomposition” refers to the use of both time-domain and frequency-domain operations rather than a single STL-like procedure. “Dual-stream” refers specifically to separate seasonal and trend processing branches. “Adaptive” refers to sparse Top-k routing over scale-specific Transformer experts. “Transformer” refers to the expert pathway rather than the whole model, since the architecture also contains CNN and MLP components (Wang et al., 29 Sep 2025).

Within forecasting research, DSAT-HD occupies a position close to decomposition-driven dual-stream systems such as xPatch, which also split seasonal and trend components and assign them to different branches, but does so in a non-transformer architecture based on EMA decomposition, an MLP trend stream, and a CNN seasonal stream (Stitsyuk et al., 2024). A plausible implication is that DSAT-HD should be read less as a pure attention model than as an overview of decomposition-based forecasting and adaptive sparse expert routing.

2. Hybrid decomposition mechanism

The forward pass begins with normalization and parallel decomposition. The paper writes: $$\mathbf{X}^{\text{norm}} = \text{InstanceNorm}(\mathbf{X})$$

$$\mathbf{S}, \mathbf{T} = \text{EMA}(\mathbf{X}^{\text{norm}})$$

$$\mathbf{X}' = \text{ChannelChange}(\mathbf{X}^{\text{norm}})$$

$\mathbf{X}_{\text{Freq}} = \text{FFT}(\mathbf{X}')$

$$\mathbf{S}^{\text{Freq}}, \mathbf{T}^{\text{Freq}} = \text{DecompFreq}(\mathbf{X}_{\text{Freq}})$$

The abstract, however, characterizes the normalization component as RevIN, while the methodology equations explicitly use InstanceNorm; the paper does not provide the RevIN normalization-denormalization equations (Wang et al., 29 Sep 2025). That discrepancy is one of the method’s recurrent interpretive caveats.

The time-domain branch uses EMA decomposition. In the preliminaries, the paper defines

$$v_0 = 0, \qquad v_t = \alpha v_{t-1} + (1-\alpha)\theta_t,\quad t>0$$

$x_t\in\mathbb{R}^M$0

and then operationally maps trend and seasonal components into separate downstream streams (Wang et al., 29 Sep 2025). Trend is therefore the EMA-smoothed component, while seasonality is the residual after subtracting the smoothed series.

The frequency-domain branch applies Fourier analysis and Top-k frequency selection: $x_t\in\mathbb{R}^M$1

$x_t\in\mathbb{R}^M$2

$x_t\in\mathbb{R}^M$3

The paper states that the frequency-domain trend is obtained by multi-kernel average pooling or multi-scale moving average, but does not provide an explicit pooling formula (Wang et al., 29 Sep 2025).

A fusion equation is also given: $x_t\in\mathbb{R}^M$4 This formulation notably omits $x_t\in\mathbb{R}^M$5, even though that component is defined elsewhere. This suggests that the mathematical presentation is schematic rather than fully closed-form. Even so, the intended architecture is clear: DSAT-HD does not treat decomposition as a preprocessing convenience, but as a structural prior that conditions both routing and stream specialization (Wang et al., 29 Sep 2025).

3. Sparse multi-scale adaptive pathway

After decomposition, DSAT-HD uses a sparse routing mechanism that assigns inputs to four Transformer experts with different patch sizes. The paper describes this through Top-k gating, a sparse dispatcher, four expert layers, and a sparse combiner: $x_t\in\mathbb{R}^M$6

$x_t\in\mathbb{R}^M$7

$x_t\in\mathbb{R}^M$8

Each expert is associated with a distinct patch size $x_t\in\mathbb{R}^M$9, so each specializes to a different temporal scale (Wang et al., 29 Sep 2025).

The internal expert description combines local patchwise interactions and global self-attention. The paper writes: $x\in\mathbb{R}^{M\times L}$0

$x\in\mathbb{R}^{M\times L}$1

$x\in\mathbb{R}^{M\times L}$2

$x\in\mathbb{R}^{M\times L}$3

$x\in\mathbb{R}^{M\times L}$4

$x\in\mathbb{R}^{M\times L}$5

This means each expert is not merely a vanilla Transformer block; it is a hybrid local-global extractor in which patchwise fusion and global attention coexist (Wang et al., 29 Sep 2025).

Routing is regularized by a balance term based on the coefficient of variation of gate mass and expert load: $x\in\mathbb{R}^{M\times L}$6

$x\in\mathbb{R}^{M\times L}$7

$x\in\mathbb{R}^{M\times L}$8

The paper describes this as minimizing expert collaboration variance. In effect, the expert bank is MoE-like, but with only four scale-specialized experts and without a fully elaborated noisy-gating equation in the main text (Wang et al., 29 Sep 2025).

A common misconception is to equate DSAT-HD’s adaptive behavior with generic learned depth or token pruning. The paper’s adaptivity is narrower and more specific: it is primarily scale routing through sparse expert selection. The abstract mentions “noise Top-k gating,” but the methodology does not provide the corresponding noisy-logit formulation (Wang et al., 29 Sep 2025).

4. Dual-stream residual learning framework

The dual-stream part of DSAT-HD is defined by explicit seasonal-trend specialization. The seasonal component is processed by a CNN branch, and the trend component by an MLP branch. The paper states: $x\in\mathbb{R}^{M\times L}$9

$$\hat{x}=(x_{L+1},\dots,x_{L+T})\in\mathbb{R}^{M\times T}$$0

$$\hat{x}=(x_{L+1},\dots,x_{L+T})\in\mathbb{R}^{M\times T}$$1

$$\hat{x}=(x_{L+1},\dots,x_{L+T})\in\mathbb{R}^{M\times T}$$2

This branch assignment embodies the model’s inductive claim that seasonal signals are better suited to convolutional local modeling, whereas trend is smoother and can be handled by feed-forward transformations (Wang et al., 29 Sep 2025).

The seasonal/trend split is therefore not merely analytical; it determines the computational graph. The CNN branch is intended for periodic, nonlinear, and short-range structures, while the MLP branch is intended for smoother low-frequency evolution. This suggests a deliberate separation of function classes rather than a redundant multi-branch ensemble.

DSAT-HD’s dual-stream design is closely related in spirit to forecasting models such as xPatch, where seasonality is assigned to a non-linear CNN stream and trend to a linear MLP stream after EMA decomposition (Stitsyuk et al., 2024). The difference is that DSAT-HD embeds this specialization inside a larger adaptive Transformer framework with sparse scale experts and frequency-domain decomposition.

The paper calls the architecture a dual-stream residual learning framework, but does not provide an explicit residual equation of the form $$\hat{x}=(x_{L+1},\dots,x_{L+T})\in\mathbb{R}^{M\times T}$$3 for the seasonal/trend branches themselves (Wang et al., 29 Sep 2025). Accordingly, the “residual” designation should be understood from the architectural description, not from a formally specified residual operator in the text.

5. Objective, datasets, and empirical profile

Training minimizes a sum of forecasting loss and balancing loss: $$\hat{x}=(x_{L+1},\dots,x_{L+T})\in\mathbb{R}^{M\times T}$$4 with

$$\hat{x}=(x_{L+1},\dots,x_{L+T})\in\mathbb{R}^{M\times T}$$5

and the COV-based expert-balance term defined above (Wang et al., 29 Sep 2025). The paper reports PyTorch implementation, AdamW optimization, and an initial learning rate of $$\hat{x}=(x_{L+1},\dots,x_{L+T})\in\mathbb{R}^{M\times T}$$6, but omits many implementation details, including hidden size, number of heads, patch sizes $$\hat{x}=(x_{L+1},\dots,x_{L+T})\in\mathbb{R}^{M\times T}$$7, Top-k value, batch size, and epoch count (Wang et al., 29 Sep 2025).

Evaluation is conducted on nine public datasets: ETTh1, ETTh2, ETTm1, ETTm2, Weather, Electricity, Traffic, Exchange, and Solar. With a uniform lookback window $$\hat{x}=(x_{L+1},\dots,x_{L+T})\in\mathbb{R}^{M\times T}$$8 and horizons $$\hat{x}=(x_{L+1},\dots,x_{L+T})\in\mathbb{R}^{M\times T}$$9, DSAT-HD is reported to achieve 26 first-place counts, tied only with xPatch in the paper’s count row, while showing strong consistency across datasets (Wang et al., 29 Sep 2025). Representative numbers include ETTh1 results of $$\mathbf{X}^{\text{norm}} = \text{InstanceNorm}(\mathbf{X})$$0, $$\mathbf{X}^{\text{norm}} = \text{InstanceNorm}(\mathbf{X})$$1, $$\mathbf{X}^{\text{norm}} = \text{InstanceNorm}(\mathbf{X})$$2, and $$\mathbf{X}^{\text{norm}} = \text{InstanceNorm}(\mathbf{X})$$3 in MSE/MAE across the four horizons; ETTm2 results of $$\mathbf{X}^{\text{norm}} = \text{InstanceNorm}(\mathbf{X})$$4, $$\mathbf{X}^{\text{norm}} = \text{InstanceNorm}(\mathbf{X})$$5, $$\mathbf{X}^{\text{norm}} = \text{InstanceNorm}(\mathbf{X})$$6, and $$\mathbf{X}^{\text{norm}} = \text{InstanceNorm}(\mathbf{X})$$7; and Solar results of $$\mathbf{X}^{\text{norm}} = \text{InstanceNorm}(\mathbf{X})$$8, $$\mathbf{X}^{\text{norm}} = \text{InstanceNorm}(\mathbf{X})$$9, $$\mathbf{S}, \mathbf{T} = \text{EMA}(\mathbf{X}^{\text{norm}})$$0, and $$\mathbf{S}, \mathbf{T} = \text{EMA}(\mathbf{X}^{\text{norm}})$$1 (Wang et al., 29 Sep 2025).

The paper also emphasizes robustness to varying input lengths. For lookback $$\mathbf{S}, \mathbf{T} = \text{EMA}(\mathbf{X}^{\text{norm}})$$2, DSAT-HD achieved the best result in 45 of 48 tests; for $$\mathbf{S}, \mathbf{T} = \text{EMA}(\mathbf{X}^{\text{norm}})$$3, it again achieved 45 of 48 best results (Wang et al., 29 Sep 2025). This is presented as evidence that the architecture generalizes beyond one fixed history length, which is consistent with its multi-scale routing rationale.

Ablations are especially informative. On ETTh2, the full model reports $$\mathbf{S}, \mathbf{T} = \text{EMA}(\mathbf{X}^{\text{norm}})$$4, versus $$\mathbf{S}, \mathbf{T} = \text{EMA}(\mathbf{X}^{\text{norm}})$$5 without hybrid decomposition, $$\mathbf{S}, \mathbf{T} = \text{EMA}(\mathbf{X}^{\text{norm}})$$6 without the multi-scale path, and $$\mathbf{S}, \mathbf{T} = \text{EMA}(\mathbf{X}^{\text{norm}})$$7 without the dual-stream framework. On Traffic, the full model reports $$\mathbf{S}, \mathbf{T} = \text{EMA}(\mathbf{X}^{\text{norm}})$$8, while removing hybrid decomposition yields $$\mathbf{S}, \mathbf{T} = \text{EMA}(\mathbf{X}^{\text{norm}})$$9, removing the multi-scale path yields $$\mathbf{X}' = \text{ChannelChange}(\mathbf{X}^{\text{norm}})$$0, and removing the dual-stream framework yields $$\mathbf{X}' = \text{ChannelChange}(\mathbf{X}^{\text{norm}})$$1 (Wang et al., 29 Sep 2025). These numbers indicate that all three components matter, but the dual-stream framework is the most critical on Traffic, and the multi-scale path is consistently important across datasets.

6. Relation to earlier architectures, ambiguities, and limitations

DSAT-HD synthesizes ideas that appeared separately in earlier work. In forecasting, xPatch already combined exponential decomposition with a CNN seasonal stream and MLP trend stream, but did so without Transformer experts or sparse adaptive routing (Stitsyuk et al., 2024). In a broader architectural sense, earlier hybrid models in vision had already explored fixed dual-stream local/global specialization and adaptive fusion, but not in the seasonal-trend forecasting setting or with DSAT-HD’s specific EMA-plus-Fourier decomposition and expert-routing scheme (Su et al., 2022).

Several limitations are explicit or inferable from the paper’s own presentation. First, the method description is incomplete. The abstract mentions RevIN and noisy Top-k gating, but the methodology gives InstanceNorm and a simplified Top-k abstraction, without explicit RevIN or noisy-gating equations (Wang et al., 29 Sep 2025). Second, several formulas are schematic and internally uneven; for example, the decomposition fusion equation includes $$\mathbf{X}' = \text{ChannelChange}(\mathbf{X}^{\text{norm}})$$2, $$\mathbf{X}' = \text{ChannelChange}(\mathbf{X}^{\text{norm}})$$3, and $$\mathbf{X}' = \text{ChannelChange}(\mathbf{X}^{\text{norm}})$$4, but omits $$\mathbf{X}' = \text{ChannelChange}(\mathbf{X}^{\text{norm}})$$5, and the ablation table reportedly lists the full model as “MSAT-HD,” presumably a typographical error (Wang et al., 29 Sep 2025).

Third, interpretability is limited. The paper provides forecasting visualizations on Electricity and ablations, but does not present expert utilization histograms, routing distributions, or detailed decomposition diagnostics (Wang et al., 29 Sep 2025). Fourth, performance is not uniformly dominant at every long horizon; Exchange at horizon 720 is specifically noted as a setting where DSAT-HD is not the best (Wang et al., 29 Sep 2025). Fifth, the architecture is comparatively complex, combining normalization, dual-domain decomposition, sparse routing, hybrid attention, and branch specialization in one stack.

An important clarification is that DSAT-HD should not be reduced to “a transformer with decomposition.” It is more accurately described as a hybrid decomposition + sparse multi-scale expert Transformer + seasonal/trend dual-stream learner (Wang et al., 29 Sep 2025). Its contribution lies in the joint organization of these components: decomposition narrows the statistical burden, sparse experts address scale diversity, and dual-stream branches align model class with signal component. This suggests that DSAT-HD is best understood not as a single new operator, but as a systems-level forecasting architecture built around coordinated specialization.