Adaptive Fused Dual-domain Seasonal MLP

• Adaptive Fused Dual-domain Seasonal MLP is a neural forecasting module that separates robust frequency signals and subtle time-domain patterns for accurate seasonal predictions.
• It employs parallel MLP branches—with FFT-based frequency processing and time-domain weak signal detection—that are adaptively fused using learnable channel-wise weights.
• Empirical results from architectures like MDMLP-EIA and xPatch show improved MSE/MAE metrics, confirming its effectiveness in capturing intricate seasonal patterns.

An Adaptive Fused Dual-domain Seasonal MLP is an architectural module for time series forecasting that decomposes seasonality into strong and weak spectral components, processes them through dedicated frequency- and time-domain MLP branches, and adaptively fuses their predictions using a learnable channel-wise mechanism. This approach is designed to capture both high- and low-energy seasonal signals, mitigate noise amplification in weak channels, and yield robust, accurate forecasts within an efficient, channel-independent neural forecasting framework (Zhang et al., 13 Nov 2025, Stitsyuk et al., 23 Dec 2024). Two recent instantiations include the MDMLP-EIA architecture, which introduces a frequency/time-split with adaptive channel fusion, and the dual-stream xPatch, which fuses MLP and CNN streams after exponential seasonal-trend decomposition.

1. Motivation and Context

Neural time series forecasting models, particularly channel-independent MLP-based methods, excel in efficiency and parameter economy but traditionally suffer from two weaknesses: inefficient integration of weak seasonal signals and poor noise handling in the fusion of seasonality features. Addressing these gaps requires separating seasonality into orthogonal domains (frequency and time), applying domain-specialized feature extraction, and adaptively fusing their outputs. Classical anomalies—where strong periodicities are well-modeled but weak, informative cycles are discarded as noise—necessitate this dual-domain approach. Adaptive fusion, with minimal additional parameters, counters overfitting and noise magnification while harnessing all informative signals in a scalable way (Zhang et al., 13 Nov 2025).

2. Problem Formulation and Seasonal Decomposition

Given a multivariate input series $x \in \mathbb{R}^{L \times C}$ (length $L$, $C$ channels), the input is first normalized (RevIN) and then decomposed via Exponential Moving Average (EMA):

• Trend extraction: For $t=1,\ldots,L-1$,

$$s_t = \alpha x_t + (1-\alpha)s_{t-1},\quad s_0 = x_0$$

$X_T = (s_0,\ldots,s_{L-1})$ (trend), $X_S = X - X_T$ (seasonal + residual).

• In MDMLP-EIA, the seasonal signal is modeled as $s(t) = s_\text{strong}(t) + s_\text{weak}(t)$, distinguishing dominant from weak periodicities by spectral energy (Zhang et al., 13 Nov 2025). In xPatch, the residual from EMA represents all seasonality, passed for nonlinear extraction (Stitsyuk et al., 23 Dec 2024).

3. Dual-domain Model Architecture

The Adaptive Fused Dual-domain Seasonal MLP comprises parallel paths for the seasonal component:

Frequency-domain branch (strong seasonality)

• Embedding and real FFT: Transform $x_2$ to frequency domain.
• FreMLP: Two-layer MLP processes FFT coefficients.
• Inverse FFT: Reconstruct time-frequency features.
• StrongSeasonalMLP: Maps features to forecast horizon.

Formally:

$$\begin{aligned} x_{21} &= \mathrm{Embed}(x_2) \ x_{22} &= \mathrm{rfft}(x_{21}) \ x_{23} &= \mathrm{FreMLP}(x_{22}) \ f_{s1} &= \mathrm{irfft}(x_{23}) \ y_{21} &= \text{StrongSeasonalMLP}(f_{s1})\in\mathbb{R}^{Q\times C} \end{aligned}$$

Time-domain branch (weak seasonality)

• WeakSeasonalMLP: Two-layer MLP ($\tanh$, dropout) on $x_2$ (shape $C\times L$) for low-energy cycles, output $y_{22}\in\mathbb{R}^{Q\times C}$.

Fusion via Adaptive Zero-initialized Channel Fusion (AZCF)

• Fused prediction:

$y_2 = y_{21} + \alpha \odot y_{22}$

with $\alpha\in\mathbb{R}^{1\times C}$, initialized at zero, learned per channel, and broadcast over the output horizon. This mechanism ensures weak-branch integration only where it reduces loss, minimizing noise risk (Zhang et al., 13 Nov 2025).

In xPatch (Stitsyuk et al., 23 Dec 2024), a related structure uses an MLP linear stream for trend and a CNN nonlinear stream for the EMA residual, with late fusion by concatenation followed by a projection.

4. Integration With Forecasting Pipeline

Within MDMLP-EIA (Zhang et al., 13 Nov 2025):

1. Decomposition: Input → RevIN + EMA → $x_1$ (trend), $x_2$ (season)
2. Trend branch: $x_1$ → TrendMLP ($L$→$Q$ per channel)
3. Seasonal branch: $x_2$ → [Freq and time MLPs] → AZCF fusion → $y_2$
4. Energy-Invariant Attention (EIA): Merges trend ($y_1$) and seasonal ($y_2$) predictions:

$$y_3 = 2\left[\beta\odot y_1 + (1-\beta)\odot y_2\right],\quad \beta\in[0,1]^{Q\times C}$$

$\beta$ is produced by a channel-attention MLP. The factor 2 keeps the $\ell_2$-energy of $y_3$ invariant under all $\beta$.

1. Final output: Inverse RevIN.

In xPatch (Stitsyuk et al., 23 Dec 2024), after EMA decomposition, each channel processes its trend part via linear MLPs and its seasonality via patch embedding, grouped depthwise CNNs, and MLP heads; outputs are concatenated and linearly combined for each forecast vector.

5. Implementation and Complexity Considerations

Key architectural parameters (Zhang et al., 13 Nov 2025):

• Dynamic Capacity Adjustment: Number of MLP neurons per layer $n\propto\lceil \sqrt{C}/\tau \rceil$ times the input length, ensuring scalability.
• MLPs: Channel-independent weight sharing, LeakyReLU (strong), $\tanh$ (weak), dropout $0.1$.
• Fusion parameters: AZCF introduces only $C$ new parameters; EIA adds $O(CQ)$.
• Overall computational complexity: $O(C^{1.5}L)$ per step.

For xPatch (Stitsyuk et al., 23 Dec 2024), EMA smoothing $\alpha$ is grid-searched and fixed, not learned. CNN seasonal stream uses patching, GUEL activations, batch norm, depthwise and pointwise convolutions for signal-specific nonlinear modeling. Fusion is via concatenation and a final linear projection.

6. Empirical Performance and Benchmarking

Ablation studies in MDMLP-EIA show (Zhang et al., 13 Nov 2025):

• MSE/MAE (five datasets, $T=\{96,192,336,720\}$; lower is better):

| Model Variant | MSE | MAE | |:-------------:|:-----:|:-----:| | w/o WeakSeasonal | 0.339 | 0.355 | | MLP fusion | 0.341 | 0.355 | | AZCF (ours) | 0.338 | 0.353 |

• On ETTh2, AZCF yields $1.1\%$ lower MSE compared to "w/o WS" and $2.0\%$ over standard fusion. The method preserves full oscillation amplitude and extreme values in long-horizon scenarios, outperforming baselines that fail to recover low-energy cycles.

xPatch achieves state-of-the-art mean MSE/MAE on 60–90% of benchmarks under various hyperparameter regimes, consistently outperforming prior non-transformer models (CARD, TimeMixer, PatchTST) by 2–8% absolute MSE and MAE margins (Stitsyuk et al., 23 Dec 2024). Each structural component—including dual-domain design and fusion—offers additive improvements in ablation, confirming the contribution of adaptive, fused dual-domain modeling for robust and accurate prediction.

7. Significance, Limitations, and Extensions

The Adaptive Fused Dual-domain Seasonal MLP advances channel-independent neural forecasting by:

• Systematically recovering both dominant and subtle periodicities.
• Limiting noise amplification via learnable, regularized per-channel fusion.
• Enabling scalable capacity through dynamic adjustment tied to channel count.

The AZCF rule is parameter-efficient ($O(C)$ vs $O(C^2)$ for full attention). A plausible implication is that similar adaptive, domain-split architectures may generalize to other multivariate sequence domains where weak signals carry crucial predictive information.

Limitations include marginal incremental gains for some datasets and fixed (rather than learned) EMA factors in certain variants (Stitsyuk et al., 23 Dec 2024). Future research may explore more expressive domain decompositions, non-linear fusion rules, or domain-adaptive smoothing parameters to further enhance model flexibility.

---

References:

• MDMLP-EIA: "MDMLP-EIA: Multi-domain Dynamic MLPs with Energy Invariant Attention for Time Series Forecasting" (Zhang et al., 13 Nov 2025)
• xPatch: "xPatch: Dual-Stream Time Series Forecasting with Exponential Seasonal-Trend Decomposition" (Stitsyuk et al., 23 Dec 2024)