TA-SparseMG: Trend-Aware Sparse Forecasting via Multi-Scale Gating for Long-Term Time Series

Abstract: Long-term time series forecasting finds extensive applications in domains such as power demand, traffic flow, meteorological observation, and renewable energy dispatch. Forecasting dynamically varying long-term time series poses inherent challenges, including statistical nonstationarity, local high-frequency disturbances, and coupled cross-period dependencies, which make it difficult for lightweight models to balance parameter efficiency and forecasting performance. To address this issue, this study presents TA-SparseMG, a lightweight cross-period forecasting model built on SparseTSF's sparse cross-period modeling framework. It incorporates three key modules: a trend-aware reversible instance normalization module, a scale-adaptive gated denoising module, and a multiscale gated-attention MLP forecasting module. The trend-aware normalization module captures input-window statistics and calibrates forecast-window distributions, effectively mitigating distribution shift. The scale-adaptive gated denoising module performs feature smoothing and residual suppression before period rearrangement, thereby reducing interference from high-frequency perturbations. The multiscale gated attention prediction module strengthens the prediction head's adaptive representational capacity via conditional gating and feature modulation. Extensive experiments across multiple LTSF benchmarks demonstrate that the proposed TA-SparseMG consistently achieves superior, stable performance. Ablation studies confirm that each module independently improves distribution adaptation, input robustness, and cross-period feature mapping capability.

• The paper introduces a modular forecasting framework that integrates trend-aware normalization, adaptive denoising, and multiscale gated attention to robustly handle nonstationary, noisy time series.
• It achieves significant reductions in MSE and MAE compared to state-of-the-art models while maintaining a lightweight and computationally efficient design.
• Empirical evaluations demonstrate scalability and robustness across diverse datasets by effectively mitigating distributional shifts and high-frequency disturbances.

TA-SparseMG: Trend-Aware Sparse Forecasting via Multi-Scale Gating for Long-Term Time Series

Introduction and Motivation

Long-term time series forecasting (LTSF) is critical across domains such as energy systems, transportation networks, meteorology, and renewable energy scheduling. The LTSF problem is uniquely challenging: real-world sequences are nonstationary, exhibit abrupt distribution drift, and combine local high-frequency noise with complex periodic dependencies. While Transformer-based and other high-capacity neural models achieve strong results, they typically incur prohibitive parameter and computation cost, making them suboptimal for resource-constrained or real-time deployment.

The lightweight cross-period modeling paradigm, exemplified by SparseTSF, re-structures long sequences via period alignment, performing prediction in a period-phase-aligned representation space to dramatically reduce model complexity. However, this paradigm struggles with distributional shift, noise propagation, and rigid prediction mappings that lack adaptive capacity for diverse temporal patterns. The TA-SparseMG framework, proposed by Liu et al., addresses these deficiencies through three complementary modules: trend-aware reversible instance normalization (TA-RevIN), scale-adaptive gated denoising, and a multiscale gated attention prediction head, each designed to enhance robustness and modeling fidelity in the cross-period paradigm.

Figure 1: Overall framework of the proposed TA-SparseMG. This modular architecture augments a lightweight cross-period backbone with normalization, denoising, and adaptive prediction modules.

Trend-Aware Reversible Instance Normalization

Distributional mismatch between historical and forecast windows is a primary bottleneck for LTSF, particularly in nonstationary environments. Traditional normalization/denormalization schemes perform poorly when restoring forecast-window statistics from strictly historical-window estimates; this can lead to significant performance degradation under mean/volatility drift. The proposed TA-RevIN module augments reversible instance normalization by explicitly modeling within-window statistical drift through partitioning the window, estimating mean and standard deviation variation, and using these estimates to adapt future-window normalization.

This extension enables the model to dynamically extrapolate normalization statistics, which substantially mitigates error propagation from distributional shift. Empirical ablation evidence demonstrates that removing TA-RevIN leads to a consistent increase in mean squared error (MSE), validating its functional contribution to robust long-term prediction.

Scale-Adaptive Gated Denoising

Local high-frequency disturbances are often amplified during periodic reshaping in lightweight cross-period models, resulting in information propagation that hinders extraction of stable periodic trends. The scale-adaptive gated denoising module acts as a preprocessing stage, filtering input representations using a set of temporally multi-scaled convolutional smoothers whose outputs are adaptively fused. The residuals between original and smoothed representations are passed through a channelwise gating mechanism and suppressed proportionally, enabling dynamic denoising that preserves informative local detail while attenuating transient perturbations.

Figure 2: High-frequency energy analysis of the scale-adaptive gated denoising module. The module achieves substantial high-frequency suppression, preserving informative structure across prediction horizons and datasets.

Frequency-domain analysis confirms that the denoising module reduces high-frequency energy—particularly in datasets and prediction regimes afflicted by volatility—without over-smoothing away important temporal structure. This preprocessing is critical for ensuring downstream cross-period modules operate on purified, contextually stable representations.

Multiscale Gated Attention Prediction Module

Period-phase aligned representations extracted from denoised input are mapped to future horizons via a novel multiscale gated attention (MSGA) prediction head. This module replaces the conventional shallow MLP mapping strategy with a dual-path gating mechanism that generates input-conditioned gates from the cross-period representation itself. These gates are fused and applied to modulate the hidden state, which is further combined with an attention-derived modulation factor. The net effect is an adaptive, context-sensitive transformation that significantly improves the modeling of complex, multi-frequency, and nonstationary cross-period dynamics.

Comprehensive ablation and spectral complexity analyses show that the MSGA module provides strong relative MSE gains, especially in medium- and high-complexity scenarios where vanilla MLPs are most deficient.

Figure 3: Relative MSE gain of the full model over the original MLP predictor across spectral-complexity groups. MSGA achieves consistent improvements in medium and high entropy regimes.

Empirical Evaluation

Quantitative Performance

Evaluated across six open-access LTSF datasets (ETTh1, ETTh2, Weather, Solar-Energy, Electricity, Traffic) and four forecast horizons, TA-SparseMG achieves lower MSE and MAE than state-of-the-art lightweight and full-capacity models in the majority of settings, notably outperforming its SparseTSF/MLP baseline in 22/24 MSE and 24/24 MAE test tasks. The architecture introduces only marginal parameter overhead (18.95K total parameters), making it substantially more efficient than Transformer-based models such as PatchTST and iTransformer, yet demonstrably more accurate.

Robustness and Input-Length Adaptability

Sensitivity studies reveal that TA-SparseMG's superior performance persists as look-back window length increases, with monotonic improvements in MSE across most datasets. This suggests effective scaling in long-horizon contexts, highlighting the model's capacity to exploit extended histories while resisting redundancy- and noise-induced degradation.

Figure 4: Sensitivity analysis of look-back length. MSE declines with longer input history, confirming robust long-context modeling.

Efficiency Analysis

Compared to both conventional and advanced lightweight baselines, TA-SparseMG maintains low parameter count, moderate MACs, and efficient runtime/memory usage. The computational cost is only slightly above the original SparseTSF/MLP due to the auxiliary modules, and the efficiency gap relative to Transformer-based architectures is substantial.

Ablation and Mechanism Studies

Ablation analysis systematically removes each of the three key modules, revealing non-redundant, mutually reinforcing gains: TA-RevIN, the denoising module, and the MSGA prediction head all yield measurable average MSE reductions. Further, frequency-domain tracking of the denoising module’s operation confirms that the system balances aggressive high-frequency suppression with preservation of salient information.

Theoretical and Practical Implications

TA-SparseMG demonstrates that parameter-efficient LTSF models can be made robust to distributional shift, locally nonstationary noise, and cross-period complexity through targeted normalization, denoising, and adaptive prediction mechanisms. The model's modular design lends itself to further extension with automated period selection, improved statistical drift estimation, and lightweight intervariable interaction. The balancing act between complexity and robustness showcased here has implications for scalable, real-time, and embedded LTSF across a range of domains.

Conclusion

TA-SparseMG is a rigorous, modular advance in lightweight cross-period long-term time series forecasting. By integrating trend-aware normalization, scale-adaptive gated denoising, and multiscale gated attention prediction within the SparseTSF framework, it demonstrably exceeds the performance of both lightweight and Transformer-based alternatives while maintaining high computational efficiency. The methodology establishes a new standard for LTSF under resource constraints and nonstationary, noisy environments, and its modularity sets the stage for further research in automated adaptation and high-dimensional correlation modeling.