- The paper introduces a dual-path architecture that fuses time-domain and wavelet-based frequency features using a Mixture-of-Experts framework.
- It demonstrates significant forecasting improvements, achieving best MSE on 14 out of 16 benchmarks with robust performance in complex, oscillatory scenarios.
- The approach enhances sample efficiency and cross-domain generalization by effectively integrating structured inductive biases through wavelet transforms.
WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting
Introduction
The increasing adoption of Time Series Foundation Models (TSFMs) for universal forecasting tasks is driven by the scalability and cross-domain generalization facilitated through large-scale pretraining on heterogeneous time series corpora. While existing TSFMs predominantly focus on expanding capacity and dataset scale in the time domain—supported by architectures such as Transformer-based models and, more recently, mixture-of-experts (MoE) designs—explicit integration of frequency-domain information remains underexplored at this frontier. The paper "WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting" (2604.10544) proposes an architecture that unifies temporal and wavelet-based frequency-domain signal representations under an MoE regime. This dual-pathway approach is motivated by empirical findings that frequency-domain features enrich temporal modeling, enhancing the capture of periodic, oscillatory, and multi-scale signal structures present in real-world sequences.
Architecture
WaveMoE introduces a dual-path temporal-frequency architecture, where both time-domain and discrete wavelet transformed (DWT) tokens are extracted from the input sequence and processed in parallel but temporally aligned streams. The tokenization pipeline segments the raw time series into patches for standard time-domain processing and applies the DWT to generate multi-scale wavelet coefficients. This design allows for precise correspondence and interaction between time and frequency representations.
Within each path, self-attention modules are employed, with scalability addressed via a sparse attention mechanism—specifically, only the top-k most informative tokens are retained for aggregation, significantly reducing computational overhead over long sequences. Both paths feed into a unified MoE routing mechanism that assigns temporally aligned token pairs (time and frequency) to experts via an MLP gating network. Each expert comprises twin feed-forward branches for path-specific processing, preserving distinct domain inductive biases while leveraging shared temporal localization. Independent prediction heads yield outputs in both the time and wavelet coefficient spaces, with joint supervision during training.
This architecture directly enables explicit fusion and scalable combination of time-frequency information, differentiating WaveMoE from prior foundation models such as Time-MoE, which operate solely in the temporal domain, and recently proposed frequency-modular models, which do not leverage MoE routing for cross-domain specialization.
Empirical Evaluation
WaveMoE is pretrained on an extensively curated version of the Time-300B dataset, augmented for enhanced domain diversity and filtered for signal quality and temporal variability, ensuring robustness across energy, finance, healthcare, IoT, and synthetic data. The experimental suite includes 16 diverse benchmarks covering canonical forecasting domains.
Main quantitative results highlight:
- Best MSE scores on 14 of 16 benchmarks and best MAE scores on 11 out of 16.
- Statistically significant improvements in average forecasting error relative to baselines (Time-MoE, Chronos, Timer, Sundial).
- Notable generalization in complex, high-frequency, and multi-scale regimes, with clear improvements in rapid change and oscillatory scenarios.
- Qualitative visualization analyses (presented in the paper's appendix) demonstrate that WaveMoE provides superior peak-trough localization and amplitude recovery in complex dynamics, while alternatives exhibit smoothing, lag, or attenuation artifacts.
These results substantiate the claim that scaling in the frequency domain, specifically through wavelet-based corpus augmentation and architectural integration, consistently enhances TSFM predictive performance, especially in settings with rich temporal structure.
Theoretical and Practical Implications
WaveMoE's design directly addresses several open challenges in foundation model deployment for time series:
- Inductive Bias Integration: The explicit encoding of wavelet-domain signals introduces structured inductive biases, aiding in the modeling of non-stationary, periodic, or abruptly changing signals that time-domain-only models may underfit or oversmooth.
- Unified Token Routing: The shared MoE router across both paths ensures that model capacity is efficiently allocated to jointly extract temporal and frequency features, supporting specialization without parameter inefficiency.
- Sample and Compute Efficiency: Sparse attention and expert gating mechanisms enable scaling to large parameter counts without prohibitive increases in training/inference cost, a critical property for foundation models on long-range or ultra-high-frequency sequences.
- Cross-domain Generalization: Balanced sampling and quality-driven dataset curation improve the model's robustness and transferability across heterogeneous real-world applications.
Future Research Directions
The results in the paper, while promising, leave several technical frontiers open:
- Exploring more adaptive or learnable time-frequency fusion strategies within the expert blocks, potentially replacing the fixed DWT basis with data-driven frequency partitions.
- Enhancing multivariate dependency modeling, which remains an open challenge given the channel-independent series decomposition employed.
- Developing interpretability and diagnostic tools suited for dual time-frequency representations, moving toward transparent forecasting pipelines for critical applications (e.g., healthcare, energy management).
- Investigating applications beyond standard forecasting, such as anomaly detection, imputation, or sequence segmentation, where frequency-sensitive modeling can further expand TSFM utility.
Conclusion
WaveMoE demonstrates that explicit integration of wavelet-based frequency representations into a scalable, dual-pathway MoE architecture yields statistically robust improvements in time series forecasting across a spectrum of domains and regimes. The work provides empirically validated evidence for the efficacy of frequency-domain scaling in TSFM pretraining and motivates a shift toward unified time-frequency architectures in foundation model research. Future developments will likely extend these mechanisms to adaptive, multivariate, and task-general settings, as the demand for interpretable and robust time series models continues to intensify.
