Cross-Domain Effectiveness of Efficient Transformer Variants

Determine whether Transformer variants for time series forecasting that reduce the computational and memory complexity of self-attention are effective across diverse time series domains.

Background

Transformer models for time series forecasting face quadratic complexity with sequence length, prompting many variants that employ sparse attention, locality mechanisms, or frequency-domain operations to reduce cost. While these approaches improve efficiency, their general applicability beyond specific datasets or settings is uncertain.

The survey explicitly notes that despite numerous proposals (e.g., sparse attention and pyramid attention mechanisms), their effectiveness across diverse domains has not been established, leaving a clear open question regarding their robustness and generality.