How Good Can Linear Models Be for Time-Series Forecasting?

Abstract: Time-series forecasting research has been moving steadily toward larger architectures, from specialized transformers to general-purpose foundation models, on the assumption that capacity is what unlocks accuracy. We take the opposite position: most of the gap can be closed at far lower cost by tuning preprocessing rather than scaling models. We use Ridge regression as the testbed, since it has a closed-form solution and interpretable weights, which let the optimal hyperparameters be read off the search directly. We search over context length, local normalization, regularization, and augmentation on eight standard benchmarks and find three patterns. (1) Optimal lookback is strongly series-specific and often non-monotonic in forecast horizon, with fitted power-law exponents ranging from $+0.46$ on ETTm2 to $-0.19$ on Exchange and Traffic, challenging the convention that longer horizons need longer history. (2) Normalizing over a learned trailing fraction of the context, rather than its entirety, is almost universally preferred. (3) Series within the same dataset often disagree on hyperparameters; the optimal degree of cross-series sharing varies from fully shared to fully per-series. The resulting models beat prior linear forecasters on most dataset-horizon entries and exceed Transformer, MLP, and CNN baselines on six of eight benchmarks. The optimized hyperparameters also serve as a diagnostic on the data itself, revealing structures that larger models absorb silently into their learned parameters. We provide an accompanying interactive online demonstration and the code at https://sakanaai.github.io/SearchCast/.

• The paper shows that optimized Ridge regression effectively competes with complex nonlinear models in forecasting accuracy and computational efficiency.
• It demonstrates that fine-tuning context length, normalization scope, data augmentation, and regularization significantly improves performance over fixed-parameter approaches.
• The study reveals that per-series hyperparameter optimization offers critical insights into nonstationarity and channel-specific dynamics in multivariate time-series data.

A Critical Examination of the Potential of Optimized Linear Models in Time-Series Forecasting

Introduction

The paper "How Good Can Linear Models Be for Time-Series Forecasting?" (2606.27282) systematically investigates the limits of linear models, specifically Ridge regression, as baselines for long-horizon time-series forecasting tasks. Contrary to dominant trends favoring increased model complexity via deep architectures—such as Transformers, MLPs, and hybrid approaches—the authors posit that much of the empirical performance gap between linear and nonlinear forecasters is not inherent to model capacity but primarily attributed to neglected or untuned preprocessing. Through exhaustive hyperparameter search over context length, normalization, augmentation, and regularization across eight multivariate benchmarks, the analysis demonstrates that a linear model, when optimally tuned, can rival or outperform much larger and computationally expensive nonlinear models in both predictive performance and computational efficiency.

Methodology and Experimental Protocol

The central experimental design adopts Ridge regression for its closed-form solution, transparency, and interpretability. The search space comprises:

• Context length ($L$): The historical window size for autoregressive input selection, varied logarithmically up to $2048$.
• Normalization scope: Global (dataset-wide) vs. local (only most recent $r \cdot L$ steps) statistics, where $r$ is a search parameter.
• Data augmentation: Additive noise in time or frequency domain, with amplitude determined through search.
• Regularization ($\alpha$): Grid-searched logarithmically at each trial.

Grouped hyperparameter optimization is performed along both time (horizon groups) and cross-series (channel grouping), allowing a flexible trade-off between model heterogeneity and computational tractability.

Evaluation is conducted on widely used, diverse multivariate forecasting benchmarks (ETT, Weather, Traffic, Electricity, Exchange) under consistent, chronological splits and compared against competitive baselines—both linear (OLS, FITS, DLinear) and nonlinear (PatchTST, iTransformer, TimeMixer, TimesNet, Autoformer).

Empirical Findings

Context Length and Forecast Horizon

Optimal context length is neither uniform across datasets nor monotonically increasing with prediction horizon:

• Fitted exponents in the relation $L^* = a H^b$ range from positive (e.g., $b = 0.46$ in ETTm2, longer horizons requiring longer historical context) to negative (e.g., $b = -0.19$ in Exchange and Traffic, reflecting nonstationarity where old history is detrimental).
• Per-series analysis reveals broad heterogeneity within datasets; optimal $L^*$ can span two orders of magnitude across channels.
• Standard practice—fixing $L = 96$ or $2048$0—is shown to be systematically suboptimal, forfeiting up to $2048$1 of attainable accuracy on some datasets.

Figure 1: Per-horizon optimal context-horizon relationships for four time series, demonstrating heterogeneity and non-monotonic scaling.

Figure 2: Optimal lookback $2048$2 vs. forecast horizon $2048$3 fitted by power laws per dataset and per series, exposing sign and magnitude diversity in adaptation behavior.

Local Normalization and Series Heterogeneity

Search nearly always prefers local normalization over a partial trailing fraction of the context window, as opposed to global or full-window statistics. This is interpreted as a necessary adaptation to local distributional shifts and short-term nonstationarity, corroborated by the empirical finding that full-window normalization is rarely, if ever, optimal.

Figure 3: Per-series hyperparameters (local ratio $2048$4 and regularization $2048$5) reveal strong intra-dataset diversity, especially on heterogeneous datasets like Weather.

Strong per-series and per-horizon variations in both normalization scope and regularization are observed, with the optimal amount of cross-series sharing being dataset-dependent.

Figure 4: Series and horizon grouping effects indicate that indiscriminate hyperparameter sharing can degrade accuracy, while moderate grouping offers regularization without losing specificity.

Data Augmentation Effects

Time- and frequency-domain augmentation are frequently selected during hyperparameter optimization, with the amplitude parameter ($2048$6) being highly dataset- and even group-dependent.

Figure 5: Analysis of augmentation selection: both domains are used, with intensity distributions reflecting intra-dataset needs for regularization or robustness.

Forecast Quality and Interpretability

Direct visualizations of forecasts under optimal and global settings demonstrate that the tuned Ridge model can avoid mean-reversion effects prevalent in under-tuned baselines, particularly in nonstationary regimes.

Figure 6: Tuned Ridge forecasts (blue) closely track ground truth dynamics, outperforming global-linear baselines (red) especially on nonstationary datasets.

Model weights reveal interpretable patterns: short-memory emphasis for nonstationary series, explicit periodic anchors matching environmental or operational cycles for quasi-periodic datasets, and dataset-specific utilization of lag structure.

Figure 7: Heatmap of forecast weights by lag and horizon demonstrates selective use of historical data, matching series dynamics.

Main Results: Competitive Accuracy and Efficiency

Across benchmarks and horizons, the optimized Ridge model (SearchCast) achieves:

• Best or second-best MSE on seven out of eight datasets against all linear and nonlinear baselines.
• Margins of $2048$7–$2048$8 against the strongest Transformer (PatchTST) on the majority of benchmarks.
• Comparable or superior accuracy at a fraction of the training cost and parameter count, with the only nonlinear outperformance found in highly channel-rich datasets (Electricity, Traffic), where shared architectures can exploit structural similarity at scale.

Diagnostic Value and Theoretical Implications

The optimized hyperparameters function as a diagnostic instrument revealing input signal properties—stationarity regimes, autocorrelation decay, and heterogeneity of statistical structure—characteristics which remain opaque in the parameterization of deep models. For instance, in datasets where highly variable optimal context lengths are selected, this indicates underlying nonstationary or regime-switching structure, often masked in black-box models.

Appendix analyses confirm that even after stringent deseasonalization, significant long-range linear autocorrelation persists, supporting the claim that extended context can be meaningfully utilized by linear models. Furthermore, rigorous residual analysis using the BDS test reveals that for most benchmarks, the gain from nonlinear architectures lies primarily in marginal structure outside what is needed for MSE optimization; in large-channel datasets, nonlinear benefits may arise from more than just capturing nonlinear dependencies, possibly from channel interaction modeling.

Practical Implications and Future Directions

This work repositions linear models—when equipped with optimized preprocessing—not as trivial baselines but as competitive, interpretable, and computationally efficient tools for long-term time-series forecasting. The results urge a thorough search over preprocessing pipelines before attributing accuracy gains solely to model complexity or hypothesizing inductive biases.

Implications for broader methodological practice include:

• Systematic, dataset-specific tuning of context, normalization, and regularization should be adopted as standard protocol for any competitive benchmarking.
• Automated pipelines such as SearchCast facilitate transparent, rapid exploration of preprocessing–model interactions, and their insights can guide model design, diagnosis, and even the construction of benchmarks themselves.
• For new datasets or application domains, analyzing the hyperparameter landscape of a linear forecaster can inform requirements for capacity, input representation, and the design of data augmentations far more rapidly than exploration with deep models.

Further work should examine joint preprocessing optimization within nonlinear models to establish fair comparisons, and extend the presented search and diagnostic protocols to foundation scales and complex, multi-modal, or event-driven time series.

Conclusion

The findings invalidate the assumption that superior accuracy in time-series forecasting demands substantially more model capacity. With targeted hyperparameter search—exploring context length, partial local normalization, regularization, and augmentation—linear Ridge regression can match or surpass both classical and state-of-the-art deep-learning models on a wide span of public benchmarks. The practical and interpretive advantages, including closed-form training, model transparency, and the diagnostic value of hyperparameter landscapes, position optimized linear models as robust, accessible baselines and analytical tools for both research and deployment in time-series analysis.