Empirical Assessment of Time-Series Foundation Models For Power System Forecasting Applications

Abstract: Accurate forecasting of electric load and renewable generation is essential for reliable and cost effective power system operations. Recent advances in transformer based and foundation machine learning models, driven by large scale pretraining, increased available data and computation, in addition to architectural innovations, have shown promise in time series forecasting across multiple domains. However, their application to power system forecasting tasks remains largely underexplored. This work presents a comprehensive, empirical benchmark of state of the art time series foundation models, transformer architectures, and deep learning baselines for solar, wind, and load forecasting using the high resolution ARPAE PERFORM dataset for the Electric Reliability Council of Texas (ERCOT) grid. Eight core capabilities are assessed, including zero shot performance, fine tuning efficiency, multivariate input and output handling, horizon sensitivity, generalization to unseen sites, probabilistic forecasting, and context window effects. Models evaluated include TimesFM, Chronos Bolt, MoiraiL, MOMENT, Tiny Time Mixer, Temporal Fusion Transformer, PatchTST, TimeXer, LSTM, and CNN. The manuscript aims to provide clear guidance on when foundation models can provide enhanced renewable and load forecasting capabilities and when other approaches remain the more practical choice for power system operations.

• The paper demonstrates that fine-tuning time-series foundation models drastically improves load, solar, and wind forecasting accuracy using the ERCOT dataset.
• The paper highlights that transformer architectures excel at leveraging long-range dependencies and multivariate inputs for context-aware predictions.
• The paper finds that although foundation models exhibit strong data efficiency, they require domain-specific adaptations to achieve reliable zero-shot performance.

Empirical Benchmarks of Time-Series Foundation Models in Power System Forecasting

Introduction and Motivation

The integration of variable renewable energy (VRE) sources into modern power systems has significantly elevated the need for accurate, scalable, and robust forecasting solutions spanning electric load, solar PV, and wind generation. Advancements in transformer architectures and time-series foundation models (FMs) have begun to disrupt established paradigms, with substantial successes in domains such as natural language processing and computer vision. Despite these innovations, the application and assessment of these models for power grid forecasting tasks remains a nascent area of research. This paper provides a rigorous, unified empirical analysis and benchmark of state-of-the-art time-series FMs, transformer models, and established deep learning methods on the ARPA-E PERFORM ERCOT dataset, targeting both deterministic and probabilistic renewable/load forecasting.

Dataset and Experimental Framework

The high-resolution PERFORM ERCOT dataset comprises 5-minute actuals and forecasted time series for load, solar, and wind operations, encompassing geolocated utility-scale sites and system-level load zones.

Figure 1: Texas ERCOT solar and wind sites relative to installed capacity and load region, visualizing spatial diversity critical for generalization analysis.

Key experimental considerations include:

• Task scope: Day-ahead, intra-day, and short-term horizons; solar/wind per-site, load per-zone.
• Spatial validation: Geographically partitioned train/test splits to directly measure spatial generalization and zero-shot capabilities.

  Figure 2: Geographic split of seen (training) versus unseen (testing) sites, establishing evaluation of zero-shot and spatial generalization performance.

• Probabilistic evaluation: Quantile and mixture-based predictive intervals, analyzed via CRPS and calibration diagrams.
• Multivariate analysis: Models are benchmarked for the ability to ingest/forecast multiple exogenous features and targets.

Evaluated Models and Distinguishing Characteristics

Foundation Models (FMs): Large, pre-trained architectures including TimesFM, Chronos-Bolt, Moirai-L, MOMENT, and Tiny Time Mixer. These leverage vast multi-domain corpora and parameter counts ranging from 1M to 700M, with mechanisms such as mixture-of-Gaussian heads, “any-variate” attention, and token-based seasonality embeddings.

Transformers (Trained from Scratch): Temporal Fusion Transformer (TFT), PatchTST, TimeXer, representing advancements in context gating, exogenous-aware attention, and patch-based efficiency.

Classical Deep Learning Baselines: LSTM and CNN, implemented with fully consistent training pipelines for fair comparison.

Core Results and Analysis

Zero-Shot and Fine-Tuning Regimes

In the zero-shot setting, all foundation models exhibited substantial error (solar nMAE > 9%), signifying that direct transfer from generic pre-training is insufficient for operational-grade accuracy in energy domains. Fine-tuning on even moderate fractions of ERCOT data (20–40%) produced dramatic improvements for FMs, yielding nMAEs that surpassed deep learning baselines trained on 100% of data. This underscores strong data efficiency and transfer adaptability for FMs, provided minimal task-specific adaptation.

Horizon Sensitivity and Context Effects

All model families exhibit increasing error with longer horizons, yet the performance gap narrows as uncertainty accumulates over 24-hour windows. Foundation and transformer models retain a pronounced advantage for short and medium horizons, critical for intra-day market bidding and grid stability.

Increasing the context window of historical input systematically improved forecasting accuracy for solar and wind, with foundation/transformer architectures able to leverage long-range dependencies more effectively than LSTM/CNN, which saturate due to architectural bottlenecks.

Generalization to Unseen Locations

Spatial generalization remains a central challenge. Foundation models such as TimesFM and Chronos-Bolt demonstrated superior performance on unseen sites relative to transformers and deep learning baselines. However, a persistent 1–2% nMAE penalty was observed for unseen locations, indicating potential yet not delivering zero-shot reliability in operational scenarios.

Multivariate and Multitask Training

Inclusion of meteorological and exogenous input variables consistently yielded significant improvements in renewable generation forecasts among transformer and foundation architectures, while yielding negligible gains for LSTM/CNN due to information bottlenecks. Joint multi-task (solar, wind, load) training produced modest additional gains for load but mixed effects for renewables.

Probabilistic Forecasting and Calibration

Foundation models with autoregressive or explicit mixture outputs (notably Chronos-Bolt) yielded the best CRPS scores and exhibited tightly calibrated interval forecasts. LSTM/CNN baselines were distinctly overconfident and poorly calibrated, impairing their use for risk-aware operational decision-making.

Figure 3: Reliability diagram demonstrates that FMs/transformers are well-calibrated, while deep learning baselines exhibit severe overconfidence.

Capability Comparison Across Model Families

The aggregated analysis via performance heatmaps and radar charts demonstrated a profile of tradeoffs:

Figure 4: Performance heatmap compares error and robustness across axes such as data efficiency, horizon, spatial generalization, and calibration.

Figure 5: Radar chart visualization of capability axes—foundation models excel in data efficiency and generalization, while transformer models are superior at leveraging multivariate covariates and are more robust over long horizons.

• FMs: Maximal data efficiency, best fine-tuning/zero-shot potential, but limited operational utility without adaptation.
• Transformers: Robust to context length, extract maximal value from exogenous inputs, and maintain calibration especially for longer-term forecasts.
• Deep learning baselines: Inferior overall, especially regarding cross-site generalization and probabilistic reliability.

Implications and Future Outlook

The empirical evidence presented necessitates a nuanced narrative around the application of modern time-series FMs in power system forecasting. While FMs encode convincing data efficiency and rapid adaptation potential, their unadapted zero-shot capabilities are not currently adequate for grid operations. Transformers trained from scratch remain most effective when site-level calibration, horizon flexibility, and weather covariate integration are critical. Notably, the advances in probabilistic calibration among FMs highlight their promise for integration in stochastic planning and optimization pipelines.

Figure 6: Qualitative time-series plot illustrates intraday solar forecasting accuracy across models, showing fine-tuned FMs tightly track operational ground truth.

Additional research directions suggested by this work include:

• Hybrid adaptation paradigms: Integrating domain-specific pretraining tasks and more sophisticated transfer protocols.
• Model architecture innovation: Enhanced mechanisms for spatial transfer, physical constraints, and uncertainty qualification.
• Operational integration: Development of tools for efficient, reliable deployment of FMs for site commissioning, rapid adaptation, and contingency planning.
• Benchmark standardization: Establishing unified public benchmarks will catalyze progress and more reproducible cross-study comparisons.

Conclusion

This empirical benchmark establishes that time-series foundation models hold significant promise for power system forecasting, especially with respect to data efficiency, fine-tuning flexibility, and probabilistic calibration. Nevertheless, practitioners must be wary of their limited zero-shot readiness for critical grid applications. Transformer architectures maintain a decisive advantage in leveraging complex multivariate signals and providing operational robustness across sites and horizons. Continued advances in specialized architectures, adaptation strategies, and comprehensive benchmarking are essential for realizing the practical potential of foundation models in operational power system management.

---

Reference: "Empirical Assessment of Time-Series Foundation Models For Power System Forecasting Applications" (2604.22077).