---
title: 'Sundial: Foundation Model for Forecasting'
url: https://www.emergentmind.com/topics/sundial-foundation-model
type: topic
---

# Sundial: Foundation Model for Forecasting

The Sundial Foundation Model designates a class of large-scale, transformer-based architectures for solar and time series forecasting, unifying heterogeneous temporal imaging and remote-sensing data sources into flexible, general-purpose representations. The most cited instantiations arise in heliophysics—integrating Solar Dynamics Observatory (SDO) multi-instrument full-disk data—and in broad time-series applications spanning meteorology, environmental monitoring, and financial domains. Sundial leverages innovations in spatiotemporal transformer backbones, spectral gating, long–short attention, and flow-based probabilistic forecasting, to enable zero-shot predictions, rapid fine-tuning, and downstream scientific modeling tasks.

## 1. Model Architecture: Spatiotemporal Transformer Backbones

Sundial extends the Surya spatiotemporal transformer framework by incorporating patch-based tokenization, dual-scale attention, and spectral features. Given paired input frames 
$\mathbf{X}_{t-1}, \mathbf{X}_t \in \mathbb{R}^{C\times H\times W}$ (e.g., $C=13$, $H=W=4096$ for SDO), Sundial applies signum-log normalization, divides each frame into non-overlapping $P\times P$ patches, and projects each patch to a $D$-dimensional token. Learned Fourier positional embeddings preserve spatial topology [2508.14112]. Model layers alternate spectral gating (frequency-space filtering via FFT) and dual-range attention blocks:

- **Spectral Gating:** Frequency coefficients $\widetilde{\mathbf{X}}$ are modulated by complex learnable weights $W_c$:
  $$
  \widetilde{\mathbf{X}}' = \widetilde{\mathbf{X}} \odot W_c
  $$
  with inverse transform and MLP residual update.

- **Long–Short Attention:** Short-range heads operate on local windows, while long-range heads effect rank-reduced global projections,
  $$
  \mathrm{Attn}_{\rm short}(Q_\Omega, K_\Omega, V_\Omega),\quad
  \mathrm{Attn}_{\rm long}(Q, \bar K, \bar V)
  $$
  with the outputs concatenated and processed via a residual MLP. Outputs are mapped back to image space by a linear decoder.

Sundial supports parameter-efficient adaptation via Low-Rank Adaptation (LoRA), learning low-rank updates on frozen Transformer weights
$$
W = W_0 + \frac{\alpha}{r}BA
$$
for fine-tuning downstream tasks [2508.14112].

## 2. Pretraining Objectives and Probabilistic Forecasting

Sundial is pretrained in two phases:

1. **One-Step Supervised Forecasting:** Minimizing MSE between predicted and observed future frames:
   $$
   \mathcal{L}_{\rm 1-step} = \frac{1}{B}\sum_{b=1}^B \left\| \mathbf{X}_{t+1}^{(b)} - f_\theta(\mathbf{X}_t^{(b)}, \mathbf{X}_{t-1}^{(b)}) \right\|_2^2
   $$
2. **Autoregressive Rollout Tuning:** Successive prediction for $T$ future frames, with cumulative MSE loss:
   $$
   \mathcal{L}_{\rm rollout} = \frac{1}{BT}\sum_{b=1}^B\sum_{k=1}^T \| \mathbf{X}_{t+k}^{(b)} - \hat{\mathbf{X}}_{t+k}^{(b)} \|^2
   $$

*Probabilistic forecasting* is realized by integrating a flow-matching objective ("TimeFlow loss"), pretraining the model to predict transport velocities between source noise and true future patches, without explicit parametric output densities [2502.00816]. At inference, Sundial samples likely trajectories by integrating learned velocity fields over multiple steps, yielding distributions over forecasts.

## 3. Data Modalities, Preprocessing, and Instrument Fusion

Sundial operates on comprehensive, multi-instrument solar datasets (e.g., SDO AIA, HMI, EVE), aggregating:

- EUV and UV imaging (AIA) at 0.6″/px, channels 94–335 Å, 12 s native cadence rebinned/stacked to 12 min.
- Full-disk vector magnetograms and Doppler maps (HMI).
- Sun-integrated EUV spectra (EVE).

Alignment employs limb- and WCS-based co-registration, exposure normalization, and solar-disk masking. Inputs are rescaled per-channel to zero mean and unit variance across the training corpus. Channel stacking and co-temporal pairing fuses all modalities into an $11$-channel tensor for model ingestion (see [2410.02530]). For non-solar time series, Sundial is trained on the TimeBench corpus ($\sim1$T points), covering meteorology, finance, sensor streams, and synthetic benchmarks [2502.00816].

## 4. Zero-Shot and Downstream Evaluation Performance

Sundial demonstrates strong zero-shot adaptation, outperforming supervised LSTM and ARIMA models on LAI prediction once sufficient historical context is provided ($T_{\mathrm{in}} \gtrsim 512$) [2511.20004]. In solar domains, Sundial achieves MSE=0.2198 for one-hour forecasting, compared to 0.5940 for persistence, and delivers 17.8% improvement at 12 hours ahead following rollout tuning [2508.14112]. For flare prediction and segmentation tasks, Sundial+LoRA or adapter heads outperform U-Net and ResNet baselines on intersection-over-union, Dice, and classification scores.

### Example Downstream Metrics

| Task                          | Baseline IoU/Dice | Sundial IoU/Dice |
|-------------------------------|-------------------|------------------|
| AR segmentation (U-Net)       | 0.688 / 0.801     | 0.768 / 0.853    |
| Flare classification (AlexNet)| TSS=0.358         | TSS=0.436        |
| EUV spectra regression (FISM) | MAPE=3.4%         | MAPE=1.48%       |

Sundial’s embeddings also cluster solar phenomena (active regions, flares, quiet Sun) with silhouette scores $\sim0.61$ [2410.02530].

## 5. Scalability, Limitations, and Model Variants

Sundial models scale from 32M to 444M parameters; larger variants yield monotonic improvements in loss and forecasting accuracy [2502.00816]. Inference time is highly efficient, working in sub-millisecond range per prediction. However, very long context is necessary for optimal zero-shot performance, especially on nonstationary or rare event time series [2511.20004]. The current implementations focus on univariate or stacked image forecasting; cross-series correlation and advanced sampling for steep trends remain ongoing research directions.

Limitations cited include:

- High compute and VRAM requirements for training on full-resolution SDO data.
- Conservative trend estimates under rapid change (mode-reversion artifacts).
- Absence of explicit physics-based priors or constraints in current architectures.
- Dependence on long, contiguous observational history for best zero-shot adaptation.

## 6. Fine-Tuning and Adaptation Strategies

Low-Rank Adaptation (LoRA) provides a scalable pathway for downstream adaptation, achieving task transfer within $<5$ million parameters by modifying frozen transformer blocks [2508.14112]. Common adapters include MLP regressors, CNN classifiers, and transformer-based inpainting heads. Fine-tuning protocols maintain base capacity while targeting domain-specific predictive signals.

Prominent downstream tasks include:

- Solar flare probability forecasting (binary classification within 24 h).
- Solar wind speed regression (e.g., RMSE=75.9 km/s, outperforming empirical MHD models).
- Spectral irradiance reconstruction (MAPE <2%).
- Active region segmentation (IoU >0.75).

## 7. Future Directions and Design Recommendations

Building on Surya and Sundial, forthcoming models are encouraged to:

- Integrate more diverse modalities (GOES-SUVI, GONG, DKIST) and multi-scale image patches (8×8, 32×32).
- Employ probabilistic or diffusion-based forecast heads (e.g., CRPS loss).
- Leverage masked spatiotemporal reconstruction and hierarchical attention for global context.
- Utilize physics-informed tokens (rotation rate, Carrington coordinates) for improved interpretability.
- Adopt distributed, mixed-precision computing on A100-class hardware, with sharded data and compressed pipelines.

A plausible implication is the expansion to real-time operational space weather forecasting and cross-domain environmental prediction, contingent on advances in multi-modal data fusion and efficient fine-tuning protocols.

---

**References**  
- Sundial: A Family of Highly Capable Time Series Foundation Models [2502.00816]  
- Surya: Foundation Model for Heliophysics [2508.14112]  
- A Foundation Model for the Solar Dynamics Observatory [2410.02530]  
- Zero-Shot Transfer Capabilities of the Sundial Foundation Model for Leaf Area Index Forecasting [2511.20004]

Source: https://www.emergentmind.com/topics/sundial-foundation-model