SolarCrossFormer: Multimodal Day-Ahead Forecasting

• SolarCrossFormer is a deep learning model that integrates satellite imagery and meteorological time series to forecast solar irradiance at 15-minute intervals for up to 24 hours ahead.
• It employs a graph-structured, cross-modal attention mechanism with locality masking to capture spatial and temporal dependencies between sensors and imagery.
• The model’s robust design handles missing data and dynamic node configurations, demonstrating competitive performance against traditional numerical weather prediction methods.

Searching arXiv for the cited SolarCrossFormer-related papers and nearby work to ground the article. SolarCrossFormer is a name most specifically associated with a deep learning model for day-ahead solar irradiance forecasting that integrates satellite imagery and ground-based meteorological time series to produce probabilistic forecasts at 15-minute resolution for horizons up to 24 hours ahead (Schubnel et al., 19 Sep 2025). In adjacent solar-domain literature, the same label has also been used more loosely as a framing for architectures or control schemes organized around cross-modal, cross-time, cross-variable, or cross-forming interactions, rather than as a single standardized model family. The primary, paper-authored use is the irradiance forecasting system introduced in 2025; related usages include a SolarCrossFormer-like interpretation of Global Cross-Time Attention Fusion for solar flare prediction, a conceptual mapping of PV-Client’s cross-variable transformer for photovoltaic power forecasting, and a practitioner-oriented framing of cross-forming control for solar PV grid-forming inverters (Vural et al., 17 Nov 2025, Gao et al., 2024, He et al., 2024).

1. Nomenclature and scope

The term has a narrow and a broad meaning. In the narrow sense, SolarCrossFormer denotes the model introduced in "SolarCrossFormer: Improving day-ahead Solar Irradiance Forecasting by Integrating Satellite Imagery and Ground Sensors" (Schubnel et al., 19 Sep 2025). In the broader sense, it has been used to describe solar-domain methods that emphasize cross-structure interactions: cross-modal fusion between imagery and sensors, cross-time summarization in multivariate time series, cross-variable attention across weather and PV channels, or cross-forming control laws in power-electronic systems.

Usage	Domain	Relation to the name
SolarCrossFormer	Day-ahead solar irradiance forecasting	Formal model name
Global Cross-Time Attention Fusion (GCTAF)	Solar flare prediction	Described as SolarCrossFormer-like, not paper-authored name
PV-Client	Photovoltaic power forecasting	Connected conceptually to a SolarCrossFormer idea
Cross-forming control	Solar PV grid-forming inverters	Framed as SolarCrossFormer in a practitioner-oriented synthesis

A common misconception is that SolarCrossFormer already denotes a single canonical architecture across solar-energy and space-weather research. The available literature does not support that reading. The irradiance-forecasting paper uses the term as an official model name, whereas the flare-prediction, PV-forecasting, and inverter-control materials use it analogically or interpretively rather than as the authors’ primary designation. This suggests that SolarCrossFormer currently functions both as a specific model identifier and as a broader conceptual shorthand for solar-domain methods built around explicit cross-structure coupling.

2. Canonical formulation in day-ahead solar irradiance forecasting

In its primary sense, SolarCrossFormer addresses probabilistic global horizontal irradiance forecasting for grid integration tasks such as dispatch, trading, and balancing (Schubnel et al., 19 Sep 2025). The task is day-ahead forecasting at 15-minute resolution, with a forecast horizon of $H = 96$ steps and an input history of the past 24 hours, $T = 96$ steps. Inputs are multimodal: time series from a ground-based network of meteorological stations and multispectral satellite imagery. Outputs are quantiles per site and lead time, specifically $\tau \in \{0.05, 0.5, 0.95\}$.

The data sources are explicit. Ground observations come from 127 MeteoSwiss stations across Switzerland and include GHI, DNI, DHI, temperature, wind speed/direction, pressure, and relative humidity. Satellite imagery comes from EUMETSAT MSG-4 with channels IR039, IR087, IR108, and VIS006. The imagery is cropped to $96 \times 96$ around Switzerland, patchified with patch size $4 \times 4$, and linearly projected into the model embedding space. Ground-station series are downsampled from 10 minutes to 15 minutes to align with the satellite cadence.

The architecture is organized around graph-structured multimodal fusion. A temporal Transformer encodes the ground-sensor history for each node. Satellite inputs are represented as patches rather than processed through a ViT backbone. Cross-modal fusion is then performed by cross-attention from nodes as queries to satellite patches as keys and values, with Rotary Positional Encoding and locality-masked multi-head attention. A second attention stage performs nodes-to-nodes message passing using locality-masked heads to capture inter-station dependencies. A temporal Transformer decoder operates over the forecast horizon, with clear-sky GHI provided as positional input, and an MLP head maps decoder states to sitewise quantile forecasts.

The graph structure is implicit rather than fixed. Node–pixel and node–node edges are realized on the fly by attention weights, while masks constrain interactions to concentric spatial rings. Temporal edges are captured by self-attention along the time axis. Because the node dimension is permutation- and cardinality-agnostic, the model can accommodate variable numbers of stations at inference time. This design is central to the model’s stated robustness in real-life operations: new time-series data can be incorporated without retraining, and forecasts can be produced for locations that provide only coordinates.

3. Representation, attention mechanisms, and probabilistic forecasting

SolarCrossFormer combines several positional and masking mechanisms to make multimodal attention spatially and temporally structured (Schubnel et al., 19 Sep 2025). Temporal positional encoding for ground time series is cyclical, based on sin/cos encodings of minute and hour. Spatial structure is injected through Rotary Positional Encoding applied to the queries and keys in cross-attention, using longitude and latitude of stations and satellite patches so that attention scores reflect spatial relationships. Clear-sky GHI is supplied to the decoder as a strong seasonal and location encoding tied to solar geometry.

The core attention operators follow standard Transformer formulations. For a single head,

$$\mathrm{SA}(Q,K,V) = \mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V.$$

SolarCrossFormer then introduces locality masking per attention head:

$$\alpha^{\mathrm{local}}_{…ij}(Q,K) = \frac{\exp\!\left(Q_{…ik}K_{…jk}/\sqrt{d} - \delta M_{ij,a}\right)} {\sum_{j'} \exp\!\left(Q_{…ik}K_{…j'k}/\sqrt{d} - \delta M_{ij',a}\right)},$$

where $M_{ij,a}=0$ inside the head-specific ring $[r_a, R_a]$ and $1$ otherwise, with $T = 96$0. In effect, each head performs distance-aware masked message passing over stations or between stations and satellite patches.

The probabilistic head predicts the $T = 96$1, $T = 96$2, and $T = 96$3 quantiles for each site and horizon. Deterministic training uses mean squared error for the median forecast, while probabilistic training uses the pinball loss

$T = 96$4

Calibration is evaluated through normalized CRPS, and interval quality is tracked with PICP and PINAW. The metric normalization uses $T = 96$5.

A distinctive operational property is the handling of missing data and unseen nodes. Missing or intentionally masked values are replaced by learnable embeddings. During training, about 15% of nodes are fully masked for past sequences and 95% of satellite patches are randomly masked. The model is therefore trained to infer a target node from neighboring nodes and satellite context even when its own history is absent. Coordinate-only forecasting is enabled by combining RoPE-based spatial conditioning with clear-sky GHI in the decoder.

4. Training protocol and empirical performance

The reported training horizon spans nine years, from 2016 to 2024, with training on 2016–2023 and evaluation on 2024 (Schubnel et al., 19 Sep 2025). Hyperparameters are explicit: image size $T = 96$6, patch size $T = 96$7, embedding dimension $T = 96$8, Transformer depth $T = 96$9, heads $\tau \in \{0.05, 0.5, 0.95\}$0, per-head dimension $\tau \in \{0.05, 0.5, 0.95\}$1, MLP ratio $\tau \in \{0.05, 0.5, 0.95\}$2, decoder dimension $\tau \in \{0.05, 0.5, 0.95\}$3, decoder depth $\tau \in \{0.05, 0.5, 0.95\}$4, decoder heads $\tau \in \{0.05, 0.5, 0.95\}$5, decoder per-head dimension $\tau \in \{0.05, 0.5, 0.95\}$6, and dropout $\tau \in \{0.05, 0.5, 0.95\}$7. Training uses gradient accumulation, early stopping, and cosine annealing with warm restarts.

The headline result is a normalized mean absolute error of approximately $\tau \in \{0.05, 0.5, 0.95\}$8 across the 24-hour horizon in 2024, averaged over the 127 Swiss stations. Under quantile training, SolarCrossFormer achieves daytime-only $\tau \in \{0.05, 0.5, 0.95\}$9 and $96 \times 96$0. Under MSE training, SolarCrossFormer records $96 \times 96$1 and $96 \times 96$2, while CrossViVit attains a slightly better $96 \times 96$3. The model is reported to be best from approximately 5 to 24 hours ahead, whereas GCLSTM is slightly better in the first approximately 4 hours due to stronger autoregression on recent local measurements.

The baseline comparisons are nuanced rather than uniformly dominant. With pinball loss, SolarCrossFormer outperforms SolarCrossFormer(no-img), GCLSTM, and CrossViVit in both NRMSE and NMAE. Calibration and sharpness reveal trade-offs: SolarCrossFormer achieves PICP $96 \times 96$4, PINAW $96 \times 96$5, and NCRPS $96 \times 96$6; GCLSTM is closer to nominal coverage with PICP $96 \times 96$7, sharper with PINAW $96 \times 96$8, and marginally better in NCRPS at $96 \times 96$9. The reported interpretation is that SolarCrossFormer is slightly conservative.

Comparison with a commercial numerical weather prediction service is site- and horizon-dependent. For Neuchâtel during October–December 2024, SolarCrossFormer outperforms NWP up to approximately 5 hours ahead and shows lower NMAE from roughly 05:00 to 14:00 with up to about 1% improvement mid-morning, whereas NWP performs better at night and late day due to lack of observations. The horizon-averaged summary gives NWP $4 \times 4$0 and $4 \times 4$1, versus SolarCrossFormer (MSE) $4 \times 4$2 and $4 \times 4$3, and SolarCrossFormer (pinball) $4 \times 4$4 and $4 \times 4$5.

Ablations identify the role of imagery and model simplification. Removing satellite imagery produces only a small average degradation overall, but a larger drop for coordinate-only forecasts and longer horizons where cloud-advection context matters. Vision Transformer backbones do not improve accuracy and increase overfitting; the reported conclusion is that masked attention and shallower cross-attention are more effective.

5. Robustness, deployment, and limitations

The operational emphasis of SolarCrossFormer lies in its ability to function under changing station availability and incomplete observations (Schubnel et al., 19 Sep 2025). Because the encoder and attention stack are permutation- and cardinality-agnostic over nodes, new stations can be added at run time using coordinates alone, without retraining. Dynamic masking during training is intended to amortize this behavior at inference time. When satellite imagery is unavailable, node–pixel cross-attention can be disabled and the model can fall back to node–node attention together with the clear-sky decoder.

Coordinate-only forecasts were evaluated both on three German stations outside the Swiss training footprint and on two Swiss stations intentionally masked during evaluation. The reported daytime-only NMAE values with imagery versus without imagery are: Feldberg $4 \times 4$6 versus $4 \times 4$7, Konstanz $4 \times 4$8 versus $4 \times 4$9, Freiburg $$\mathrm{SA}(Q,K,V) = \mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V.$$0 versus $$\mathrm{SA}(Q,K,V) = \mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V.$$1, Neuchâtel (masked) $$\mathrm{SA}(Q,K,V) = \mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V.$$2 versus $$\mathrm{SA}(Q,K,V) = \mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V.$$3, and Interlaken (masked) $$\mathrm{SA}(Q,K,V) = \mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V.$$4 versus $$\mathrm{SA}(Q,K,V) = \mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V.$$5. With local history available, Interlaken reaches $$\mathrm{SA}(Q,K,V) = \mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V.$$6 and Neuchâtel $$\mathrm{SA}(Q,K,V) = \mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V.$$7. The reported interpretation is that masking primarily affects the first approximately hour, after which the model leverages neighbors and satellite context.

The model is reported to perform best in spring, summer, and autumn. Fog-prone microclimates remain challenging, particularly when neighboring stations are at higher altitude and experience fewer fog episodes. More generally, the paper identifies several limitations: reliance on satellite availability and quality, risk of overfitting in imagery-heavy models without strong regularization, scaling costs with image resolution and node count, and the lack of explicit topography or microclimate modeling. Suggested future directions include incorporating additional sensors such as public cameras and PV production, fusing coarse NWP for stronger late-day horizons, and adding auxiliary future-image forecasting losses to better capture cloud dynamics.

A broader practical implication follows from the reported design: SolarCrossFormer is not merely a multimodal forecaster but a robustness-oriented forecasting framework in which masking, coordinate conditioning, and locality-aware attention are treated as first-class modeling components. This suggests that its main contribution is architectural rather than purely benchmark-driven.

6. Related SolarCrossFormer-like formulations in solar and space-weather research

A second use of the label appears in relation to solar flare prediction, where the underlying paper does not use the name SolarCrossFormer but introduces Global Cross-Time Attention Fusion, or GCTAF, as a transformer-based classifier for multivariate time series derived from solar active regions (Vural et al., 17 Nov 2025). GCTAF uses the SWAN-SF dataset built from SDO/HMI vector magnetograms, with each instance a segment of shape $$\mathrm{SA}(Q,K,V) = \mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V.$$8 where $$\mathrm{SA}(Q,K,V) = \mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V.$$9 magnetic field parameters and a 1-hour cadence. Segments use a 12-hour observation window and a 24-hour prediction window. The distinctive mechanism is a small set of learnable global tokens, with final configuration $$\alpha^{\mathrm{local}}_{…ij}(Q,K) = \frac{\exp\!\left(Q_{…ik}K_{…jk}/\sqrt{d} - \delta M_{ij,a}\right)} {\sum_{j'} \exp\!\left(Q_{…ik}K_{…j'k}/\sqrt{d} - \delta M_{ij',a}\right)},$$0, that query the entire input sequence by cross-attention and are then concatenated back with local tokens before encoder processing and classification. Averaged across four chronological train–test splits, GCTAF reports Accuracy $$\alpha^{\mathrm{local}}_{…ij}(Q,K) = \frac{\exp\!\left(Q_{…ik}K_{…jk}/\sqrt{d} - \delta M_{ij,a}\right)} {\sum_{j'} \exp\!\left(Q_{…ik}K_{…j'k}/\sqrt{d} - \delta M_{ij',a}\right)},$$1, HSS2 $$\alpha^{\mathrm{local}}_{…ij}(Q,K) = \frac{\exp\!\left(Q_{…ik}K_{…jk}/\sqrt{d} - \delta M_{ij,a}\right)} {\sum_{j'} \exp\!\left(Q_{…ik}K_{…j'k}/\sqrt{d} - \delta M_{ij',a}\right)},$$2, GS $$\alpha^{\mathrm{local}}_{…ij}(Q,K) = \frac{\exp\!\left(Q_{…ik}K_{…jk}/\sqrt{d} - \delta M_{ij,a}\right)} {\sum_{j'} \exp\!\left(Q_{…ik}K_{…j'k}/\sqrt{d} - \delta M_{ij',a}\right)},$$3, and TSS $$\alpha^{\mathrm{local}}_{…ij}(Q,K) = \frac{\exp\!\left(Q_{…ik}K_{…jk}/\sqrt{d} - \delta M_{ij,a}\right)} {\sum_{j'} \exp\!\left(Q_{…ik}K_{…j'k}/\sqrt{d} - \delta M_{ij',a}\right)},$$4, outperforming the listed baselines. The paper explicitly states that the model is not called SolarCrossFormer; the connection is conceptual, based on its cross-time global-token mechanism.

A third, conceptually related usage concerns photovoltaic power forecasting through PV-Client, the "Cross-variable Linear Integrated ENhanced Transformer for Photovoltaic power forecasting" (Gao et al., 2024). Here the SolarCrossFormer association is again interpretive rather than nominal. PV-Client replaces cross-time attention with cross-variable attention, treating variables rather than time steps as tokens. The setup uses $$\alpha^{\mathrm{local}}_{…ij}(Q,K) = \frac{\exp\!\left(Q_{…ik}K_{…jk}/\sqrt{d} - \delta M_{ij,a}\right)} {\sum_{j'} \exp\!\left(Q_{…ik}K_{…j'k}/\sqrt{d} - \delta M_{ij',a}\right)},$$5 historical steps, $$\alpha^{\mathrm{local}}_{…ij}(Q,K) = \frac{\exp\!\left(Q_{…ik}K_{…jk}/\sqrt{d} - \delta M_{ij,a}\right)} {\sum_{j'} \exp\!\left(Q_{…ik}K_{…j'k}/\sqrt{d} - \delta M_{ij',a}\right)},$$6 forecast steps, and $$\alpha^{\mathrm{local}}_{…ij}(Q,K) = \frac{\exp\!\left(Q_{…ik}K_{…jk}/\sqrt{d} - \delta M_{ij,a}\right)} {\sum_{j'} \exp\!\left(Q_{…ik}K_{…j'k}/\sqrt{d} - \delta M_{ij',a}\right)},$$7 variables: historical PV power plus weather forecasts for radiation, temperature, humidity, wind speed, and surface pressure. The model uses RevIN, an encoder-only Transformer with two layers, a projection head, and a channel-independent linear trend module for PV power. The reported station-level results are: Jingang, MSE $$\alpha^{\mathrm{local}}_{…ij}(Q,K) = \frac{\exp\!\left(Q_{…ik}K_{…jk}/\sqrt{d} - \delta M_{ij,a}\right)} {\sum_{j'} \exp\!\left(Q_{…ik}K_{…j'k}/\sqrt{d} - \delta M_{ij',a}\right)},$$8 and Acc $$\alpha^{\mathrm{local}}_{…ij}(Q,K) = \frac{\exp\!\left(Q_{…ik}K_{…jk}/\sqrt{d} - \delta M_{ij,a}\right)} {\sum_{j'} \exp\!\left(Q_{…ik}K_{…j'k}/\sqrt{d} - \delta M_{ij',a}\right)},$$9; Xinqingnian, MSE $M_{ij,a}=0$0 and Acc $M_{ij,a}=0$1; Hongxing, MSE $M_{ij,a}=0$2 and Acc $M_{ij,a}=0$3. Ablations report that removing the linear module increases average MSE by $M_{ij,a}=0$4, and removing RevIN increases average MSE by $M_{ij,a}=0$5.

A fourth, much more distant usage appears in power-electronics control rather than statistical forecasting. "Cross-Forming Control and Fault Current Limiting for Grid-Forming Inverters" defines cross-forming as voltage angle forming plus current magnitude forming, and a practitioner-oriented synthesis frames this as a SolarCrossFormer approach for solar PV grid-forming inverters (He et al., 2024). In that framing, the key idea is to preserve voltage angle forming while enforcing $M_{ij,a}=0$6 during fault ride-through. Explicit and implicit implementations are given, together with an equivalent constant virtual impedance interpretation and extensions of transient-stability analysis under current saturation. This usage is terminologically the most remote from the forecasting literature, because it refers to a control concept rather than a Transformer architecture.

Taken together, these related formulations show that SolarCrossFormer has become a cross-domain label applied to disparate solar-domain techniques that share an emphasis on structured coupling across modalities, variables, time positions, or electrical quantities. This suggests a family resemblance rather than a unified method class. The only formal paper title that directly uses the name, however, remains the day-ahead irradiance forecasting model introduced in 2025 (Schubnel et al., 19 Sep 2025).