Event-aware TSF: Non-stationary Forecasting

• EventTSF is a forecasting approach for non-stationary time series where both endogenous extreme events and exogenous textual events shape predictions.
• It leverages methods such as the Extreme-Adaptive Transformer with sparse attention and a multimodal autoregressive diffusion framework to capture dynamic regime shifts.
• This framework enhances forecasting accuracy and efficiency across domains like hydrology, finance, and traffic by incorporating event-aware mechanisms.

Event-aware Non-stationary Time Series Forecasting (EventTSF) denotes forecasting settings in which the underlying temporal distribution is non-stationary and future behavior is materially shaped by events. In the recent literature represented by two arXiv formulations, “event” is used in two distinct but related senses: as an endogenous extreme event within the time series itself, and as an exogenous natural-language event aligned to temporal segments. The first formulation emphasizes rare but consequential extremes in hydrologic streamflow and introduces the Extreme-Adaptive Transformer (Exformer) (Shrestha et al., 2 Jul 2026). The second formulation treats textual events as an additional modality and proposes an autoregressive diffusion framework, also named EventTSF, for integrating historical time series with future event embeddings (Ge et al., 19 Aug 2025). Taken together, these works situate EventTSF at the intersection of non-stationary modeling, event conditioning, multimodal learning, sparse attention, and uncertainty-aware generative forecasting.

1. Scope and formal problem settings

A central premise of EventTSF is that non-stationarity is not merely background drift but is structurally linked to events. In the hydrologic extreme-event formulation, one observes a multivariate time series $\{x_t\}_{t=1}^T \subset \mathbb{R}^D$ whose marginal distribution and autocovariance may drift over time, together with a small subset of time steps

$$\mathcal{E}=\{\,t:\;x_t^{(d)}\text{ is “extreme” for some }d\in\{1,\dots,D\}\,\},$$

where extremeness is defined by thresholding an outlier score. The construction proceeds by computing an outlier score $s_t$ and assigning

$\ell_t=\mathbf{1}_{\,s_t>\tau\,}\in\{0,1\},$

then aggregating step labels to patch-level indicators $e_m$ after forming patches of size $p$. The forecasting task is to learn

$$f:\;\{x_1,\dots,x_I\}\mapsto \{\hat x_{I+1},\dots,\hat x_{I+O}\},$$

while minimizing prediction error under both normal and extreme regimes (Shrestha et al., 2 Jul 2026).

In the multimodal textual-event formulation, the observed data consist of a univariate time series $X_{1:L}=\{x_l\}_{l=1}^L$ and a sequence of $N$ non-overlapping textual event embeddings $C=\{c_1,c_2,\dots,c_n\}$, where each $$\mathcal{E}=\{\,t:\;x_t^{(d)}\text{ is “extreme” for some }d\in\{1,\dots,D\}\,\},$$0 describes the corresponding time segment $$\mathcal{E}=\{\,t:\;x_t^{(d)}\text{ is “extreme” for some }d\in\{1,\dots,D\}\,\},$$1. The joint dataset is $$\mathcal{E}=\{\,t:\;x_t^{(d)}\text{ is “extreme” for some }d\in\{1,\dots,D\}\,\},$$2, and the problem assumes

$$\mathcal{E}=\{\,t:\;x_t^{(d)}\text{ is “extreme” for some }d\in\{1,\dots,D\}\,\},$$3

Given the last $$\mathcal{E}=\{\,t:\;x_t^{(d)}\text{ is “extreme” for some }d\in\{1,\dots,D\}\,\},$$4 historical pairs $$\mathcal{E}=\{\,t:\;x_t^{(d)}\text{ is “extreme” for some }d\in\{1,\dots,D\}\,\},$$5 and future event embeddings $$\mathcal{E}=\{\,t:\;x_t^{(d)}\text{ is “extreme” for some }d\in\{1,\dots,D\}\,\},$$6, the model forecasts the next $$\mathcal{E}=\{\,t:\;x_t^{(d)}\text{ is “extreme” for some }d\in\{1,\dots,D\}\,\},$$7 values through

$$\mathcal{E}=\{\,t:\;x_t^{(d)}\text{ is “extreme” for some }d\in\{1,\dots,D\}\,\},$$8

This formulation treats events as explicit external context rather than latent anomalies within the series (Ge et al., 19 Aug 2025).

Aspect	Extreme-event formulation	Textual-event formulation
Event source	Thresholded outlier labels from $$\mathcal{E}=\{\,t:\;x_t^{(d)}\text{ is “extreme” for some }d\in\{1,\dots,D\}\,\},$$9	Natural-language embeddings $s_t$0
Core assumption	Normal and extreme regimes differ	$s_t$1
Forecast input	Historical time series only	Historical pairs plus future event embeddings

The coexistence of these formulations clarifies that EventTSF is not tied to a single modality or a single event ontology. Instead, it encompasses forecasting under evolving temporal distributions where event information changes the predictive structure.

2. Endogenous event modeling with the Extreme-Adaptive Transformer

Exformer is an encoder-only Transformer designed to explicitly model temporal dependencies involving both normal and extreme events (Shrestha et al., 2 Jul 2026). Its starting point is the observation that standard self-attention treats all time points uniformly and may therefore underrepresent rare extreme patterns, especially in highly skewed hydrologic streamflow distributions.

The architecture first decomposes the raw input sequence $s_t$2 into seasonal and trend components, $s_t$3 and $s_t$4. The seasonal component is passed to the Exformer encoder, while the trend component is forecasted linearly. The encoder uses a dimension-invariant embedding to produce a multi-channel patch representation, where $s_t$5 is the patch length, $s_t$6, and each temporal patch $s_t$7 receives a binary label $s_t$8. Across $s_t$9 layers, Extreme-Adaptive Attention computes queries, keys, and values from flattened patch embeddings, after which layer outputs are concatenated, projected from length $\ell_t=\mathbf{1}_{\,s_t>\tau\,}\in\{0,1\},$0 to forecast horizon $\ell_t=\mathbf{1}_{\,s_t>\tau\,}\in\{0,1\},$1, and recombined with the trend forecast.

Its core innovation is a sparse attention mechanism with three components. Local attention captures short-term temporal dependencies for normal queries within a window of size $\ell_t=\mathbf{1}_{\,s_t>\tau\,}\in\{0,1\},$2. Stride attention captures periodic temporal dependencies for normal queries using stride step $\ell_t=\mathbf{1}_{\,s_t>\tau\,}\in\{0,1\},$3. Extreme attention is reserved for extreme queries and selectively models event-aware dependencies between normal and extreme streamflow patterns; operationally, the score definition in the details enforces $\ell_t=\mathbf{1}_{\,s_t>\tau\,}\in\{0,1\},$4 for the extreme component. For a single head, these score matrices are defined through masked forms of $\ell_t=\mathbf{1}_{\,s_t>\tau\,}\in\{0,1\},$5, with masking to $\ell_t=\mathbf{1}_{\,s_t>\tau\,}\in\{0,1\},$6 outside the corresponding admissible pattern.

Two masks organize these patterns. A binary mask $\ell_t=\mathbf{1}_{\,s_t>\tau\,}\in\{0,1\},$7 from DozerAttention encodes the union of the Local and Stride patterns, and an extreme-aware mask $\ell_t=\mathbf{1}_{\,s_t>\tau\,}\in\{0,1\},$8 enforces label matching. The final extreme-adaptive score uses $\ell_t=\mathbf{1}_{\,s_t>\tau\,}\in\{0,1\},$9 when $e_m$0 and $e_m$1 when $e_m$2, followed by softmax over the unmasked entries. A notable design point is that there is no learned gating vector; the label-aware mask dynamically switches the sparse attention pattern on a per-query basis. This makes the regime distinction explicit rather than implicit.

The event labels themselves are obtained by fitting a Gaussian Mixture Model to the distribution of $e_m$3, producing an outlier score $e_m$4, then thresholding at percentile $e_m$5. Those labels are pooled from the time-step level $e_m$6 to the patch level $e_m$7 and are used both to construct $e_m$8 and to gate between the normal and extreme attention components.

3. Multimodal EventTSF with autoregressive diffusion and flow matching

The multimodal EventTSF framework addresses settings in which non-stationary dynamics are intertwined with events in other modalities such as texts (Ge et al., 19 Aug 2025). It identifies three fundamental issues: the difficulty of fine-grained synchronization between time-varying discrete textual events and continuous time series, the inherent temporal uncertainty introduced by textual semantics, and the misalignment between textual event embeddings and multi-resolution temporal patterns.

Its forecasting process is autoregressive over segments $e_m$9. At each segment, the model injects one future event embedding $p$0 and emits the next segment $p$1. The generative core is a diffusion-like flow matching scheme. For segment $p$2, define

$p$3

and the linear interpolation

$p$4

Its instantaneous velocity is

$p$5

The model learns a neural velocity field

$p$6

where $p$7 is a learnable latent summarizing previous segments. The corresponding flow matching objective is

$p$8

To handle event-induced uncertainty, the framework does not fix the number of ODE integration steps. Instead it adaptively controls them using the event semantics:

$p$9

The technical description states that large $$f:\;\{x_1,\dots,x_I\}\mapsto \{\hat x_{I+1},\dots,\hat x_{I+O}\},$$0 means more fine-grained steps when an event demands higher precision. In this sense, uncertainty handling is implemented through adaptive numerical integration rather than through a stochastic variance schedule.

Training and inference are both segmentwise. During training, the model samples $$f:\;\{x_1,\dots,x_I\}\mapsto \{\hat x_{I+1},\dots,\hat x_{I+O}\},$$1, constructs $$f:\;\{x_1,\dots,x_I\}\mapsto \{\hat x_{I+1},\dots,\hat x_{I+O}\},$$2, predicts the velocity, accumulates $$f:\;\{x_1,\dots,x_I\}\mapsto \{\hat x_{I+1},\dots,\hat x_{I+O}\},$$3, and updates $$f:\;\{x_1,\dots,x_I\}\mapsto \{\hat x_{I+1},\dots,\hat x_{I+O}\},$$4 via a cross-attention of the denoiser output with $$f:\;\{x_1,\dots,x_I\}\mapsto \{\hat x_{I+1},\dots,\hat x_{I+O}\},$$5. During inference, it first encodes the first $$f:\;\{x_1,\dots,x_I\}\mapsto \{\hat x_{I+1},\dots,\hat x_{I+O}\},$$6 historical segments noise-free to obtain $$f:\;\{x_1,\dots,x_I\}\mapsto \{\hat x_{I+1},\dots,\hat x_{I+O}\},$$7, then generates each future segment from Gaussian noise using the event-conditioned integration schedule.

4. Architectural mechanisms for event awareness

The two EventTSF formulations differ substantially in how event awareness is operationalized. Exformer is a sparse-attention encoder that separates normal-regime sparse attention from extreme-regime sparse attention, while the multimodal EventTSF framework is a generative autoregressive model whose denoiser is a multimodal U-shaped diffusion transformer, denoted M-U-DiT (Ge et al., 19 Aug 2025).

M-U-DiT comprises an encoder stack of $$f:\;\{x_1,\dots,x_I\}\mapsto \{\hat x_{I+1},\dots,\hat x_{I+O}\},$$8 down-sampling Transformer blocks $$f:\;\{x_1,\dots,x_I\}\mapsto \{\hat x_{I+1},\dots,\hat x_{I+O}\},$$9, a single bottleneck block $X_{1:L}=\{x_l\}_{l=1}^L$0, and a decoder stack of $X_{1:L}=\{x_l\}_{l=1}^L$1 up-sampling Transformer blocks $X_{1:L}=\{x_l\}_{l=1}^L$2 with skip-connections from encoder outputs. The noisy segment $X_{1:L}=\{x_l\}_{l=1}^L$3 and previous latent $X_{1:L}=\{x_l\}_{l=1}^L$4 are first fused through cross-attention,

$X_{1:L}=\{x_l\}_{l=1}^L$5

The model also constructs global conditioning vectors

$X_{1:L}=\{x_l\}_{l=1}^L$6

which are added to the block inputs. Standard attention is given by

$X_{1:L}=\{x_l\}_{l=1}^L$7

with MultiHead defined as a concatenation of $X_{1:L}=\{x_l\}_{l=1}^L$8 parallel attention heads.

This contrasts with Exformer’s event mechanism, which is discrete and mask-based rather than continuous and cross-modal. A common misconception is that event-aware forecasting necessarily implies external multimodal conditioning. The literature here shows otherwise: Exformer derives event labels from the series itself, whereas EventTSF uses textual embeddings as explicit exogenous conditioning. Another misconception is that event awareness must be implemented with learned soft gating. Exformer explicitly states that there is no learned gating vector; the switch between Local+Stride and Extreme attention is performed by a label-aware mask (Shrestha et al., 2 Jul 2026).

5. Objectives, optimization, and empirical validation

Exformer is trained end-to-end with a weighted mean-squared error that emphasizes extreme targets:

$X_{1:L}=\{x_l\}_{l=1}^L$9

where $N$0 upweights errors on extreme steps to counter class imbalance, and $N$1 in $N$2 was found effective. Training uses Adam with learning rate $N$3, batch size $N$4, and $N$5 weight decay $N$6, with early stopping on a held-out validation set (Shrestha et al., 2 Jul 2026).

Its experiments use four real-world hydrologic streamflow datasets—Ross, Saratoga, UpperPen, and SFC—each containing 15-minute streamflow and rainfall measurements over three decades, with a 3-day forecast horizon of $N$7 points. It is compared against nine baselines, including Informer, FEDformer, DAN, NEC+, PFformer, iTransformer, NLinear, and DLinear, and evaluated in RMSE and MAPE. The reported 3-day forecasting results are: Ross, 4.20 / 0.05 versus PFformer 4.21 / 0.10; Saratoga, 1.61 / 0.07 versus PFformer 1.69 / 0.10; UpperPen, 0.96 / 0.04 versus PFformer 1.01 / 0.06; and SFC, 15.12 / 0.12 as second best RMSE while PFformer attains 14.98 / 0.18. Exformer achieves lowest RMSE on 3 of 4 sites and best MAPE on all four; ablation shows that removing the Extreme component degrades MAPE by up to 50%. Its attention complexity is reported as $N$8, with approximately $N$9 fewer FLOPs than PFformer and approximately $C=\{c_1,c_2,\dots,c_n\}$0 less memory.

The multimodal EventTSF model uses $C=\{c_1,c_2,\dots,c_n\}$1 summed over all $C=\{c_1,c_2,\dots,c_n\}$2 in a batch, with AdamW, learning rate $C=\{c_1,c_2,\dots,c_n\}$3, weight decay $C=\{c_1,c_2,\dots,c_n\}$4, a OneCycleLR schedule, warmup over 20% of steps, batch size $C=\{c_1,c_2,\dots,c_n\}$5, early stopping on validation every 5 epochs, and dropout decaying from $C=\{c_1,c_2,\dots,c_n\}$6 to $C=\{c_1,c_2,\dots,c_n\}$7 (Ge et al., 19 Aug 2025). Evaluation spans 8 synthetic and real-world datasets, including Atmospheric Physics, Traffic–Public Events, Temperature–Rainfall for Houston, SF, and NY, Electricity–Extreme Weather, and Traffic–News, against 12 baselines from both diffusion-based and deterministic families.

The reported results are broad rather than site-by-site: EventTSF improves average forecasting accuracy by +10.7% across all metrics and is $C=\{c_1,c_2,\dots,c_n\}$8 faster in training efficiency than the second-best model, especially for long horizons with sublinear scaling. For probabilistic forecasting, CRPS decreases by 92.0% on Synthetic, from 0.5769 to 0.0462, and by 59.6% on Atmospherics. For deterministic forecasting, MAE decreases by 82.5% on Synthetic, from 0.3141 to 0.0552, and MSE decreases by 94.4% on Atmospherics. Ablation results report that removing $C=\{c_1,c_2,\dots,c_n\}$9 increases WQL by 516%, replacing M-U-DiT increases MSE by 18.5%, and disabling event-driven timesteps increases CRPS by 15.3%.

6. Interpretation, applications, and emerging research directions

The two formulations suggest complementary views of event awareness. Exformer argues that explicitly separating normal-regime sparse attention from extreme-regime attention avoids diluting rare extreme-event signals among the much more numerous normal-to-normal interactions, while still honoring local and seasonal temporal structure (Shrestha et al., 2 Jul 2026). The multimodal EventTSF framework argues that robust forecasting in non-stationary environments requires fine-grained temporal–textual synchronization, event-controlled uncertainty handling, and multiresolution cross-modal fusion (Ge et al., 19 Aug 2025).

The application profiles are correspondingly different. Exformer is validated in hydrologic streamflow forecasting, where extreme peaks matter for flood monitoring, water resource management, and early warning systems. The same source further states that the paradigm can generalize beyond hydrology to financial risk forecasting such as crashes, meteorological extremes such as heat waves, and domains in which non-stationary series are punctuated by critical rare events. The multimodal EventTSF framework is evaluated in settings where event text is available, including public-event-aware traffic, meteorology, electricity under extreme weather, and traffic with multi-hundred-word news.

A plausible implication is that EventTSF is best understood not as a single model family but as a problem class unified by regime dependence and event conditioning. One branch uses internally derived event labels to restructure attention over the observed sequence; the other uses externally supplied event embeddings to condition a generative forecasting process. Both reject the assumption that non-stationarity can be handled adequately by single-modality, event-agnostic sequence models. In that respect, EventTSF marks a shift from generic temporal modeling toward forecasting systems in which the semantics, rarity, and timing of events become first-class components of the predictive architecture.