Sample-Aware Time-Series Forecasting

• Sample-aware time-series forecasting is a modeling approach that selectively extracts and reweights individual samples to address multi-delay effects, irregular sampling, and distribution shifts.
• It employs advanced architectures—such as TimePro, DAM, and Trio—that dynamically identify critical historical data points, enabling nuanced and robust temporal predictions.
• This approach improves uncertainty quantification and rare-event prediction while offering practical gains in convergence speed and accuracy across diverse, multivariate datasets.

Sample-aware time-series forecasting comprises modeling paradigms, architectures, and training schemes that explicitly leverage information about the individual samples—specific points, windows, or historical histories—in both data and loss landscape, enabling nuanced adaptation to sample heterogeneity, temporal irregularity, multi-delay effects, imbalance regimes, model uncertainty, and distribution shift. Unlike classical methods that flatten or aggregate temporal data, sample-aware methods extract, reweight, select, or adapt representations at the sample or sub-sample level, yielding improved robustness, adaptability, and forecasting accuracy across irregular, multivariate, imbalanced, or domain-shifted time series.

1. Theoretical Motivation and Multi-Delay Problem

Time-series data, especially in multivariate settings, exhibit complex dependencies characterized by variable-specific lags and non-uniform temporal influences: the so-called “multi-delay issue.” In this regime, the effect of a change in predictor $i$ at time $t$ may manifest in the target not at a fixed horizon but at $t+\tau_i$, with $\tau_i$ distinct for each channel. Classical models—e.g., flat convolutional neural networks, recurrent models with shared parameterization, or fixed-window attention—process samples uniformly, neglecting this causal heterogeneity and failing to select the subset of past samples truly informative for each variable and step. Sample-aware methods, by design, adaptively identify and extract those salient historical samples (per variable, per forecast), enabling correct resolution of variable-specific delays and non-trivial temporal representations (2505.20774).

The necessity for sample-awareness is further underscored by:

• Irregular sampling (missingness/gaps, as in clinical or sensor data (Srivastava et al., 2020))
• Imbalanced or rare-event forecasting (e.g., fault detection, financial jumps (Silvestrin et al., 2021))
• Distribution shifts (e.g., regime change post-event (Masserano et al., 2023))
• Domain universality (heterogeneous sampling rates, pattern diversity (Darlow et al., 2024, Liu et al., 24 May 2025))

2. Model Architectures and Sample-Selection Mechanisms

Sample-aware models fall into several architectural classes, each introducing a principled mechanism for sample-level selection, reweighting, or retrieval:

Model class	Sample-aware mechanism	Representative papers
Variable- and time-aware state space models	Learn offsets/selects time samples per variable	TimePro (2505.20774)
Continuous-time, randomly sampled history encoders	Actively sample/encode arbitrary time-value pairs	DAM (Darlow et al., 2024)
Temporal-spatial-sample attention transformers	Retrieve relevant historical example pairs	Trio (Chen et al., 5 Jun 2026)
Dual/rare-vs-common pattern prototype banks	Sample-specific context routing over dynamic banks	DPAD (Yang et al., 23 Jan 2026)
Recursive downsample-convolve-interact multi-resolution nets	Recursive partitioning at sample granularity	SCINet (Liu et al., 2021)
Finite-sample linear predictors	Explicit use of sample-specific error/estimation	(Grigoryeva et al., 2012)

2.1. Adaptive State and Sample Selection (TimePro)

TimePro introduces a Mamba-based pure encoder that, after patching and embedding, alternates between two modules:

• HyperMamba: Scans along the variable axis, then adaptively samples each variable’s own past using a learned set of sample offsets $\delta_h(t,m)$. Offsets are realized via convolutional projection and differentiable interpolation, targeting the M most salient past time points per variable per patch, forming a “variable- and time-aware hyper-state”.
• TimeFFN: Applies a feed-forward network for short-range, per-variable adaptation.

Dynamically, the network bypasses unnecessary processing of all past samples, focusing instead on those critical for current prediction/lightweight SSM state tuning. This is highly effective for resolving multi-delay relationships and has empirically yielded substantial linear-complexity gains on long-term multivariate settings (2505.20774).

2.2. Random and Irregular Sample Encoding (DAM)

DAM (Domain-Agnostic Model) eschews fixed-grid inputs, encoding time series as random sets of $(t_i,v_i)$ pairs, actively sampled from a long-tailed history regime. Temporal embedding combines learned sinusoidal basis functions and value projections. A transformer backbone yields basis function coefficients, leading to a continuous-time expansion

$$f(t) = \sum_{k=1}^K \left[\theta_{k,1}\sin(2\pi\nu_k t) + \theta_{k,2}\cos(2\pi\nu_k t)\right]+\text{affine mark}$$

This design directly handles arbitrary sampling, missing data, and domain heterogeneity, and produces competitive “foundation model” behavior across a diversity of tasks—even with a single univariate DAM (no cross-series modeling) (Darlow et al., 2024).

2.3. Sample-Aware Multi-Axis Attention (Trio)

Trio architectures combine temporal, spatial, and explicitly sample-level (“sample attention”) axes. During training/inference, episodes are reorganized into historical lookback–future pairs plus a current query. After patching and pre-encoding, three attention modules operate sequentially:

• Spatial attention: Inter-channel dependencies.
• Temporal attention: Intra-sequence and patch-level order.
• Sample attention: Cross-window retrieval over S+1 lookback–future pairs, enabling the model to explicitly reason over which past episodes have predictive value for the current example.

This yields strong empirical improvements, especially in cases of non-stationarity or variable delays, and, when used with causally structured synthetic priors, advances toward true generalization in sample-aware forecasting (Chen et al., 5 Jun 2026).

3. Sample-Aware Training and Data-Centric Approaches

Sample-awareness can be instantiated at the training regime, modifying sample presentation, weighting, or loss as a function of sample-specific characteristics:

3.1. Gaussian Loss-Weighted Sampling

A generic issue in time-series training is the heavy-tailed loss distribution: overfitting to outliers and under-exploiting informative moderate-difficulty examples. The Gaussian loss-weighted sampler computes for each sample a dynamic weight

$$w_i = \exp\left( -\frac{(l_i-\mu_l)^2}{2\sigma_l^2} \right)$$

where $l_i$ is the current loss, and $\mu_l, \sigma_l$ represent running mean and std. This weighting penalizes both extremes—very easy and very hard samples—inducing a light-tailed effective sample-distribution, reducing overfitting and improving both convergence speed and MSE/MAE by 1–4% on several benchmarks (You et al., 2024).

3.2. Predictability-Aware Hierarchical Loss (APTF)

APTF defines a per-sample predictability score $t$0 from normalized loss, sorts samples into hierarchical buckets, and applies bucket-dependent weights within a multi-stage scheme. Low-predictability samples (high-loss) are downweighted more strongly as training progresses, while an amortization network cross-validates bucketing to minimize misclassification due to model bias. This hierarchical, sample-aware loss focuses optimization on reliably learnable patterns while still exposing the model to hard instances, yielding notable reductions in both error and training volatility (Zhang et al., 18 Feb 2026).

3.3. Imbalanced and Rare Event Handling

Silvestrin et al. define sample importance weights $t$1 (e.g., magnitude of future variation), using them to stochastically under-sample common/easy samples and retain rare-important points. Several principled sampling regimes—thresholding, stochastic under-sampling, inverse-histogram—are compared. The selection of sampling regime is guided by worst-case RMSE across rebalanced evaluation distributions, achieving favorable rare-event accuracy trade-offs (Silvestrin et al., 2021).

4. Architectural Advances for Contextual and Dynamic Pattern Retrieval

4.1. Dual-Prototype Adaptive Disentanglement (DPAD)

DPAD augments any backbone with a dual-prototype bank: (i) a common-pattern bank parameterized to recurrent global trends/seasonalities, and (ii) a rare-pattern bank adapted to infrequent but critical anomalies. For each input, DPAD retrieves (top-K) common prototypes and, when detected, a rare event prototype, fusing these via context-aware routing into the final forecast. Disentanglement-guided loss ensures clear role distinction, rarity preservation, and diversity among prototypes. DPAD yields 9–13% average MSE reduction on long-term benchmarks, with consistent benefit in outlier and zero-shot settings (Yang et al., 23 Jan 2026).

4.2. Sample Convolution and Interaction Networks (SCINet)

SCINet exploits the property that a time series’ temporal structure is preserved under even/odd downsampling, recursively decomposing the series, extracting convolutional features at each resolution, and interacting them by gating and residual connections. The operation is performed at the sample level (i.e., even and odd-indexed samples), and realignment across layers enables rich representation of hierarchical, cross-scale dependencies. SCINet exhibits efficient $t$2 scaling and achieves superior accuracy on a range of multivariate and spatial-temporal series (Liu et al., 2021).

5. Sample-Aware Methods for Uncertainty Quantification and Distribution Shift

5.1. Bayesian Dictionary Scenario Forecasting

Sample-aware scenario forecasting by Bayesian dictionary learning involves learning a joint overcomplete dictionary for $t$3, then performing Gibbs sampling over sparse, sample-specific codes $t$4, yielding a coherent empirical ensemble of forecast trajectories per input. This framework allows entire trajectories—not just marginal intervals—to be sampled and reasoned over, permitting application-specific risk assessment, scenario coverage, and event probabilities. Empirical studies confirm parity or superiority to physical model ensembles on temperature forecasting (Veeramachaneni, 2012).

5.2. Adaptive Sampling under Distribution Shift

Masserano et al. introduce a meta-level sample selection mechanism: learn a discrete sampling distribution $t$5 over historical time steps, tailored by Bayesian optimization to minimize held-out validation loss post-distribution shift. Lightweight retraining is conditioned on adaptively sampled windows. This approach enables base models to focus their learning on the historical segments most relevant to the new regime, yielding statistically significant error reductions under synthetic mean-shift and corruption, as well as real-world epidemic breakpoints (Masserano et al., 2023).

6. Robustness to Irregular Sampling and Missing Data

Sample-aware forecasting is particularly advantageous for irregular, sparse, and missing data regimes. Models can:

• Represent and propagate explicit sampling times or elapsed intervals as inputs per sample and channel, as in GRU-based models with mask and $t$6-features. This informs the hidden state of the true clock rather than synthetic binning, and enables simultaneous prediction of both value and event time (Srivastava et al., 2020).
• Encode time–value pairs without imposing a fixed grid, as in DAM, directly processing sets of observations at arbitrary times (Darlow et al., 2024).
• Apply forward-fill imputation with learned masks, ensuring networks learn to discount or ignore stale imputed values contextually.

7. Practical Impact and Future Directions

Sample-aware time-series forecasting, via sample selection, weighting, and explicit context adaptation, has demonstrably advanced the state of the art across diverse settings:

• Long-term multivariate, multi-delay benchmarks (2505.20774)
• Universal/domain-agnostic forecasting (Darlow et al., 2024, Liu et al., 24 May 2025)
• Robust rare-event prediction and imbalanced regimes (Silvestrin et al., 2021)
• Fast convergence and improved robustness under noisy, heterogeneous data (You et al., 2024, Zhang et al., 18 Feb 2026)
• Explicit uncertainty quantification (Veeramachaneni, 2012)

Core limitations include computational overhead from sample-loss tracking (You et al., 2024), one-time training resource intensity for foundation models (Darlow et al., 2024), and open questions regarding optimal memory organization or retrieval in large-scale sample-attention (Chen et al., 5 Jun 2026). Integrating cross-series dependencies and extending sample-aware architectures to complex spatial or graph-temporal structures represent promising future research avenues.