Curriculum Chaos Forecasting (CCF)

• CCF is a curriculum pretraining paradigm that organizes synthetic trajectories by intrinsic dynamical complexity for improved chaotic forecasting.
• It employs complexity metrics like the largest Lyapunov exponent and dimensional proxies to progressively train models from simpler to chaotic regimes.
• CCF enhances model transfer in low-data, long-horizon settings by balancing between over-specialized and random mixing pretraining methods.

Curriculum Chaos Forecasting (CCF) is a complexity-aware curriculum pretraining paradigm for sequence models applied to chaotic dynamical systems. Introduced in “Learning to Predict Chaos: Curriculum-Driven Training for Robust Forecasting of Chaotic Dynamics” (Vejendla, 5 Oct 2025), it organizes synthetic training trajectories by intrinsic dynamical complexity rather than by source identity or raw data volume. The curriculum progresses from periodic or quasi-periodic regimes toward increasingly chaotic and higher-dimensional dynamics, using the largest Lyapunov exponent and a dimension proxy as ordering variables. CCF is positioned between two failure modes in machine-learning practice for chaotic forecasting: over-specialization to a single canonical benchmark and indiscriminate mixing of heterogeneous time series. Its stated aim is to improve transfer from synthetic dynamical data to unseen real-world series, especially in low-data and long-horizon settings (Vejendla, 5 Oct 2025).

1. Dynamical motivation and conceptual scope

CCF is motivated by the standard instability of chaotic systems under small perturbations. The core relation used in the paper is

$$\|\delta x(t)\| \approx \|\delta x(0)\| e^{\lambda_{\max} t},$$

where $\lambda_{\max}$ is the largest Lyapunov exponent. In autoregressive forecasting, a small one-step prediction error acts as the next perturbation, so rollout quality deteriorates rapidly in strongly chaotic regimes. The paper therefore frames long-horizon prediction through the estimate

$$T_{\text{valid}} \sim \frac{\ln(\eta/\epsilon)}{\lambda_{\max}},$$

with $\epsilon$ the initial forecasting error and $\eta$ an acceptable tolerance. This formulation makes two routes to better forecasting explicit: reduce initial error, or improve robustness to progressively more chaotic dynamics (Vejendla, 5 Oct 2025).

Within that framing, CCF addresses two opposed but equally problematic practices. One is heavy training on a single canonical chaotic system such as Lorenz-63, which can yield strong in-distribution performance but weak transfer. The other is unstructured large-scale pretraining on many unrelated time series, which the paper argues encourages the model to learn an “average” temporal behavior rather than regime-specific nonlinear structure. CCF is proposed as a middle ground: diverse synthetic pretraining, but with ordering imposed by dynamical systems theory rather than random mixing (Vejendla, 5 Oct 2025).

The scope of CCF is therefore narrower than generic curriculum learning and broader than single-system chaos forecasting. It is not a new sequence architecture, not a new rollout loss, and not a general theory of chaotic predictability. It is a data-scheduling strategy for forecasting models, designed for cross-system transfer.

2. Formal organization of the curriculum

The curriculum is defined from two trajectory-level complexity quantities: the largest Lyapunov exponent $\lambda_{\max}$ and a quantity denoted $d$, presented as attractor dimension in the abstract but implemented as a proxy in the methods. For each trajectory $x$, the paper defines the complexity tuple

$(\lambda_{\max}(x), d(x)).$

It also introduces a composite score

$C(x) = w_1 \lambda_{\max}(x) + w_2 d(x),$

while stating that the experiments mainly use a simpler staged strategy rather than emphasizing the weighted score (Vejendla, 5 Oct 2025).

A common misconception is that CCF computes a formal invariant-set attractor dimension. The methods section does not report Kaplan–Yorke, correlation dimension, or box-counting estimates. In practice, it uses state-space dimension for ODEs and the number of variables after spatial discretization for PDEs as a proxy: $\lambda_{\max}$0 Similarly, $\lambda_{\max}$1 is reported as estimated by a standard algorithm on sliding windows of the trajectory, citing Wolf et al. (1985), but the paper does not provide numerical settings for the estimator (Vejendla, 5 Oct 2025).

The actual scheduler is staged thresholding. At stage $\lambda_{\max}$2, the active subset is described as

$\lambda_{\max}$3

At epoch $\lambda_{\max}$4, mini-batches are sampled uniformly from the current stage subset: $\lambda_{\max}$5 The thresholds increase over training, so the model first sees easier trajectories and later harder ones. The curriculum spans 50 epochs. The exact threshold values, number of stages, and stage boundaries are not reported numerically. The paper therefore describes a staged, effectively bucketed curriculum with uniform sampling within each active bucket, not a continuous or softly weighted schedule (Vejendla, 5 Oct 2025).

3. Synthetic library and transfer benchmarks

CCF is built on a synthetic library of more than 50 dynamical systems. The paper explicitly names Lorenz-63, Rössler, Duffing oscillator, and van der Pol oscillator among lower-dimensional systems, and Lorenz-96, the Kuramoto–Sivashinsky equation, and the Mackey–Glass delay-differential equation among higher-dimensional systems. The complete inventory is not reported. For each system, parameters are swept to generate trajectories covering stable fixed points, periodic orbits, and full chaos, so the library is a family of regimes rather than a static benchmark set (Vejendla, 5 Oct 2025).

The reported synthetic corpus contains over 20,000 unique trajectories, each of length 4,096 time steps, after normalizing the sampling rate. All signals are standardized to zero mean and unit variance. The paper does not report exact parameter ranges, initial-condition sampling, solvers, discretization settings, transient removal, or synthetic train/validation/test splits, so the construction is conceptually clear but not fully specified for exact reproduction (Vejendla, 5 Oct 2025).

Transfer evaluation is conducted on four unseen real-world datasets: Sunspots, NAB Power, MIT-BIH ECG, and El Niño SST. The reported lengths are 3,212, 5,000, 6,500, and 4,500 respectively, with sampling rates of 1/month, 1/5 min, 125 Hz, and 1/month. For each real dataset, the first 70% is used for fine-tuning or training and the remaining 30% for testing. The pretrained models are then fine-tuned on only 5% of the target training split, making the benchmark a low-data transfer setting. The paper notes this protocol but leaves some ambiguity about the remaining fraction of the nominal training split (Vejendla, 5 Oct 2025).

4. Model class, training regime, and evaluation protocol

The main forecasting backbone is a 2-layer GRU with 128 hidden units per layer. Additional architectures evaluated are a simple RNN, a BiGRU, and a Transformer decoder, although detailed hyperparameters for those non-GRU models are not reported. The implementation uses PyTorch with Adam, learning rate $\lambda_{\max}$6, betas $\lambda_{\max}$7, weight decay $\lambda_{\max}$8, batch size 256, fine-tuning learning rate $\lambda_{\max}$9, and a 50-epoch curriculum span (Vejendla, 5 Oct 2025).

Training uses teacher forcing with MSE loss. The method does not introduce a specialized rollout loss, scheduled sampling variant, latent regularizer, or horizon-weighted objective. This is an important point in interpreting the results: the reported gains are attributed to curriculum structure rather than to a new training loss. The paper contrasts CCF with three baselines: Real-Only training on target data alone, Random-Mix pretraining on the same synthetic library without curriculum, and Single-System pretraining on Lorenz-96 with $$T_{\text{valid}} \sim \frac{\ln(\eta/\epsilon)}{\lambda_{\max}},$$0. It explicitly states that all methods use the same GRU architecture and the same total training compute or number of seen samples, isolating ordering rather than compute budget or data volume as the main variable (Vejendla, 5 Oct 2025).

The primary evaluation metric is VPH-10, defined as the average number of iterative forecasting steps before prediction error exceeds 10% of the ground-truth trajectory’s standard deviation. The paper also reports MSE. It does not report MAE, separate RMSE, spectral fidelity measures, calibration metrics, or invariant-statistics metrics beyond the transfer narrative. Main result tables and plots are averaged over 5 random seeds with 95% confidence intervals (Vejendla, 5 Oct 2025).

5. Empirical performance and ablation results

The headline empirical claim is that CCF extends valid prediction horizon by up to 40% over random-order pretraining and more than doubles it relative to training on real-world data alone. On Sunspots, VPH-10 rises from $$T_{\text{valid}} \sim \frac{\ln(\eta/\epsilon)}{\lambda_{\max}},$$1 under Random-Mix and $$T_{\text{valid}} \sim \frac{\ln(\eta/\epsilon)}{\lambda_{\max}},$$2 under Real-Only to $$T_{\text{valid}} \sim \frac{\ln(\eta/\epsilon)}{\lambda_{\max}},$$3 under CCF, while MSE falls to $$T_{\text{valid}} \sim \frac{\ln(\eta/\epsilon)}{\lambda_{\max}},$$4. On NAB Power, the corresponding values are $$T_{\text{valid}} \sim \frac{\ln(\eta/\epsilon)}{\lambda_{\max}},$$5, $$T_{\text{valid}} \sim \frac{\ln(\eta/\epsilon)}{\lambda_{\max}},$$6, and $$T_{\text{valid}} \sim \frac{\ln(\eta/\epsilon)}{\lambda_{\max}},$$7, with CCF MSE $$T_{\text{valid}} \sim \frac{\ln(\eta/\epsilon)}{\lambda_{\max}},$$8. On MIT-BIH ECG, CCF reaches $$T_{\text{valid}} \sim \frac{\ln(\eta/\epsilon)}{\lambda_{\max}},$$9 versus $\epsilon$0 for Random-Mix and $\epsilon$1 for Real-Only. On El Niño SST, CCF reaches $\epsilon$2 versus $\epsilon$3 and $\epsilon$4. These patterns support three stated conclusions: synthetic pretraining is useful, curriculum ordering matters beyond data volume, and single-system pretraining transfers weakly (Vejendla, 5 Oct 2025).

Ablation studies reinforce the central ordering claim. In a held-out synthetic Lorenz-63 setting, a forward curriculum from easy to hard learns faster and reaches the highest VPH-10, whereas a reversed curriculum struggles early and never catches up. The paper interprets this as evidence that starting with predictable systems helps the model learn stable local dynamics before confronting strongly chaotic behavior. It presents that interpretation as a mechanistic hypothesis supported by training curves rather than as a direct representation-level proof (Vejendla, 5 Oct 2025).

The architecture ablation on Sunspots shows gains across multiple backbone families: Simple-RNN improves from $\epsilon$5 to $\epsilon$6, GRU from $\epsilon$7 to $\epsilon$8, BiGRU from $\epsilon$9 to $\eta$0, and Transformer from $\eta$1 to $\eta$2. The reported relative gains are consistently around 37–41%, which the paper uses to support the claim that CCF is architecture-agnostic. A separate robustness experiment adds Gaussian noise to MIT-BIH ECG test inputs and reports that CCF maintains higher VPH-10 at all noise levels and degrades more gracefully than Random-Mix, although exact numbers per noise level are not reported in the supplied text (Vejendla, 5 Oct 2025).

6. Interpretation, neighboring research, and limitations

CCF suggests a practical rule for forecasting chaotic or partially chaotic real-world series: if synthetic pretraining is used, organize trajectories by dynamical complexity and train from simpler regimes to harder ones rather than relying on a single benchmark or random heterogeneous mixing. The main extra computational cost is offline preprocessing: generating the synthetic library and computing complexity labels. The paper reports that this took approximately 48 hours on a modern GPU, after which online training overhead relative to Random-Mix is negligible because the difference is mainly score lookup and stage-restricted sampling (Vejendla, 5 Oct 2025).

The method also has explicit limitations. Its complexity variables are simplified proxies and may fail to capture all relevant dynamics, especially in stochastic systems. The synthetic-to-real gap remains, even with a diverse library. The discrete curriculum may be suboptimal, and the paper explicitly points to continuous or adaptive pacing as future work. It also notes that $\eta$3 can vary locally and that some systems may have effectively infinite-dimensional structure. Several implementation details remain underreported, including exact stage thresholds, full system inventory, numerical solvers, and precise synthetic data generation settings (Vejendla, 5 Oct 2025).

Related work places CCF within a broader but heterogeneous landscape. “Flipped Classroom: Effective Teaching for Time Series Forecasting” studies curricula over teacher forcing and free running on six chaotic systems, showing that curriculum can operate over training dynamics even when the data source is fixed; this is a different axis from CCF’s cross-system complexity ordering (Teutsch et al., 2022). “Chaos as an interpretable benchmark for forecasting and data-driven modelling” provides a generative database of 131 chaotic systems annotated with dynamical quantities such as largest Lyapunov exponent, Lyapunov spectrum, correlation dimension, multiscale entropy, Pesin entropy bound, and Kaplan–Yorke dimension, which suggests a broader substrate for complexity-aware pretraining and evaluation (Gilpin, 2021). “ChaosNexus: A Foundation Model for Universal Chaotic System Forecasting with Multi-scale Representations” argues that cross-system generalization depends more on the diversity of training systems than on sheer data volume, a result that aligns with CCF’s emphasis on principled organization of synthetic dynamics rather than indiscriminate scaling (Liu et al., 26 Sep 2025). Scenario-adaptive approaches on the CTF-4-Science Lorenz benchmark, such as “Adaptive Reservoir Computing for Multi-Scenario Chaotic System Forecasting” and “Divide-and-Conquer Modeling for the CTF-4-Science Lorenz Benchmark,” reach a related conclusion at a different granularity: chaotic forecasting regimes often require different synchronization, validation, or model-selection policies rather than one fixed procedure (Zaregarizi et al., 27 May 2026, Li, 8 Jun 2026). “Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspective” is also adjacent in spirit, because it treats forecasting as evolution on reconstructed attractors with multi-scale memory, though it does not formulate a staged complexity curriculum (Hu et al., 2024).

Taken together, these neighboring lines of work suggest that “curriculum” in chaotic forecasting is not one-dimensional. CCF makes complexity-aware data ordering the primary object, while other studies emphasize training-scale curricula, attractor-aware representations, or scenario-conditioned inference. The distinctive contribution of CCF is to assert that cross-system synthetic pretraining should itself be structured by dynamical complexity, and to show that this organization alone materially extends valid prediction horizons on unseen real-world datasets (Vejendla, 5 Oct 2025).