ChaosNexus: Universal Chaotic Forecasting

• ChaosNexus is a foundation model for universal chaotic system forecasting that integrates a hierarchical multi-scale backbone, Mixture-of-Experts, and attractor-aware regularization.
• It overcomes challenges in chaotic dynamics by addressing sensitivity to initial conditions, multi-scale trajectory structures, and data sparsity through transfer learning.
• The model demonstrates notable improvements in short-term prediction accuracy and long-term attractor fidelity, validated on synthetic benchmarks and global weather forecasting.

ChaosNexus is a foundation model for universal chaotic system forecasting with multi-scale representations. It is designed for zero-shot and few-shot forecasting on novel or data-limited chaotic systems, a setting motivated by the extreme sensitivity of chaotic dynamics to initial conditions, the multi-scale structure of their trajectories, and the scarcity of observational data in many scientific domains. The model is pre-trained on a diverse corpus of chaotic dynamics and combines a hierarchical multi-scale backbone, Mixture-of-Experts specialization, and attractor-aware regularization to improve both short-term forecasting and the fidelity of long-term attractor statistics across synthetic and real-world benchmarks (Liu et al., 26 Sep 2025).

1. Problem formulation and forecasting objective

ChaosNexus addresses a central difficulty in nonlinear dynamics: traditional forecasting methods are typically trained for a specific system and therefore lack the generalization capacity required for zero-shot or few-shot deployment on novel systems. The problem is sharpened by three properties emphasized in the underlying work: extreme sensitivity to initial conditions, complex multi-scale dynamics, and data sparsity. The paper further distinguishes between pointwise trajectory prediction and statistical predictability, arguing that although short-term trajectories are unpredictable at long horizons, chaotic systems remain confined to attractors with invariant statistical and geometric properties (Liu et al., 26 Sep 2025).

Within this formulation, the target of forecasting is not restricted to minimizing short-horizon error. ChaosNexus is also evaluated on long-term attractor faithfulness, using quantities such as correlation dimension error and KL divergence between attractors. This reframes universal chaotic forecasting as a representation-learning problem over families of dynamical systems rather than as repeated single-system fitting.

A common misconception is that universal chaotic forecasting implies exact long-range trajectory recovery. The reported evaluation does not support that interpretation. Instead, the emphasis falls on robust zero-shot transfer, few-shot adaptation, and improved reproduction of long-term attractor statistics, which is a materially different objective from asymptotically exact pointwise prediction (Liu et al., 26 Sep 2025).

2. Architectural design: ScaleFormer, experts, and multi-scale fingerprints

The core architecture is a multi-scale backbone named ScaleFormer. ScaleFormer is described as a U-Net-inspired hierarchical encoder-decoder Transformer with patch merging for downsampling and patch expansion for upsampling. Input trajectories are divided into short temporal patches and embedded with additional polynomial and Fourier features inspired by Koopman theory. Each Transformer block uses dual axial attention, factorizing attention across time and variable axes, and convolutional skip connections preserve high-resolution information throughout the hierarchy (Liu et al., 26 Sep 2025).

A second architectural component is the Mixture-of-Experts mechanism. Each Transformer block is augmented with multiple expert MLPs and a shared expert, while a gating network dynamically selects specialized experts per input patch. The model uses sparse expert selection together with a load balancing loss. The MoE output is given as

$$\mathrm{MoE}(\bar{\mathbf{h}}_p) = \phi_{M+1,p}\,\mathrm{FFN}_{M+1}(\bar{\mathbf{h}}_p) + \sum_{i=1}^{M}\phi_{i,p}\,\mathrm{FFN}_i(\bar{\mathbf{h}}_p).$$

This decomposition separates universal and system-specific processing pathways. In the paper’s interpretation, the shared expert captures dynamics that recur across systems, while the selected experts adapt to local dynamical structure.

The final representational component is multi-scale feature fusion with frequency fingerprinting. A wavelet scattering transform supplies a stable multi-scale frequency fingerprint for each system, based on oscillatory and modulatory behavior in the spectral domain. Decoder outputs from multiple scales are pooled and concatenated with this fingerprint before forecasting. Taken together, ScaleFormer, sparse MoE routing, and frequency fingerprints constitute the defining architectural triad of ChaosNexus (Liu et al., 26 Sep 2025).

3. Pre-training corpus and learning objective

ChaosNexus is pre-trained on an evolutionarily generated corpus of approximately 20,000 synthetic chaotic ODE systems spanning thousands of attractor geometries across a wide dynamic range. This pre-training regime is intended to expose the model to broad variation in governing equations, coupling parameters, and dynamic regimes, thereby inducing transferable representations for out-of-distribution chaotic forecasting (Liu et al., 26 Sep 2025).

The model is trained with a compound objective combining pointwise forecasting, expert balancing, and attractor-aware regularization. The pointwise loss is

$$\mathcal{L}_{\text{mse}} = \frac{1}{B}\sum_{n=1}^{B} \left\| \hat{\mathbf{X}}^{n}_{T+1:T+H} - \mathbf{X}^{n}_{T+1:T+H} \right\|_2^2,$$

and the MMD-based regularization term is

$$\mathcal{L}_{\text{reg}} = \frac{1}{B^2}\sum_{i,j}\kappa(\hat{\mathbf{X}}^i,\hat{\mathbf{X}}^j) + \frac{1}{B^2}\sum_{i,j}\kappa(\mathbf{X}^i,\mathbf{X}^j) - \frac{2}{B^2}\sum_{i,j}\kappa(\hat{\mathbf{X}}^i,\mathbf{X}^j).$$

The final training objective is

$$\mathcal{L} = \mathcal{L}_{\text{mse}} + \lambda_1 \mathcal{L}_{\text{balance}} + \lambda_2 \mathcal{L}_{\text{reg}}.$$

This loss makes explicit that ChaosNexus is optimized ոչ only for trajectory accuracy but also for statistical consistency of generated dynamics. The role of MMD regularization is especially important in the reported experiments, because long-term attractor fidelity is one of the principal evaluation criteria (Liu et al., 26 Sep 2025).

4. Synthetic benchmark performance and attractor fidelity

The principal synthetic benchmark uses roughly 20,000 systems for pre-training and a zero-shot evaluation on 9.3K held-out out-of-distribution systems; the abstract also describes this as a large-scale testbed comprising over 9,000 synthetic chaotic systems. Baselines include Panda, which is itself pretrained on chaotic systems with a PatchTST backbone, and generic time-series foundation models such as Chronos, TimesFM, Timer-XL, Moirai-MoE, and Time-MoE (Liu et al., 26 Sep 2025).

Performance is reported with both pointwise and attractor-level metrics. The symmetric mean absolute percentage error is

$$\text{sMAPE} = \frac{200}{T}\sum_{t=1}^{T} \frac{\|\mathbf{x}_t-\hat{\mathbf{x}}_t\|_1} {\|\mathbf{x}_t\|_1+\|\hat{\mathbf{x}}_t\|_1}.$$

Attractor-level evaluation uses the correlation dimension error $D_{\mathrm{frac}}$, defined as the RMSE between model and true fractal attractor dimension, and the KL divergence $D_{\mathrm{stsp}}$, which compares long-term invariant probability distributions.

On this benchmark, ChaosNexus reports $34.4 \pm 17.0$ for sMAPE@128, $49.7 \pm 15.3$ for sMAPE@512, $0.20 \pm 0.13$ for $$\mathcal{L}_{\text{mse}} = \frac{1}{B}\sum_{n=1}^{B} \left\| \hat{\mathbf{X}}^{n}_{T+1:T+H} - \mathbf{X}^{n}_{T+1:T+H} \right\|_2^2,$$0, and $$\mathcal{L}_{\text{mse}} = \frac{1}{B}\sum_{n=1}^{B} \left\| \hat{\mathbf{X}}^{n}_{T+1:T+H} - \mathbf{X}^{n}_{T+1:T+H} \right\|_2^2,$$1 for $$\mathcal{L}_{\text{mse}} = \frac{1}{B}\sum_{n=1}^{B} \left\| \hat{\mathbf{X}}^{n}_{T+1:T+H} - \mathbf{X}^{n}_{T+1:T+H} \right\|_2^2,$$2. The corresponding Panda results are $$\mathcal{L}_{\text{mse}} = \frac{1}{B}\sum_{n=1}^{B} \left\| \hat{\mathbf{X}}^{n}_{T+1:T+H} - \mathbf{X}^{n}_{T+1:T+H} \right\|_2^2,$$3, $$\mathcal{L}_{\text{mse}} = \frac{1}{B}\sum_{n=1}^{B} \left\| \hat{\mathbf{X}}^{n}_{T+1:T+H} - \mathbf{X}^{n}_{T+1:T+H} \right\|_2^2,$$4, $$\mathcal{L}_{\text{mse}} = \frac{1}{B}\sum_{n=1}^{B} \left\| \hat{\mathbf{X}}^{n}_{T+1:T+H} - \mathbf{X}^{n}_{T+1:T+H} \right\|_2^2,$$5, and $$\mathcal{L}_{\text{mse}} = \frac{1}{B}\sum_{n=1}^{B} \left\| \hat{\mathbf{X}}^{n}_{T+1:T+H} - \mathbf{X}^{n}_{T+1:T+H} \right\|_2^2,$$6, while Chronos-SFT reports $$\mathcal{L}_{\text{mse}} = \frac{1}{B}\sum_{n=1}^{B} \left\| \hat{\mathbf{X}}^{n}_{T+1:T+H} - \mathbf{X}^{n}_{T+1:T+H} \right\|_2^2,$$7, $$\mathcal{L}_{\text{mse}} = \frac{1}{B}\sum_{n=1}^{B} \left\| \hat{\mathbf{X}}^{n}_{T+1:T+H} - \mathbf{X}^{n}_{T+1:T+H} \right\|_2^2,$$8, $$\mathcal{L}_{\text{mse}} = \frac{1}{B}\sum_{n=1}^{B} \left\| \hat{\mathbf{X}}^{n}_{T+1:T+H} - \mathbf{X}^{n}_{T+1:T+H} \right\|_2^2,$$9, and $$\mathcal{L}_{\text{reg}} = \frac{1}{B^2}\sum_{i,j}\kappa(\hat{\mathbf{X}}^i,\hat{\mathbf{X}}^j) + \frac{1}{B^2}\sum_{i,j}\kappa(\mathbf{X}^i,\mathbf{X}^j) - \frac{2}{B^2}\sum_{i,j}\kappa(\hat{\mathbf{X}}^i,\mathbf{X}^j).$$0. The paper summarizes these results as more than $$\mathcal{L}_{\text{reg}} = \frac{1}{B^2}\sum_{i,j}\kappa(\hat{\mathbf{X}}^i,\hat{\mathbf{X}}^j) + \frac{1}{B^2}\sum_{i,j}\kappa(\mathbf{X}^i,\mathbf{X}^j) - \frac{2}{B^2}\sum_{i,j}\kappa(\hat{\mathbf{X}}^i,\mathbf{X}^j).$$1 improvement in correlation-dimension error over the best non-ChaosNexus baseline and more than $$\mathcal{L}_{\text{reg}} = \frac{1}{B^2}\sum_{i,j}\kappa(\hat{\mathbf{X}}^i,\hat{\mathbf{X}}^j) + \frac{1}{B^2}\sum_{i,j}\kappa(\mathbf{X}^i,\mathbf{X}^j) - \frac{2}{B^2}\sum_{i,j}\kappa(\hat{\mathbf{X}}^i,\mathbf{X}^j).$$2 improvement in KL divergence between attractors, emphasizing fidelity of long-term statistics rather than only pointwise accuracy (Liu et al., 26 Sep 2025).

Ablation results reinforce the multi-component design. Removing patch merging and expansion, Mixture-of-Experts layers, MMD regularization, or the frequency fingerprint significantly degrades both short-term accuracy and long-term statistical faithfulness. This suggests that the reported gains are not attributable to a single module but to the interaction between hierarchical temporal representation, expert specialization, and attractor-aware training (Liu et al., 26 Sep 2025).

5. Real-world deployment: global weather forecasting

For real-world evaluation, ChaosNexus is tested on WEATHER-5K, a dataset of 5,672 stations with 10 years of global hourly data. The reported variables are temperature, dew point, wind speed, wind direction, and sea-level pressure. In the zero-shot setting, with no fine-tuning on weather data, ChaosNexus achieves a 5-day mean absolute error below $$\mathcal{L}_{\text{reg}} = \frac{1}{B^2}\sum_{i,j}\kappa(\hat{\mathbf{X}}^i,\hat{\mathbf{X}}^j) + \frac{1}{B^2}\sum_{i,j}\kappa(\mathbf{X}^i,\mathbf{X}^j) - \frac{2}{B^2}\sum_{i,j}\kappa(\hat{\mathbf{X}}^i,\mathbf{X}^j).$$3 for temperature forecasting (Liu et al., 26 Sep 2025).

The same section reports a strong contrast with deep learning baselines such as FEDformer, CrossFormer, PatchTST, and Koopa. Even when those baselines are given 473K in-domain fine-tuning samples, they remain at least $$\mathcal{L}_{\text{reg}} = \frac{1}{B^2}\sum_{i,j}\kappa(\hat{\mathbf{X}}^i,\hat{\mathbf{X}}^j) + \frac{1}{B^2}\sum_{i,j}\kappa(\mathbf{X}^i,\mathbf{X}^j) - \frac{2}{B^2}\sum_{i,j}\kappa(\hat{\mathbf{X}}^i,\mathbf{X}^j).$$4 MAE on the same 5-day horizon. Few-shot adaptation further improves ChaosNexus: with only $$\mathcal{L}_{\text{reg}} = \frac{1}{B^2}\sum_{i,j}\kappa(\hat{\mathbf{X}}^i,\hat{\mathbf{X}}^j) + \frac{1}{B^2}\sum_{i,j}\kappa(\mathbf{X}^i,\mathbf{X}^j) - \frac{2}{B^2}\sum_{i,j}\kappa(\hat{\mathbf{X}}^i,\mathbf{X}^j).$$5 to $$\mathcal{L}_{\text{reg}} = \frac{1}{B^2}\sum_{i,j}\kappa(\hat{\mathbf{X}}^i,\hat{\mathbf{X}}^j) + \frac{1}{B^2}\sum_{i,j}\kappa(\mathbf{X}^i,\mathbf{X}^j) - \frac{2}{B^2}\sum_{i,j}\kappa(\hat{\mathbf{X}}^i,\mathbf{X}^j).$$6 fine-tuning data, performance increases, especially at longer horizons, and this improvement is described as valid across all meteorological variables (Liu et al., 26 Sep 2025).

These results are important because they separate cross-system generalization from classical in-domain forecasting. In the reported setup, the model is not merely competitive after large-scale task-specific retraining; it is already competitive in zero-shot mode and then improves under minimal supervision. A plausible implication is that the pre-trained representation is capturing dynamical regularities that are shared across synthetic chaotic systems and meteorological data, even though the latter is a real-world multivariate forecasting problem.

6. Scaling law, scientific significance, and relation to the wider chaos literature

A central claim of the paper concerns scaling behavior. Increasing the number of trajectories per system in the training corpus offers only marginal improvements, whereas increasing the number of distinct systems yields substantial zero-shot gains. The paper therefore formulates a general principle for scientific foundation models: cross-system generalization stems from the diversity of training systems rather than sheer data volume (Liu et al., 26 Sep 2025).

This scaling result has broader methodological significance. It shifts emphasis from denser sampling of a narrow family of equations toward wider coverage of dynamical regimes, attractor geometries, and coupling structures. This suggests a data-design criterion specific to scientific machine learning for nonlinear systems: breadth of dynamical variation is more informative for transfer than repeated observation of the same underlying mechanism.

In the broader arXiv literature, chaos has been used for signal processing and secure communication through analog Chua-circuit architectures (Li, 2017), analyzed as a synchronization phenomenon in simple and complex networks with applications to secure communications and parameter estimation (Eroglu et al., 2017), and harnessed as a controllable source of variability for sampling from neural generative models (Chaudhuri et al., 2024). ChaosNexus addresses a distinct problem: zero-shot and few-shot forecasting across unseen chaotic systems, with explicit evaluation on both short-term prediction and long-term attractor statistics (Liu et al., 26 Sep 2025).

Taken together, these features place ChaosNexus within an emerging line of research that treats chaos not only as an obstacle to prediction, but also as a domain in which transferable representations can be learned. Its specific contribution is to combine multi-scale sequence modeling, sparse expert specialization, and attractor-aware objectives in a foundation-model framework for universal chaotic forecasting (Liu et al., 26 Sep 2025).