---
title: Chaos Expansion Networks
url: https://www.emergentmind.com/topics/chaos-expansion-networks
type: topic
---

# Chaos Expansion Networks

Chaos expansion networks are a class of surrogate modeling and operator learning architectures that combine spectral chaos expansions—originally from polynomial chaos expansion (PCE) theory—with neural network or deep probabilistic computational strategies. The principal innovation is the formal embedding of data-driven, spectral (often orthogonal) basis expansions into neural computation graphs, enabling tractable and accurate surrogate modeling, scalable operator learning, and rigorous uncertainty quantification (UQ) in high-dimensional and stochastic settings. These networks generalize classical chaos expansion surrogates by leveraging neural parameterizations and architectural advances to overcome the limitations of traditional PCE in handling arbitrary input distributions, strong dependence, and high parameter dimensionality [2107.10428, 2306.14753, 2502.11835, 2601.01021, 2411.03384, 2507.21273].

## 1. Mathematical Foundations: Chaos Expansions and Neural Parameterizations

Classical polynomial chaos expansions express a response $Y = f(\xi)$ to uncertain inputs $\xi \in \mathbb{R}^d$ as a sum over orthogonal polynomials:
\[
\hat{y}^{(p)}(\xi) = \sum_{i=1}^M c_i \Phi_i(\xi)
\]
where $\{\Phi_i\}$ is an orthonormal basis w.r.t. the law of $\xi$, and $M = \binom{d+p}{p}$ for total order $p$. Classical PCE requires that the distributions of $\xi$ match the Askey scheme (e.g., Hermite for Gaussian). Arbitrary PCE (aPC) replaces this by constructing univariate orthonormal polynomials directly from raw moments, enabling basis construction for inputs with nonparametric or empirical distributions [2107.10428, 2306.14753].

Chaos expansion networks extend this paradigm by making the chaos coefficients $\{c_i\}$ or the basis itself data-adaptive and trainable:
- **Adaptive coefficients:** $C_i(\xi;\theta)$ parameterized by DNNs, yielding surrogates $\hat{y}^{(p)}(\xi)=\sum_{i=1}^M C_i(\xi;\theta)\Phi_i(\xi)$ (Deep aPCE) [2107.10428].
- **Neural basis:** direct replacement of polynomial basis $\{\Phi_i\}$ by learned neural networks $\{\phi_k(\xi;\theta_k)\}$, orthogonalized empirically over the data (Neural Chaos) [2502.11835].
- **Layerwise polynomial expansions:** At each node or layer, responses are expanded in an aPC basis constructed on empirical activations (Deep aPC NN) [2306.14753].
- **Operator parameterizations:** For stochastic PDEs, the chaos coefficients $u_\alpha(t,x)$ of a Wiener chaos expansion are learned with neural operator backbones (FNO, GNO, MLP) [2601.01021, 2411.03384].

This framework admits rigorous computation of output moments, variances, and Sobol indices, and directly encodes an uncertainty quantification pipeline within neural surrogates [2507.21273].

## 2. Learning Algorithms and Semi-Supervision

Chaos expansion networks exploit the orthogonality structure of the chaos basis for efficient training—often via hybrid loss functions blending supervised and unsupervised (semi-supervised) components.

### Deep Adaptive aPC (Deep aPCE) [2107.10428]
- **Labeled loss:** Mean-absolute-error between surrogate and observed outputs over small $N_{gd}$ labeled samples.
- **Unlabeled regularization:** Enforce mean and variance relationships implied by the orthogonality of the chaos basis on large batches of $N_{ce}\gg N_{gd}$ unlabeled data. These constraints ensure property alignment: network-predicted mean matches first chaos coefficient; variance matches sum of higher-order means squared.
- **Optimization:** All terms combined into a global loss and optimized via Adam; the network dynamically refines $C_i(\xi;\theta)$ per input. This approach yields order-of-magnitude reductions in labeled sample costs with preserved or improved surrogate accuracy.

### Neural Chaos Basis Learning [2502.11835]
- **Sequential dictionary learning:** The neural basis functions $\phi_k$ are added sequentially, each time minimizing the residual error and ensuring empirical orthogonality to the previously learned basis vectors.
- **Variants:** Purely continuous (joint optimization of neural $\phi_k$ and coefficient networks $u_k(x)$), or hybrid discrete-continuous with CMD (Canonical Multiplicative Decomposition) substeps for precise orthogonality in sample spaces.
- **Overfitting Control:** Early stopping, small width, normalization, and tracking the train/test generalization gap as the basis grows.

### SPDE Operator Learning [2601.01021, 2411.03384]
- For stochastic (P)DEs, after projecting noises onto a finite Wick-Hermite basis, the chaos coefficients are learned as continuous operator-valued maps from initial conditions and projected noise to space (and time) indices, fully reconstructing solution trajectories in a single network evaluation.

## 3. Architecture: Embedding Chaos Expansions in Deep Models

Chaos expansion networks are instantiated in several architectural patterns.

| Approach                  | Basis Parameterization                      | Adaptive Elements                |
|---------------------------|---------------------------------------------|-----------------------------------|
| Deep aPCE [2107.10428]    | Data-driven aPC basis; DNN on coeffs        | Coefficient network per input     |
| Neural Chaos [2502.11835] | NN stochastic basis, orthogonalized (joint) | Both basis and trunk networks     |
| Deep aPC NN [2306.14753]  | Layerwise empirical aPC basis per layer     | Basis structure adapts per layer  |
| DeepPCE [2507.21273]      | Hierarchical, sum-product circuit of PCEs   | Multi-layer, decompositional      |
| Chaos-Operator [2601.01021, 2411.03384] | Wick-Hermite basis (fixed); operator backbone as NN | All chaos coefficients as operator nets |

In all cases, orthonormality is either prescribed (classical chaos), constructed empirically (aPC, NN-based), or enforced during training via explicit moment-matching constraints.

DeepPCE and similar approaches exploit sum-product circuit architectures to decompose the exponential combinatorics of classical PCE into manageable, hierarchically composed local expansions. This yields scalability to input dimensions $D\sim 10^2$–$10^3$, enabling exact analytic moment and sensitivity calculations in high-dimensional UQ tasks [2507.21273].

## 4. Uncertainty Quantification and Analytical Properties

A key strength of chaos expansion networks is the preservation of closed-form, rigorously computable statistical properties:
- **Output mean:** Coefficient of order-0 basis function.
- **Variance:** Sum of squared coefficients of all higher-order terms.
- **Sobol indices:** Sums over subsets of coefficients with support on specific input variable groupings.
- **Full trajectory sampling:** In SPDE/SDE operator models, a single forward propagation reconstructs the entire stochastic solution via the truncated chaos expansion, bypassing expensive Monte Carlo rollouts [2601.01021, 2411.03384].

These properties are preserved in adaptive and deep settings as long as orthogonality holds at each layer or in the neural basis parameterization. This enables direct, differentiable UQ in complex or operator-valued settings.

## 5. Scalability, High-dimensionality, and Limitations

Traditional PCE-based surrogates are limited by the combinatorial explosion of the basis: for $D$ dimensions and degree $K$, $O(D^K)$ terms appear. Chaos expansion networks address this by:
- **Hierarchical partitioning:** DeepPCE partitions variables into small regions/scopes; expansions are composed via sum-product circuits.
- **Adaptive chaos coefficients:** DNNs dynamically modulate expansion weights, exploiting nonlinearity beyond shallow polynomial truncations.
- **Basis learning:** Neural Chaos and Deep aPC NN learn orthogonal bases directly from raw data, overcomes fixed distributional assumptions or strong dependence.

Empirical studies demonstrate that DeepPCE and Deep aPCE achieve error rates and surrogate fidelity in high-$D$ settings ($D=40$–$100$) that match or exceed classic surrogates, while drastically reducing the need for labeled data or simulations [2107.10428, 2507.21273].

A limitation remains that sample requirements may still grow with dimension in extremely high-dimensional settings ($D\gg 100$). Adaptive or sparse basis selection, as well as further architectural innovations, remain open research areas [2107.10428].

## 6. Applications and Case Studies

Chaos expansion networks are validated on a wide range of forward modeling, UQ, and operator learning problems:
- **Engineering UQ:** Structural reliability (clutch system, satellite frame), high-dimensional stochastic mechanics, thermal–mechanical dynamics. Deep aPCE achieves $\leq 0.3\%$ probability error with $<20$ labels versus much higher costs in UDR or classic PCE [2107.10428].
- **Stochastic PDEs:** SPDE/SDE solution operators for Allen–Cahn, stochastic Navier–Stokes, financial (OU, Heston) processes, and graph Schrödinger bridges [2601.01021, 2411.03384].
- **Operator learning:** PCE and DeepPCE enable uncertainty-aware, operator-valued surrogates for time-dependent and parametric PDEs, offering analytic UQ with performance rivaling DeepONet and FNO [2508.20886, 2507.21273].
- **Surrogate acceleration in Bayesian inference:** PCE surrogates for generative model–based inverse problems (e.g., GPR tomography), utilizing hybrid VAE–PCE–PCA pipelines [2307.04228].

Scalable, accurate analytic inference (mean, variance, Sobol indices) is achieved in all cases, with computational costs far below black-box Monte Carlo or MLP-based UQ [2507.21273, 2508.20886].

## 7. Theoretical Guarantees and Open Challenges

Chaos expansion networks inherit and extend the approximation theory of PCE: rates depend on the smoothness of the target, basis order, and neural network capacity. Theoretical results provide explicit $O(N^{-1/2})$ decay in error for random-feature chaos networks, and practical error rates are reported for a variety of SPDE and surrogate modeling benchmarks [2411.03384, 2107.10428].

Challenges include:
- Adaptive basis truncation and term selection in very high dimensions, to circumvent the exponential scaling of classic PCE.
- Extension to non-Gaussian and non-Hermite chaos (e.g., Lévy noise, Poisson-Charlier chaos), which require new combinatorial and algorithmic frameworks.
- Generalization and stability in highly nonlinear or multimodal surrogate regimes.
- Automated enforcement or optimization of network-based orthogonality constraints in continuous bases [2502.11835].

The integration of data-driven basis learning, deep architectures, and UQ positions chaos expansion networks as a central methodology in scientific machine learning, uncertainty quantification, and stochastic operator approximation.

Source: https://www.emergentmind.com/topics/chaos-expansion-networks