Deep Polynomial Chaos Expansion

• Deep Polynomial Chaos Expansion is a deep architecture that extends classical PCE to provide exact analytic uncertainty quantification for high-dimensional inputs.
• It employs a hierarchical sum-product circuit that composes local polynomial chaos expansions, overcoming the curse of dimensionality inherent in classical PCE.
• Trained via gradient descent, DeepPCE achieves competitive surrogate accuracy and efficient analytical computation of Sobol indices in complex simulation tasks.

Searching arXiv for the cited DeepPCE paper and closely related polynomial-chaos/deep-PCE works to ground the article in current literature. Deep Polynomial Chaos Expansion (DeepPCE) is a deep, circuit-based generalization of classical polynomial chaos expansion (PCE) designed to preserve PCE’s key advantage—closed-form uncertainty quantification—while scaling to much higher-dimensional inputs than standard PCE can handle. In the formulation introduced in "Deep Polynomial Chaos Expansion" (Exenberger et al., 28 Jul 2025), DeepPCE replaces the flat, monolithic polynomial expansion of classical PCE with a deep structured computation graph inspired by probabilistic circuits. The resulting model is intended for surrogate modeling in physical simulation and uncertainty quantification, where exact computation of quantities such as means, variances, covariances, conditional moments, and Sobol sensitivity indices is central.

1. Classical PCE as the starting point

Classical PCE approximates a response function $f^*(X)$ for a random input vector

$$X=\{X_1,\dots,X_D\}, \qquad p(X)=\prod_{d=1}^D p(X_d),$$

by a finite polynomial chaos expansion

$$f_{\text{PCE}}(X)=\sum_{\alpha\in\mathcal{A}} w_{\alpha}\,\phi_{\alpha}(X),$$

where $\alpha=(\alpha_1,\dots,\alpha_D)$ is a multi-index, $\mathcal{A}$ is the set of allowed multi-indices, and the multivariate basis functions are tensor products of univariate orthonormal polynomials,

$\phi_{\alpha}(X)=\prod_{d=1}^D \phi_{\alpha_d}(X).$

Because the univariate polynomials are orthonormal with respect to the marginals $p(X_d)$, and because the input distribution is factorized, the multivariate tensor basis is orthonormal: $$\mathbb{E}_X[\phi_{\alpha}(X)\phi_{\alpha'}(X)] = \delta_{\alpha,\alpha'}.$$

This orthogonality yields the standard closed-form identities that make PCE attractive in uncertainty quantification. The mean is the constant coefficient,

$\mathbb{E}[f(X)] = w_{\alpha_0},$

where $\alpha_0=(0,\dots,0)$, and the variance is

$$X=\{X_1,\dots,X_D\}, \qquad p(X)=\prod_{d=1}^D p(X_d),$$0

For a subset $$X=\{X_1,\dots,X_D\}, \qquad p(X)=\prod_{d=1}^D p(X_d),$$1, the variance of the conditional expectation is

$$X=\{X_1,\dots,X_D\}, \qquad p(X)=\prod_{d=1}^D p(X_d),$$2

with

$$X=\{X_1,\dots,X_D\}, \qquad p(X)=\prod_{d=1}^D p(X_d),$$3

which yields Sobol indices such as

$$X=\{X_1,\dots,X_D\}, \qquad p(X)=\prod_{d=1}^D p(X_d),$$4

The central limitation is combinatorial growth in the number of basis functions. For polynomial order $$X=\{X_1,\dots,X_D\}, \qquad p(X)=\prod_{d=1}^D p(X_d),$$5 and dimension $$X=\{X_1,\dots,X_D\}, \qquad p(X)=\prod_{d=1}^D p(X_d),$$6, the paper states

$$X=\{X_1,\dots,X_D\}, \qquad p(X)=\prod_{d=1}^D p(X_d),$$7

which is the curse of dimensionality in its classical form. Even with truncation and sparse or adaptive selection, standard PCE is usually limited to only a few dozen dimensions (Exenberger et al., 28 Jul 2025).

2. DeepPCE as a deep generalization of PCE

DeepPCE addresses the tradeoff between analytic tractability and high-dimensional scalability by combining PCE with the compositional structure of probabilistic circuits. The key idea is not to use one global expansion over the full input space, but to place local PCEs at the leaves of a deep sum-product computation graph. Because probabilistic circuits support tractable inference when they are smooth and decomposable, exact statistical quantities can be propagated through the circuit by forward passes once leaf-level moments are known (Exenberger et al., 28 Jul 2025).

A shallow PCE is recovered when the whole input space is handled by a single PCE leaf. DeepPCE instead partitions variables into smaller scopes, builds local PCEs on those scopes, and composes them hierarchically. This allows the model to represent exponentially many multivariate polynomial interactions compactly. The paper explicitly frames this as a deep generalization of PCE analogous to how deep probabilistic circuits generalize shallow mixture models.

The architecture uses three operation types.

Input or PCE layers are local polynomial chaos expansions over subsets $$X=\{X_1,\dots,X_D\}, \qquad p(X)=\prod_{d=1}^D p(X_d),$$8: $$X=\{X_1,\dots,X_D\}, \qquad p(X)=\prod_{d=1}^D p(X_d),$$9 The construction is overparameterized, so multiple input nodes may represent the same scope with different weights.

Product layers multiply outputs of children with disjoint scopes. In tensorized form, for child outputs $$f_{\text{PCE}}(X)=\sum_{\alpha\in\mathcal{A}} w_{\alpha}\,\phi_{\alpha}(X),$$0, the paper uses

$$f_{\text{PCE}}(X)=\sum_{\alpha\in\mathcal{A}} w_{\alpha}\,\phi_{\alpha}(X),$$1

The decomposability condition is essential because it makes integrals factorize.

Sum layers form linear combinations of same-scope child outputs: $$f_{\text{PCE}}(X)=\sum_{\alpha\in\mathcal{A}} w_{\alpha}\,\phi_{\alpha}(X),$$2

The required structural constraints follow standard probabilistic-circuit conditions. Smoothness requires that children of each sum node have identical scope. Structured decomposability requires that children of each product node have disjoint scopes, and identical scopes must be decomposed in the same way. These constraints are precisely what make exact inference possible (Exenberger et al., 28 Jul 2025).

3. Exact moments, conditional quantities, and Sobol indices

A defining property of DeepPCE is that it preserves exact analytic inference for moments and sensitivity measures. The mechanism is structural: expectation distributes over sums, expectation factorizes over decomposable products under factorized inputs, and orthogonality collapses basis interactions at the leaves. As a result, global quantities are obtained by computing leaf statistics and propagating them upward through the circuit by a standard forward pass (Exenberger et al., 28 Jul 2025).

At a leaf PCE, all non-constant basis functions have zero mean, so

$$f_{\text{PCE}}(X)=\sum_{\alpha\in\mathcal{A}} w_{\alpha}\,\phi_{\alpha}(X),$$3

For two leaf PCEs over the same scope,

$$f_{\text{PCE}}(X)=\sum_{\alpha\in\mathcal{A}} w_{\alpha}\,\phi_{\alpha}(X),$$4

and therefore

$$f_{\text{PCE}}(X)=\sum_{\alpha\in\mathcal{A}} w_{\alpha}\,\phi_{\alpha}(X),$$5

The supplement derives the corresponding recursions for the full circuit and concludes that global covariance matrices can be computed exactly by forward propagation from leaf-level second moments.

The model also supports exact conditional inference. For a subset $$f_{\text{PCE}}(X)=\sum_{\alpha\in\mathcal{A}} w_{\alpha}\,\phi_{\alpha}(X),$$6 of variables fixed to values $$f_{\text{PCE}}(X)=\sum_{\alpha\in\mathcal{A}} w_{\alpha}\,\phi_{\alpha}(X),$$7, the conditional expectation of a leaf PCE is

$$f_{\text{PCE}}(X)=\sum_{\alpha\in\mathcal{A}} w_{\alpha}\,\phi_{\alpha}(X),$$8

where

$$f_{\text{PCE}}(X)=\sum_{\alpha\in\mathcal{A}} w_{\alpha}\,\phi_{\alpha}(X),$$9

This quantity also propagates upward through the circuit by a forward pass, giving exact conditional means and conditional covariances.

Sobol sensitivity analysis remains analytically tractable. For output $\alpha=(\alpha_1,\dots,\alpha_D)$0, the first-order index is

$\alpha=(\alpha_1,\dots,\alpha_D)$1

The paper also gives the law-of-total-covariance form

$\alpha=(\alpha_1,\dots,\alpha_D)$2

which allows exact computation of the quantities required for Sobol decomposition. The practical consequence is explicit: one does not need Monte Carlo sampling to estimate these sensitivity measures (Exenberger et al., 28 Jul 2025).

4. Training procedure and computational behavior

Unlike classical PCE, DeepPCE does not admit a closed-form coefficient solve once the model is composed into a deep circuit. It is therefore trained like a neural network using gradient descent on an $\alpha=(\alpha_1,\dots,\alpha_D)$3 loss, with Adam and early stopping in the reported experiments (Exenberger et al., 28 Jul 2025).

The paper highlights several training issues. Leaf polynomial values can vary over several orders of magnitude; product layers can amplify vanishing or exploding behavior in high dimensions; and the model can be sensitive to initialization. To mitigate this, higher-order polynomial weights are initialized with smaller variance than lower-order terms, and batch normalization is applied after each sum layer. Batch normalization can be absorbed into the constant term and scaling of the polynomial expansion at inference time, so it does not break orthogonality or exact tractability.

The complexity argument is qualitative but central. Classical PCE scales combinatorially in $\alpha=(\alpha_1,\dots,\alpha_D)$4 and $\alpha=(\alpha_1,\dots,\alpha_D)$5, whereas DeepPCE uses hierarchical tensorized composition, so that the number of parameters and explicit basis terms grows far more slowly. Exact moment and sensitivity computations remain just forward passes through the circuit once leaf statistics are computed. This is the stated practical reason DeepPCE can handle 100-dimensional sensitivity analysis that would be out of reach for standard full PCE (Exenberger et al., 28 Jul 2025).

A plausible implication is that DeepPCE transfers the representational economy of deep tractable circuits into the polynomial-chaos setting. The paper’s formulation supports this interpretation directly: depth replaces explicit enumeration of a global polynomial basis by hierarchical composition of local polynomial modules.

5. Empirical results on sensitivity analysis and PDE surrogates

The experiments in the DeepPCE paper are organized around two claims: exact high-dimensional sensitivity analysis and competitive surrogate accuracy relative to multi-layer perceptrons (MLPs) (Exenberger et al., 28 Jul 2025).

On a 100-dimensional synthetic benchmark with known structure and analytic Sobol indices, DeepPCE successfully recovers first-order Sobol indices. The comparisons include a shallow PCE with total-order truncation, a shallow PCE with hyperbolic truncation, and an MLP whose Sobol indices are estimated by Monte Carlo. DeepPCE performs on par with the best shallow sparse PCE in identifying variance contributions. The paper also reports that computing Sobol indices for the MLP via Monte Carlo requires a factor of about $\alpha=(\alpha_1,\dots,\alpha_D)$6 more wall-clock time than the analytical DeepPCE computation, corresponding to hours versus about a second.

The PDE surrogate experiments consider two high-dimensional benchmarks with $\alpha=(\alpha_1,\dots,\alpha_D)$7 input fields: Darcy flow, a 2D PDE with random permeability input field, and steady-state diffusion, a 2D PDE with random diffusion coefficient input. These inputs are beyond the practical reach of classical PCE. On holdout test data, DeepPCE achieves predictive performance comparable to MLPs:

• Darcy flow: DeepPCE relative MSE around $\alpha=(\alpha_1,\dots,\alpha_D)$8 versus MLP around $\alpha=(\alpha_1,\dots,\alpha_D)$9.
• Steady-state diffusion: DeepPCE around $\mathcal{A}$0 versus MLP around $\mathcal{A}$1.

The paper notes that DeepPCE can be somewhat sensitive to initialization, with some runs performing worse, but when it converges it matches the MLP closely. In some Darcy examples, DeepPCE predictions appear less noisy than MLP predictions. This suggests that the architectural constraints imposed for tractability do not preclude competitive approximation quality on high-dimensional surrogate tasks.

6. Relation to adjacent polynomial-chaos and deep-learning lines of work

DeepPCE belongs to a broader family of efforts that reinterpret polynomial chaos as a trainable machine-learning model, but it is technically distinct from several nearby approaches.

"Data-driven polynomial chaos expansion for machine learning regression" (Torre et al., 2018) shows that classical PCE can be trained purely from data and can achieve prediction accuracy comparable to neural networks and support vector machines, while retaining uncertainty quantification capabilities. That work is directly relevant to the machine-learning use of PCE, including sparse coefficient estimation, noise robustness, and dependence modeling via copulas and the Rosenblatt transform, but it does not introduce a layered, hierarchical, or compositional deep PCE architecture. Its relevance is foundational rather than architectural.

"Consistency regularization-based Deep Polynomial Chaos Neural Network Method for Reliability Analysis" (Zheng et al., 2022) embeds PCE coefficients as learnable weights in a neural-network-like model and combines a high-order main model with a low-order adaptive PCE auxiliary model under consistency regularization. That framework is close in spirit to DeepPCE because it treats polynomial-chaos coefficients as trainable parameters and uses iterative optimization, but its objective is sample-efficient fitting of high-order PCE for reliability analysis with few labeled data and abundant unlabeled data, rather than exact tractable inference through probabilistic-circuit structure.

"The Deep Arbitrary Polynomial Chaos Neural Network or how Deep Artificial Neural Networks could benefit from Data-Driven Homogeneous Chaos Theory" (Oladyshkin et al., 2023) offers a different deep-PCE viewpoint. It proposes layerwise data-adapted arbitrary polynomial chaos (aPC) bases inside a deep network and interprets conventional deep networks as implicitly using first-degree polynomial representations. In that formulation, each layer becomes a chaos expansion over the previous layer’s outputs, with orthonormal bases constructed from raw moments. This is a deep generalization of aPC, but it is not the circuit-based DeepPCE construction of local PCE leaves combined through smooth and decomposable sum-product structure.

"Polynomial Chaos Expansion for Operator Learning" (Sharma et al., 28 Aug 2025) extends classical PCE into operator learning for PDE surrogates and physics-constrained settings. It is relevant because it demonstrates that PCE can function as a non-neural operator learner with analytic uncertainty quantification, but it remains a tensor-product spectral construction rather than a deep compositional PCE model.

A common misconception is to treat all trainable or multilayer polynomial-chaos methods as equivalent. The available literature indicates otherwise. DeepPCE in the sense of (Exenberger et al., 28 Jul 2025) is specifically defined by hierarchical composition of local PCEs inside a tractable circuit, with exact global statistical inference preserved by smoothness and structured decomposability.

7. Distinctive features, limitations, and implications

The distinctive contribution of DeepPCE is the combination of two properties that are usually separated in high-dimensional surrogate modeling: the analytic uncertainty quantification of classical PCE and the scalability associated with deep compositional architectures (Exenberger et al., 28 Jul 2025). It generalizes standard PCE by localizing polynomial expansions to subspaces, composing them hierarchically with sum and product layers, and using circuit tractability so that exact moments, conditional quantities, covariances, and Sobol indices remain available.

This design also clarifies what DeepPCE is not. It is not a standard neural surrogate to which Monte Carlo post-processing is later attached, because its statistical queries are exact rather than sample-based. It is not classical sparse PCE with aggressive truncation, because it does not attempt to enumerate all global basis functions explicitly. And it is not merely a neuralized coefficient-fitting scheme, because its tractability depends on structural conditions from probabilistic circuits rather than only on parameterization.

Its limitations are also explicit in the paper. Training is no longer a closed-form solve; initialization matters; product compositions can amplify instability; and some runs can perform worse than others. The method therefore exchanges the linear simplicity of shallow PCE for a constrained deep optimization problem. This suggests a broader methodological tradeoff: DeepPCE is most compelling when exact analytic uncertainty quantification must be retained in settings whose input dimension makes flat PCE impractical.

In that sense, DeepPCE can be understood as a bridge between polynomial chaos and deep tractable circuits. Within the published formulation, its central claim is neither generic deep-learning superiority nor universal replacement of classical PCE, but a more specific proposition: high-dimensional surrogate modeling can remain polynomial-chaos-based if the polynomial structure is reorganized into a smooth, decomposable, hierarchical circuit rather than a single global expansion (Exenberger et al., 28 Jul 2025).