Data-Efficient Non-Gaussian Semi-Nonparametric Density Estimation for Nonlinear Dynamical Systems

Abstract: Accurate representation of non-Gaussian distributions of quantities of interest in nonlinear dynamical systems is critical for estimation, control, and decision-making, but can be challenging when forward propagations are expensive to carry out. This paper presents an approach for estimating probability density functions of states evolving under nonlinear dynamics using Seminonparametric (SNP), or Gallant-Nychka, densities. SNP densities employ a probabilists' Hermite polynomial basis to model non-Gaussian behavior and are positive everywhere on the support by construction. We use Monte Carlo to approximate the expectation integrals that arise in the maximum likelihood estimation of SNP coefficients, and introduce a convex relaxation to generate effective initial estimates. The method is demonstrated on density and quantile estimation for the chaotic Lorenz system. The results demonstrate that the proposed method can accurately capture non-Gaussian density structure and compute quantiles using significantly fewer samples than raw Monte Carlo sampling.

• The paper presents a novel SNP density estimation framework that uses a Hermite polynomial expansion modulated by a Gaussian reference to ensure positivity and normalization.
• It employs Monte Carlo sampling with a convex relaxation strategy for efficient maximum likelihood estimation, achieving accurate density and quantile computations with far fewer samples.
• The method is validated on the chaotic Lorenz system, effectively capturing complex, multi-modal distributions and providing a robust tool for uncertainty quantification in nonlinear dynamics.

Data-Efficient Non-Gaussian Semi-Nonparametric Density Estimation for Nonlinear Dynamical Systems

Introduction

The accurate estimation of non-Gaussian probability density functions (PDFs) for states governed by nonlinear, potentially chaotic dynamical systems is a central challenge in uncertainty quantification, nonlinear estimation, and robust control. Traditional approaches such as Gaussian assumptions or kernel density estimation are inadequate in high-dimensional, nonlinear scenarios due to model mismatch and the curse of dimensionality. The paper "Data-Efficient Non-Gaussian Semi-Nonparametric Density Estimation for Nonlinear Dynamical Systems" (2604.09375) introduces a seminonparametric (SNP) density estimation framework leveraging Monte Carlo (MC) sampling and a Hermite polynomial basis to efficiently approximate complex, non-Gaussian distributions with strong guarantees on positivity and normalization.

Background and Problem Formulation

Let $x_k$ denote a $d$-dimensional state vector evolved through a nonlinear discrete-time system and additional parametric uncertainty $\psi$. The key task is to reconstruct the evolved PDF and, when needed, the cumulative distribution function (CDF) of $x_k$ from MC propagated samples for downstream estimation, control, or decision-making. Under nonlinear dynamics, initial Gaussian statistics cannot be assumed to persist—the necessity for a flexible, tractable, and data-efficient density estimator is acute.

Previous approaches, including Gaussian mixtures, nonparametric kernel estimators, particle methods, and moment-based expansions (Edgeworth/Gram-Charlier), either become impractical in high-dimensions, are computationally intensive, or fail to guarantee positivity of the PDF under polynomial truncation. SNP, or Gallant–Nychka, densities offer an overview: a Hermite polynomial expansion modulated by a Gaussian reference, squared to enforce positivity, and normalized accordingly.

Semi-Nonparametric Densities: Theoretical Construction

The SNP density is defined as:

$p(z) = \frac{\phi(z)P(z)^2}{S},$

where $\phi(z)$ is a reference Gaussian, $P(z)$ is a Hermite polynomial series (of order up to $K$), and $S$ is a normalization constant:

$S = \mathbb{E}_\phi[P(z)^2].$

By construction, this ensures $d$0 everywhere. The coefficients of $d$1 are determined not by the moments/cumulants of the data but via maximum likelihood estimation (MLE) based on observed samples.

For the multivariate case, a full Hermite polynomial basis with cross terms is used, indexed over all multi-indices $d$2 with $d$3, allowing for the representation of high-order correlations and nonlinear dependencies.

Efficient MLE via Monte Carlo and Convex Relaxation

MLE for the coefficients is complicated by the non-convexity of the objective and the presence of intractable expectation integrals. These integrals are numerically approximated via MC sampling, and the optimization target for the univariate case is:

$d$4

where $d$5 is the Hermite basis vector evaluated at sample $d$6. The multivariate generalization follows analogously using the multivariate Hermite basis.

To mitigate convergence to suboptimal local minima and to provide a data-efficient initialization, the paper introduces a convex relaxation: the absolute value term within the log is removed, and the normalization log term is replaced with a quadratic upper bound. The problem is then solved for both positive and negative domains, and the resulting solutions serve as effective warm starts for the full nonlinear optimization.

Figure 1: Objective values comparison across different polynomial orders with 100 MC samples.

Empirical results confirm that the positive branch of the convex relaxation closely approximates the nonlinear optimum, resulting in small objective changes post-refinement and thus establishing the efficacy of the initialization strategy.

Application to Nonlinear Dynamics: Lorenz System

The utility of the SNP estimator is demonstrated on the chaotic Lorenz system, where the state distribution experiences strong nonlinear deformation and becomes highly non-Gaussian with multiple modes. Propagated MC samples are shown to strongly deviate from Gaussianity, motivating the need for expressive density approximators.

Figure 2: Monte Carlo Point Cloud.

Figure 3: Monte Carlo point cloud projections of 1000 sample propagations.

A 1000-sample SNP density using a $d$7th-order multivariate Hermite polynomial accurately captures the bi-modal structure aligned with the Lorenz attractors.

Figure 4: SNP density generated from 1000 MC points with $d$8.

High-fidelity agreement is observed between the SNP-estimated marginal densities and MC sample projections, especially for regions of high probability mass, even with substantially fewer samples than required for histogram-based MC density estimates.

Quantile and Probability Region Computation

An analytical expression for the CDF is derived using properties of Hermite polynomials, leveraging their orthogonality and recurrence relations. This enables rapid evaluation of interval probabilities (quantiles) and the probability content of multi-dimensional regions (e.g., boxes) without resorting to numerical integration or brute-force sampling.

Figure 5: Density estimate with Monte Carlo cloud and constraint box.

Through systematic comparison, SNP-based quantile evaluation attains accuracy similar to MC evaluation with $d$9 samples, but using orders of magnitude fewer propagated samples for the density fit.

Figure 6: Quantile evaluation comparisons.

Significantly, the $\psi$0 SNP density constructed from as few as $\psi$1 points outperformed conventional MC quantile estimation at the same sample count, and marginal improvements were observed when increasing from $\psi$2 to $\psi$3 for this application.

Implications and Future Directions

This framework addresses a major hurdle in stochastic nonlinear dynamical systems: efficient, tractable, and theoretically grounded non-Gaussian density estimation. The described SNP-MC approach:

• Is data-efficient: Accurate density fits and quantile computations are achieved with two to four orders of magnitude fewer samples than required for MC-based histogram or regression density estimation.
• Provides analytic access to PDFs and CDFs: The Hermite-based construction enables closed-form evaluation not only of the PDF, but also of various integral functionals critical for risk, chance-constrained control, and estimation.
• Offers extensibility: The framework supports higher-order modeling and can be combined with more sophisticated sampling techniques (importance sampling, polynomial chaos cubature) to further enhance computational efficiency and fit quality.

Practically, the approach is well suited for uncertainty propagation, nonlinear Bayesian estimation (especially for strongly non-Gaussian transition and likelihood densities), and rapid region-probability computations in safety-critical or chance-constrained applications such as spacecraft guidance or real-time decision-making.

Theoretically, the paper provides an explicit derivation of marginal PDFs and CDFs for fully coupled multivariate SNP densities, filling gaps in the literature on high-dimensional, positive, and normalized non-Gaussian expansions, and operationalizing them for nonlinear system state distributions.

Conclusion

This work demonstrates a scalable, data-efficient, and analytically tractable approach to non-Gaussian density estimation for nonlinear systems via MC-accelerated SNP densities (2604.09375). By exploiting the properties of Hermite polynomials in a maximum likelihood framework, combined with convex relaxations for robust initialization, the method enables both high-fidelity density fitting and efficient quantile computation. Potential future enhancements include integration with adaptive cubature or polynomial chaos for further reductions in required sample size, as well as application to high-dimensional, real-world systems in estimation and control.