DiffHybrid-UQ: Uncertainty in Hybrid Models

• The paper introduces DiffHybrid-UQ, a Bayesian framework that quantifies both aleatoric and epistemic uncertainties using deep ensemble learning and the unscented transformation.
• It employs ensemble SGD trajectories with SWAG to approximate the posterior over neural and physical parameters, ensuring robust scientific inference.
• The method is scalable and simple to implement with standard deep learning libraries, making it applicable to both ODE and PDE systems with partial data.

Searching arXiv for the specified paper and closely related hybrid/UQ work. DiffHybrid-UQ is a method for uncertainty quantification in differentiable hybrid neural models, introduced to address the quantification and propagation of inherent uncertainties in models that integrate numerical representations of known physics into deep neural networks (Akhare et al., 2023). It operates within a probabilistic Bayesian framework, combines Bayesian model averaging with deep ensemble Bayesian learning and the unscented transformation, and is designed to distinguish and quantify both aleatoric uncertainties arising from data noise and epistemic uncertainties resulting from model-form discrepancies and data sparsity. The framework is described as simple to implement, scalable in parallel computing environments, and validated on problems governed by both ordinary and partial differentiable equations (Akhare et al., 2023).

1. Definition and model class

Differentiable hybrid neural models, or DiffHybrid models, combine known physics with neural networks so that partially known or unknown physical behavior can be learned from data (Akhare et al., 2023). In this setting, uncertainty quantification is not an auxiliary diagnostic but part of the model specification, because the predictive distribution must account both for uncertainty in the observations and for uncertainty in the trainable physical and neural components.

DiffHybrid-UQ formalizes this requirement by treating prediction as a posterior predictive inference problem. The method is explicitly built for hybrid neural differentiable models rather than for hybrid dynamical systems in the switching or reset sense. That distinction matters because the relevant uncertainty sources differ: in DiffHybrid-UQ, the central issues are measurement noise, unknown physical parameters, neural-network parameter uncertainty, and model-form discrepancy (Akhare et al., 2023).

A concise summary of the framework is given below.

Component	Role	Mechanism
Aleatoric uncertainty	Measurement noise or inherent stochasticity	Additive Gaussian noise modeled by the hybrid neural model
Epistemic uncertainty	Model, parameter, and data sparsity uncertainty	Ensembles of SGD trajectories with SWAG + DeepEnsemble
Predictive distribution	Full uncertainty-aware output	Bayesian model averaging with posterior samples

This organization places DiffHybrid-UQ within scientific machine learning rather than within purely statistical regression, because the predictive object is constrained by numerical physics and by differentiable computational structure (Akhare et al., 2023).

2. Bayesian formulation and predictive distribution

The framework adopts a posterior predictive distribution for the model output $\mathbf{V}$ at input $\tilde{\mathbf{x}}$ given data $\mathcal{D}$:

$$p(\mathbf{V}\mid \tilde{\mathbf{x}}, \mathcal{D}) = \int p(\mathbf{V}\mid \tilde{\mathbf{x}}, \boldsymbol{\theta})\, p(\boldsymbol{\theta}\mid \mathcal{D})\, d\boldsymbol{\theta}.$$

Here, $$p(\mathbf{V}\mid \tilde{\mathbf{x}}, \boldsymbol{\theta})$$ is the likelihood under a particular model parameterization, and $p(\boldsymbol{\theta}\mid \mathcal{D})$ is the posterior over trainable parameters (Akhare et al., 2023). In practice, the predictive distribution is approximated by Monte Carlo sampling over parameter draws $\boldsymbol{\theta}^{(j)}$:

$$p(\mathbf{V}\mid \tilde{\mathbf{x}}, \mathcal{D}) \approx \frac{1}{M}\sum_{j=1}^{M} p(\mathbf{V}\mid \tilde{\mathbf{x}}, \boldsymbol{\theta}^{(j)}).$$

This is Bayesian model averaging in operational form. It produces a predictive distribution rather than only a point estimate, and it also provides the basis for separating total predictive uncertainty into aleatoric and epistemic components. The method therefore treats uncertainty propagation and posterior approximation as coupled problems rather than as independent post-processing steps (Akhare et al., 2023).

Within this formulation, DiffHybrid-UQ approximates the posterior over both network parameters and physical parameters. A plausible implication is that the method is intended to support scientific inference tasks in which calibration of latent physical quantities and robust prediction of observables are both required.

3. Aleatoric uncertainty and nonlinear propagation

Aleatoric uncertainty in DiffHybrid-UQ represents randomness from measurement noise or inherent stochasticity in the data. It is modeled as additive Gaussian noise:

$$\mathbf{V}\mid \tilde{\mathbf{x}}, \boldsymbol{\theta} = \bar{\mathbf{v}}_{\boldsymbol{\theta}}(\tilde{\mathbf{x}}) + \boldsymbol{\epsilon}, \qquad \boldsymbol{\epsilon}\sim \mathcal{N}\!\left(0,\operatorname{diag}\!\left(\Sigma_{\mathbf{V},\boldsymbol{\theta}}^{2}\right)\right).$$

The hybrid neural model outputs both a mean prediction and a variance field at each point,

$$\bar{\mathbf{v}}_{\boldsymbol{\theta}}(\tilde{\mathbf{x}}), \quad \operatorname{diag}\!\left(\Sigma_{\mathbf{V},\boldsymbol{\theta}}^{2}(\tilde{\mathbf{x}})\right) = \mathrm{DiffHybrid}(\tilde{\mathbf{x}};\boldsymbol{\theta}),$$

which induces the Gaussian likelihood

$\tilde{\mathbf{x}}$0

These definitions make aleatoric uncertainty an explicit output of the learned hybrid model rather than a fixed noise assumption (Akhare et al., 2023).

The key propagation mechanism for aleatoric uncertainty is the unscented transformation. For linear components, propagation is analytical because affine transforms of Gaussians remain Gaussian. For nonlinear components, DiffHybrid-UQ uses the unscented transformation to propagate mean and covariance by deterministically selecting sigma points, propagating them through the nonlinear functions, and reconstructing transformed mean and covariance from weighted averages (Akhare et al., 2023). The paper characterizes this procedure as more efficient than Monte Carlo sampling and superior to local linearization, especially for sharp nonlinearities.

This feature is central because differentiable hybrid neural models typically contain nonlinear operators inherited from the discretized physics, from neural parameterizations, or from both. The unscented transformation therefore functions as the bridge between local probabilistic outputs and uncertainty-aware hybrid-model dynamics.

4. Epistemic uncertainty and posterior approximation

Epistemic uncertainty in DiffHybrid-UQ is associated with model uncertainty, parameter uncertainty, and data sparsity. The paper identifies the main computational challenge as the intractability of classical Bayesian inference in high-dimensional neural parameter spaces and addresses it by using a deep ensemble SWAG framework (Akhare et al., 2023).

The epistemic component is estimated through ensembles of independently initialized SGD training trajectories. Along each trajectory, SWAG stores a running mean $\tilde{\mathbf{x}}$1, a diagonal variance $\tilde{\mathbf{x}}$2, and a low-rank approximation to posterior covariance using a deviation matrix $\tilde{\mathbf{x}}$3 (Akhare et al., 2023). Posterior samples are then drawn efficiently from this approximation. The method is presented as a practical approximation to the posterior distribution of both the network parameters and the physical parameters.

The total predictive variance is decomposed by the law of total variance into an aleatoric term and an epistemic term:

$\tilde{\mathbf{x}}$4

In practical computation, the predictive mean is estimated by averaging the mean outputs across sampled parameterizations, and the predictive variance is estimated as the sum of the average predicted variances and the variance of the predictive means across samples (Akhare et al., 2023). The first term corresponds to aleatoric uncertainty and the second to epistemic uncertainty.

The paper emphasizes that ensembling across multiple SGD paths together with local SWAG Gaussian posteriors enables the method to capture multi-modality and epistemic uncertainty more robustly than a single local approximation. This suggests that DiffHybrid-UQ is especially aimed at settings in which posterior structure is shaped by both nonconvex neural optimization and partial physical observability.

5. Implementation, scalability, and computational profile

DiffHybrid-UQ is described as simple to implement because its main components—ensembles, SWAG, and the unscented transformation—are compatible with standard deep learning libraries and differentiable programming (Akhare et al., 2023). The framework is also parallelizable: ensemble training and SGD runs can be distributed across multiple compute nodes or GPUs. This is paired with low overhead, since SWAG avoids storing full SGD trajectories by maintaining a diagonal covariance estimate together with a low-rank correction.

The paper further states that the approach has no fragile hyperparameters in the sense associated with variational inference or Bayesian neural networks with KL terms. It is presented as suitable for large neural models and for PDE and ODE solvers, including cases with partial and indirect observation data (Akhare et al., 2023).

These properties are consequential for scientific machine learning workloads. In hybrid neural differentiable models, training cost is often dominated by repeated forward and backward passes through numerical solvers or solver-informed architectures. A scalable posterior approximation that can be parallelized without requiring full Markov chain Monte Carlo is therefore operationally significant. The method’s use of coarse-grid training in some PDE experiments, while correcting discretization error with a learned neural component, reinforces this computational orientation (Akhare et al., 2023).

6. Empirical behavior and relation to adjacent UQ frameworks

The reported empirical evaluation covers both ODE and PDE problems. In ODE experiments, including Hamiltonian systems with full, sparse, and single-variable data, DiffHybrid-UQ recovered means and credible intervals while distinguishing aleatoric and epistemic uncertainty (Akhare et al., 2023). Compared with Hamiltonian Monte Carlo described as a gold standard sampling method, it matched or outperformed uncertainty capture particularly in extrapolation or data-sparse regions. The method also increased epistemic uncertainty adaptively as the model extrapolated beyond the training trajectory.

In PDE experiments on reaction-diffusion systems over $\tilde{\mathbf{x}}$5D grids, the framework was trained with partial observation data, including cases where one state variable was observed and another was not (Akhare et al., 2023). It propagated both mean and spatiotemporal uncertainty fields, supported inference of physical parameters such as diffusion coefficients with quantified uncertainty, and improved parameter identifiability when the physical process dominated system behavior. The paper also reports a discretization-error-correction setting in which coarse-grid training with data from a high-fidelity simulator preserved prediction accuracy while lowering computational cost.

The broader significance of DiffHybrid-UQ becomes clearer when it is placed alongside adjacent UQ literatures. In one earlier line of work, uncertainty quantification for hybrid dynamical systems focused on systems with continuous dynamics and discrete events, and extended polynomial chaos by using a wavelet-based Wiener-Haar expansion, a boundary layer approach for reset conditions, and a transport-theory-based formulation (Sahai et al., 2011). In another line, a module-based hybrid uncertainty quantification framework for nonlinear multi-physics simulation allowed separate physics modules to use non-intrusive, semi-intrusive, or intrusive methods and coupled them through generalized polynomial chaos expansions (Mittal et al., 2014). DiffHybrid-UQ addresses a different computational object: differentiable hybrid neural models with Bayesian model averaging, ensemble SGD posterior approximation, and unscented-transform-based nonlinear propagation (Akhare et al., 2023).

A distinct later development introduces a diffusion-based posterior sampling framework for industrial data-driven models that seeks intrinsically calibrated uncertainty via faithful posterior sampling rather than post-hoc calibration (Ma et al., 2 Apr 2026). This suggests a broader trajectory in UQ research toward methods that represent posterior structure more directly, although the model class, calibration objective, and inference mechanism differ from those of DiffHybrid-UQ.

Within that landscape, DiffHybrid-UQ is best understood as a Bayesian uncertainty-propagation method specialized to differentiable hybrid neural modeling: it couples explicit aleatoric modeling, approximate posterior inference over neural and physical parameters, and scalable predictive aggregation in a single framework (Akhare et al., 2023).