Semi-Parametric Bayesian Inference

• Semi-parametric Bayesian inference is a framework that estimates finite-dimensional parameters in the presence of an infinite-dimensional nuisance using Bayesian uncertainty quantification.
• It leverages least-favorable submodels and the semiparametric Bernstein–von Mises theorem to achieve frequentist efficiency while flexibly accounting for model misspecification.
• Applications include partial linear models, causal inference, and high-dimensional regression, employing methodologies like Dirichlet process mixtures and modular MCMC.

Semi-Parametric Bayesian Inference is a framework for statistical modeling and inference where a finite-dimensional parameter of scientific interest is estimated in the presence of an infinite-dimensional nuisance component, with Bayesian methods used to quantify uncertainty. This approach aims to achieve the frequentist efficiency properties of parametric inference while allowing substantial model flexibility and robustness to model misspecification. Semiparametric Bayesian inference plays a central role in contemporary statistics, enabling principled inference in models ranging from partial linear regression to causality, transformation models, inverse problems, and complex structured data.

1. Semiparametric Model Structure

A semiparametric Bayesian model specifies observations $X_1, \dots, X_n$ as i.i.d. from a family of distributions parameterized by a finite-dimensional parameter $\theta \in \Theta \subset \mathbb{R}^p$ (the target) and an infinite-dimensional nuisance $\eta \in H$ (function space). The model is assumed dominated, i.e.,

$$p_{\theta, \eta}^{(n)}(X_1, \ldots, X_n) = \prod_{i=1}^n p_{\theta, \eta}(X_i).$$

A product prior $\Pi = \Pi_\Theta \otimes \Pi_H$ is placed, with $\Pi_\Theta$ "thick" at the true $\theta_0$ and $\Pi_H$ assigning positive mass to every neighborhood of the true $\eta_0$. Identification of $(\theta_0, \eta_0)$ is assumed, and it is required that the model supports a "least-favorable submodel" $\theta \in \Theta \subset \mathbb{R}^p$0 minimizing Kullback-Leibler divergence. The key technical device is marginalization and localization via least-favorable curves and reparameterization of $\theta \in \Theta \subset \mathbb{R}^p$1 to $\theta \in \Theta \subset \mathbb{R}^p$2, so that $\theta \in \Theta \subset \mathbb{R}^p$3 lies on the least-favorable curve (Bickel et al., 2010).

2. Bernstein–von Mises Theorem and Semiparametric Efficiency

The central theoretical result in semiparametric Bayesian inference is the semiparametric Bernstein–von Mises (sBvM) theorem. It states that, under regularity conditions, the marginal posterior distribution for $\theta \in \Theta \subset \mathbb{R}^p$4 converges in total variation (in probability) to a normal distribution centered at an efficient estimator, with covariance achieving the semiparametric information bound:

$\theta \in \Theta \subset \mathbb{R}^p$5

where $\theta \in \Theta \subset \mathbb{R}^p$6 and $\theta \in \Theta \subset \mathbb{R}^p$7, with $\theta \in \Theta \subset \mathbb{R}^p$8 the efficient score.

Key assumptions include differentiability (LAN structure), existence and smoothness of a least-favorable curve, sufficiently rich (entropy-controlled) nuisance space, sufficient prior mass in Kullback–Leibler neighborhoods, and parametric-rate (√n) contraction of the posterior for $\theta \in \Theta \subset \mathbb{R}^p$9 (Bickel et al., 2010).

This result rigorously establishes that Bayesian credible sets for $\eta \in H$0 coincide asymptotically with frequentist confidence sets derived from efficient estimators, even when the nuisance parameter is infinite-dimensional and no finite-dimensional likelihood dominates (Bickel et al., 2010).

3. Methodologies for Semiparametric Bayesian Inference

Multiple methodological strategies have been developed to operationalize Bayesian inference in semiparametric models.

• Least-favorable submodels and integrated likelihoods: The sBvM theorem is proved by expanding the integrated marginalized likelihood along the least-favorable submodel, showing that the log-likelihood admits a LAN expansion in $\eta \in H$1, with the nuisance marginalized using the prior.
• Dirichlet process and Bayesian bootstrap: In fully nonparametric setups, inference on functionals can be targeted via augmentation and reweighting of nonparametric priors (θ-augmentation), ensuring perfect prior control over the functional of interest (Meng et al., 2022). The Bayesian bootstrap enables efficient plug-in or two-step inference in models with orthogonal scores, even in highly flexible settings (Sabbagh et al., 23 Feb 2026).
• Flexible regression and modular MCMC: Probabilistic programming platforms such as Liesel implement semiparametric regression models combining additive spline (nonparametric) components with parametric regressors, exploiting graph-based model representations and customizable MCMC kernels for efficient inference in high-dimensional or structured models (Riebl et al., 2022).
• Structured DP mixtures and clustering: In settings with latent heterogeneity, such as random effects or change-point models, Dirichlet process mixture priors enable semi-parametric modeling of error distributions, latent class/cluster structure, or infinite mixtures, with posterior inference achieved via blocked or split-merge MCMC samplers (Wu et al., 2020, Mastrantonio, 2018, Tian et al., 2022).

4. Applications and Exemplars

Semiparametric Bayesian inference underpins several key areas of applied statistics:

• Partial linear models: Estimation of linear coefficients in partial linear regression with nonparametric function nuisance (e.g., $\eta \in H$2) can be conducted with credible intervals achieving exact frequentist efficiency, provided a sufficient prior (e.g., Gaussian process) is used for the nuisance (Bickel et al., 2010).
• Generalized least squares under heteroscedasticity: In high-dimensional regression or panel data, Dirichlet process mixture priors for error law or random effects enable robust GLS estimates, achieving lower posterior variance and MSE compared to parametric or fixed mixture models (Wu et al., 2020).
• Causal inference and missing data: Bayesian semiparametric approaches can handle arbitrary nuisance structure—such as nonparametric propensity scores or covariate distributions—by combining independent GP priors, DP- or Bayesian-bootstrap priors, and efficient λ-tilted priors for functional-parameter inference (Ray et al., 2018, Meng et al., 2022).
• High-dimensional and transformation models: For sparse regression with unknown error distribution, spike-and-slab (parametric) and DP or location-mixture (nonparametric) priors attain near-optimal rates and valid inference even under non-classical error assumptions (Lee et al., 2020, Kowal et al., 2023).
• Complex structured data: Semi-parametric models for clustered recurrent events (with zero-inflation and terminal events) and survival process (via multi-level DP priors and frailty) enable fully Bayesian, robust analysis of clinical and longitudinal data (Tian et al., 2022).

5. Theoretical Implications: Efficiency and Asymptotics

The sBvM theory establishes that, under suitable conditions:

• The marginal posterior for the finite-dimensional interest parameter is asymptotically normal and centered at an efficient estimator, with variance equaling the semiparametric information bound derived from the efficient influence function.
• Bayesian credible intervals for target parameters provide exact frequentist coverage in large samples, provided the model exhibits LAN and the prior contracts at parametric rate (Bickel et al., 2010).
• For functionals (e.g., means, quantiles, average treatment effects), θ-augmentation and plug-in estimators under Dirichlet process or Bayesian bootstrap priors guarantee posterior contraction and asymptotic normality, when the appropriate orthogonality conditions are met (Meng et al., 2022, Sabbagh et al., 23 Feb 2026).
• In irregular models (e.g., estimation of boundaries), exponential (non-normal) BvM limits hold, with analogous interpretation for credible sets (Kleijn, 2013).

6. Extensions and Practical Considerations

• Prior construction and regularity: The practical success of semiparametric Bayesian inference critically depends on prior support properties, smoothness/entropy of the nuisance, and invariance under least-favorable directions (no-bias conditions). Gaussian process, Dirichlet process, and spline/series priors are commonly used.
• Model choice and computation: Modular MCMC, MC-based inference for transformations, and custom probabilistic programming frameworks (e.g., Liesel/Goose) provide practical tools for implementing semi-parametric Bayesian approaches across a wide range of problems (Riebl et al., 2022, Kowal et al., 2023).
• Limitations and open problems: Contracting the marginal posterior at √n rate and ensuring domination/integrated LAN remain the most stringent and technically demanding requirements. Explicit construction of least-favorable submodels is problem-specific; recent work exploits "approximate submodels" to relax this necessity (Kleijn, 2013).
• Robustness and misspecification: Semi-parametric Bayesian approaches naturally provide robustness to distributional misspecification of the nuisance law; inference on functionals avoids accumulation of semiparametric bias when prior invariance and orthogonality criteria are satisfied (Bickel et al., 2010, Sabbagh et al., 23 Feb 2026).

7. Representative Theorems and Formulas

Feature	Statement(s)	Reference
Marginal posterior asymptotics (regular case)	$\eta \in H$3	(Bickel et al., 2010)
Efficient influence & information	$\eta \in H$4, $\eta \in H$5	(Bickel et al., 2010)
θ-augmentation marginal posterior	$\eta \in H$6	(Meng et al., 2022)
Two-step plug-in under Neyman orthogonality	Posterior for θ computed with nuisance fixed at consistent estimator, maintains correct asymptotics when orthogonality holds	(Sabbagh et al., 23 Feb 2026)
Posterior consistency for functionals	$\eta \in H$7	(Meng et al., 2022)

• The semiparametric Bernstein–von Mises theorem (Bickel et al., 2010)
• Targeting functional parameters with semiparametric Bayesian inference (Meng et al., 2022)
• Liesel: A Probabilistic Programming Framework for Developing Semi-Parametric Regression Models and Custom Bayesian Inference Algorithms (Riebl et al., 2022)
• A Semi-Parametric Bayesian Generalized Least Squares Estimator (Wu et al., 2020)
• Semiparametric posterior limits (Kleijn, 2013)
• Semi-parametric Bayesian inference under Neyman orthogonality (Sabbagh et al., 23 Feb 2026)
• Robust Bayesian Synthetic Likelihood via a Semi-Parametric Approach (An et al., 2018)
• Monte Carlo inference for semiparametric Bayesian regression (Kowal et al., 2023)
• Bayesian High-dimensional Semi-parametric Inference beyond sub-Gaussian Errors (Lee et al., 2020)
• Semiparametric Bayesian causal inference (Ray et al., 2018)
• Bayesian semi-parametric inference for clustered recurrent events with zero-inflation and a terminal event (Tian et al., 2022)
• Semi-parametric Bayesian change-point model based on the Dirichlet process (Mastrantonio, 2018)

This body of theory and methodology constitutes the core of modern semiparametric Bayesian inference, enabling valid uncertainty quantification and frequentist-optimal inference for finite-dimensional targets under minimal parametric assumptions on nuisance components.