---
title: Bayesian Empirical Likelihood
url: https://www.emergentmind.com/topics/bayesian-empirical-likelihood
type: topic
---

# Bayesian Empirical Likelihood

Bayesian empirical likelihood (BEL) is a methodology within Bayesian statistics that replaces the parametric likelihood with an empirical likelihood—a nonparametric, data-driven likelihood constructed from moment or estimating equation constraints. BEL facilitates Bayesian inference in models where the true likelihood is computationally intractable or unavailable, while maintaining the desirable properties of both Bayesian and nonparametric inference. This strategy has seen rapid development, with critical advances in theoretical foundations, computational algorithms, and application domains including quantile regression, dynamic models, variable selection, and survey analysis.

## 1. Formulation and Theoretical Foundations

BEL centers on integrating empirical likelihood (EL) into the Bayesian framework by treating the EL as a (pseudo-)likelihood for posterior updating. Given independent data $y_1, \ldots, y_n$, the empirical likelihood for parameter $\theta$ under a set of constraints (e.g., moment or score equations) is constructed as
\[
L_{EL}(\theta|y) = \max_{p \in P_\theta} \prod_{i=1}^n p_i,
\]
where $P_\theta = \left\{ p \in [0,1]^n : \sum_{i=1}^n p_i = 1, \sum_{i=1}^n p_i h(y_i, \theta) = 0 \right\}$ for a suitably chosen constraint function $h(y, \theta)$ (such as $y - \theta$ for the mean, or more generally, a vector of estimating equations).

In Bayesian computation via empirical likelihood (BCel), the posterior is then
\[
\pi(\theta|y) \propto \pi(\theta) \cdot L_{EL}(\theta|y).
\]
This construction yields posteriors with convergence properties analogous to those of fully parametric Bayesian posteriors when the constraint functions are correctly specified and as $n \to \infty$ [1205.5658], [1803.06645].

A major theoretical result is that under regularity, the BEL posterior is asymptotically normal with mean centered at the maximum empirical likelihood estimator (MELE) and variance matching that of MELE (e.g., for quantile regression [1207.5378]). Wilks-type and Bernstein–von Mises theorems hold under suitable conditions, ensuring valid asymptotic uncertainty quantification [2312.17015], [1801.00243], [2109.07792].

## 2. Algorithmic and Computational Aspects

The BCel algorithm typically employs importance sampling:

| Step               | Description                                                        |
|--------------------|--------------------------------------------------------------------|
| 1. Prior Sampling  | Draw $\theta_i \sim \pi(\theta)$                                   |
| 2. Weighting       | Compute $\omega_i = L_{EL}(\theta_i|y)$                            |
| 3. Posterior Sample| Use $(\theta_i, \omega_i)$ as a weighted posterior sample          |

For higher efficiency, adaptive multiple importance sampling (AMIS) can refine the proposal distribution iteratively (e.g., mixtures of Student’s $t$) [1205.5658].

Key computational considerations include:
- **Constrained optimization:** Each EL evaluation requires solving a convex optimization problem (solved in practice via Newton–Lagrange methods, e.g., R package "emplik").
- **Support irregularity:** The convex hull constraint restricts BEL support to a subset of the parameter space where constraints are feasible, leading to nonconvex and possibly disconnected posterior support [2209.01289].
- **Advanced MCMC:** Standard random walk proposals may perform poorly due to irregular support, motivating the use of Hamiltonian Monte Carlo (HMC), which leverages the diverging gradients of the log empirical likelihood at the support boundary to "reflect" proposals back inside. This improves mixing and effective exploration [2209.01289], [2006.10258].
- **Variational Inference and Expectation Propagation:** Recent advances employ expectation-propagation (EP) to achieve fast, scalable Gaussian approximations of the BEL posterior, rigorously shown to be asymptotically equivalent to the exact BEL under increasing sample size [2510.21174].

## 3. Choice and Role of Constraints

The performance of BEL is critically dependent on the specification of constraint functions $h(y, \theta)$:
- **Classical settings:** For mean estimation, $h(y, \theta) = y - \theta$ suffices.
- **Complex models:** Score functions or composite scores (e.g., pairwise composite likelihoods in population genetics) are often used where the likelihood is analytically intractable [1205.5658].
- **ABC integration:** In ABC-style inference, summary statistics (moments, quantiles, etc.) serve as constraints, and the empirical likelihood is constructed to match the observed and simulated summaries [1810.01675], [2011.07721], [2403.05080].
- **Quantile regression:** In the absence of a parametric likelihood, sample quantile estimating equations define the empirical likelihood constraints [1207.5378].

Over- or under-specification of constraints can negatively impact both efficiency and identifiability. Testing constraint sets against simulated data or prior predictive checks helps calibrate their informativeness [1205.5658].

## 4. Posterior Inference, Extensions, and Robustification

BEL has been extended and adapted for multiple inferential tasks:
- **Quantile Regression:** BEL yields asymptotically normal posteriors for quantile parameters, with variance matching regression quantile estimators. Informative (shrinkage) priors can be incorporated across quantile levels to borrow strength and reduce uncertainty in sparse data regions [1207.5378].
- **Hierarchical and Spatial Modeling:** Empirical likelihood can be specified as the data level in hierarchical models, with parametric or reduced-rank process models inducing dependencies (e.g., spatial autocorrelation), as in semiparametric hierarchical empirical likelihood (SHEL) models for areal or point-referenced spatial data [1405.3880].
- **Robust BEL:** Exponentially tilted empirical likelihood (BETEL) and its robust variant (RBETEL) accommodate outlier contamination by introducing latent indicators for "good" vs. "bad" data, up-weighting uncontaminated observations in the likelihood through modified estimating equations and marginalization [1801.00243].
- **Regularization:** In high-dimensional or ill-posed scenarios, penalization (e.g., penalized empirical likelihood) targets both Lagrange multipliers and parameters to enhance identification and stability, yielding consistent and asymptotically normal estimators [2412.17354], [2006.10258].
- **Convex Hull Relaxation:** Regularized exponentially tilted empirical likelihood (RETEL) introduces pseudo-data from continuous exponential family distributions, removing the support-restricting convex hull constraint and ensuring that posterior inference is well defined even for small samples or extreme parameter values [2312.17015].

## 5. Applications and Performance

BEL has demonstrated strong empirical performance in a broad array of settings:

| Domain                     | Summary of BEL Implementation and Outcomes                                             |
|----------------------------|---------------------------------------------------------------------------------------|
| Population genetics        | Composite moment constraints in models with intractable likelihoods, large reduction in simulation and improved calibration compared to ABC [1205.5658].      |
| Time series (e.g., ARCH, GARCH) | Moment-based constraint functions on reconstructed latent innovations, robust and effective estimation [1205.5658].                       |
| Quantile-based models      | Robust uncertainty quantification when analytic likelihood is unavailable; improved efficiency via shrinkage across quantiles [1207.5378].                 |
| Bayesian variable selection| Nonparametric BEL with sparsity-inducing priors (such as LASSO, SCAD) demonstrates consistency for large parameters and correct zeroing for noise variables; effective MCMC via Laplace-approximation-based proposals [2206.05204]. |
| Survey data                | Bayesian jackknife empirical likelihood enables calibrated inference with complex design weights or calibration constraints in complex surveys [2309.06781].      |
| Model selection            | Extension to reversible jump MCMC (RJMCMC) with hierarchical proposals based on empirical likelihood maximization achieves feasible, higher-acceptance sampling in models of varying dimension [2209.01269].        |

Furthermore, simulation experiments widely confirm the asymptotic theory: credible intervals from BEL achieve correct coverage, and, in numerous real-data cases, BEL point estimators exhibit lower mean squared error and credible intervals with improved calibration compared to parametric, pseudo-likelihood, or frequentist approaches.

## 6. Methodological Developments and Limitations

BEL methodologies manifest several strengths:
- **No requirement for full parametric likelihood:** Enables posterior inference under moment information alone.
- **Self-diagnostics:** Effective sample size, sensitivity to constraint choice, and comparison across constraint sets are naturally embedded in importance sampling and weight diagnostics.
- **Flexible prior structures:** Priors (including shrinkage for variable selection and hierarchical models) can be integrated directly, allowing incorporation of domain knowledge.

Critical limitations remain:
- **Constraint specification:** Faithful inference relies on valid, identifying constraint functions. Mis-specification leads to poorly calibrated posteriors. There is an ongoing need for principled constraint selection or data-driven diagnostic tools.
- **Computational burden:** Each EL evaluation is a constrained optimization, posing substantial computational challenges in high dimensions or complex models. While HMC and expectation-propagation mitigate some issues, scaling to very high dimensions remains difficult [2510.21174].
- **Irregular support:** The presence of nonconvex support complicates both practical implementation and theoretical analysis. Regularization via pseudo-data (e.g., RETEL) and robustification via indicator augmentation (RBETEL) offer partial solutions.

## 7. Extensions, Variants, and Research Directions

Recent research has expanded BEL in the following directions:
- **Approximate Bayesian Computation (ABC):** Empirical likelihood is used to compute approximate posteriors in ABC without requiring user-specified distances or tolerances, instead employing constraints on summary statistics. Such methods are theoretically justified via information projection and shown to be consistent [1810.01675], [2011.07721], [2403.05080].
- **Variational Inference and EP:** Expectation-propagation and variational methods for BEL deliver efficient, large-scale computation while attaining asymptotic equivalence to the exact BEL posterior [2510.21174].
- **Regularized and penalized BEL:** Approaches such as penalized empirical likelihood and RETEL address nonconvexity, parameter identifiability, and support shrinkage, providing improved finite-sample performance and broader support for Bayesian inference [2312.17015], [2412.17354].
- **Software:** R packages such as "emplik" and "elhmc" implement core optimization routines and HMC sampling for BEL, facilitating practical application across diverse models [2209.01289].

Open problems include: automatic constraint selection, theoretical guarantees for high-dimensional or misspecified constraint scenarios, improved robustness features, and more scalable algorithms for high-throughput applications.

---

Bayesian empirical likelihood represents a robust, flexible alternative to parametric likelihood-based Bayesian inference, particularly suitable for models with intractable likelihoods or only partial moment information. Its ongoing development spans foundational theory, computational methods, and broad applications ranging from quantile regression to population genetics and survey analysis, with significant recent progress in computational tractability and robustness.

Source: https://www.emergentmind.com/topics/bayesian-empirical-likelihood