Maximum Empirical Likelihood Estimation

• MELE is the empirical likelihood analogue of MLE, using moment restrictions instead of a full parametric model.
• It optimizes a nonparametric likelihood subject to constraints, with extensions like adjusted and extended EL addressing feasibility and domain issues.
• Its robust asymptotic theory and global optimization strategies enable consistent inference in dependent, high-dimensional, and many-constraint settings.

Searching arXiv for recent and foundational papers on maximum empirical likelihood estimation and closely related empirical likelihood variants. Maximum empirical likelihood estimation (MELE) is the empirical-likelihood analogue of maximum likelihood estimation for models defined by estimating equations or moment restrictions rather than a fully specified parametric density. In its standard form, empirical likelihood assigns nonnegative weights to observed data and maximizes a nonparametric likelihood subject to constraints such as $E[g(X,\theta_0)]=0$ or $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$. MELE then selects the parameter value that maximizes the resulting profile empirical likelihood, equivalently minimizes the empirical log-likelihood ratio. Across the literature, the method appears in i.i.d. estimating-equation models, generalized empirical likelihood and maximum-entropy formulations, dependent-data problems in the frequency domain, high-dimensional inference, semiparametric efficiency constructions with many constraints, Bayesian computation via empirical likelihood, and specialized rare-event Monte Carlo schemes built around empirical likelihood maximization (Tsao et al., 2013, Rochet, 2012, Liang et al., 2023, Liu et al., 2010, Wang et al., 2023, 0708.0197, Chang et al., 2018, Mengersen et al., 2012, Huang et al., 2013).

1. Definition and basic optimization structure

In the estimating-equation framework, the parameter of interest $\theta_0\in\mathbb R^p$ is defined by

$E[g(X,\theta_0)] = 0,$

with $g(X,\theta)\in\mathbb R^q$, and the data $X_1,\dots,X_n$ are i.i.d. copies of $X$. The literature explicitly treats both just-determined problems, where $q=p$, and over-determined problems, where $q>p$ (Tsao et al., 2013). A closely related moment-condition formulation writes the target as the unique $\theta_0\in\Theta\subset\mathbb R^d$ satisfying

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$0

with $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$1 and $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$2 (Rochet, 2012).

For fixed $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$3, the original empirical likelihood maximizes a multinomial-type likelihood over weights $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$4 or $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$5. One standard formulation is

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$6

with empirical log-likelihood ratio

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$7

Equivalent notation also appears as

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$8

or, in profile form,

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$9

(Tsao et al., 2013, Liu et al., 2010, Liang et al., 2023).

When the constraints are feasible, Lagrange multiplier arguments yield the familiar weights

$\theta_0\in\mathbb R^p$0

or

$\theta_0\in\mathbb R^p$1

where the multiplier solves

$\theta_0\in\mathbb R^p$2

or its analogous form under the chosen notation (Tsao et al., 2013, Mengersen et al., 2012, Liang et al., 2023, Liu et al., 2010).

MELE is defined by maximizing the empirical likelihood over $\theta_0\in\mathbb R^p$3, equivalently minimizing the empirical log-likelihood ratio. Representative definitions are

$\theta_0\in\mathbb R^p$4

$\theta_0\in\mathbb R^p$5

and, in fixed-dimensional classical notation,

$\theta_0\in\mathbb R^p$6

(Liu et al., 2010, Liang et al., 2023, Chang et al., 2018).

This optimization-based definition makes empirical likelihood a nonparametric likelihood method under moment constraints. The method is repeatedly described as “maximum likelihood under moment constraints” or as an estimator that “plays the role of a likelihood-based estimator, but without an explicit parametric likelihood” (Liu et al., 2010, Mengersen et al., 2012). A plausible implication is that the central object in MELE is not a parametric sampling density but the compatibility between observed data and imposed estimating equations.

2. Geometric constraints, existence, and domain corrections

A fundamental limitation of ordinary empirical likelihood is that it is only defined when the origin belongs to the convex hull of the estimating-function vectors. In the i.i.d. estimating-equation setting, the domain $\theta_0\in\mathbb R^p$7 of the original empirical likelihood is generally a subset of $\theta_0\in\mathbb R^p$8, often bounded, because feasibility requires

$\theta_0\in\mathbb R^p$9

If this convex-hull condition fails, the weights do not exist and the empirical likelihood is undefined (Tsao et al., 2013). The same difficulty is emphasized in adjusted empirical likelihood, where the ratio is defined only if $E[g(X,\theta_0)] = 0,$0 lies in the convex hull of $E[g(X,\theta_0)] = 0,$1 (Liu et al., 2010).

This domain mismatch matters statistically. The original empirical likelihood domain may fail to coincide with the true parameter space $E[g(X,\theta_0)] = 0,$2, and this is identified as a principal reason for undercoverage and poor finite-sample performance, especially in over-determined problems (Tsao et al., 2013). In small samples or with high-dimensional estimating functions, nonexistence of solutions can be frequent enough to hinder practical use (Liu et al., 2010).

Two distinct remedies appear in the supplied literature.

The first is extended empirical likelihood. Rather than altering the empirical likelihood objective itself, the construction expands the restricted domain through a geometric mapping centered at the maximum empirical likelihood estimator $E[g(X,\theta_0)] = 0,$3: $E[g(X,\theta_0)] = 0,$4 This “composite similarity mapping” is a similarity transformation on each empirical-likelihood contour and is surjective from $E[g(X,\theta_0)] = 0,$5 onto $E[g(X,\theta_0)] = 0,$6 under Conditions 1–3; with an additional nested-contour condition it becomes bijective (Tsao et al., 2013). Because the mapping need not be one-to-one, a generalized inverse is defined by

$E[g(X,\theta_0)] = 0,$7

and the extended empirical log-likelihood ratio is

$E[g(X,\theta_0)] = 0,$8

The result is an empirical likelihood defined on the full parameter space while retaining the original contour shape (Tsao et al., 2013).

The second is adjusted empirical likelihood, which forces feasibility by adding pseudo-observations. With $E[g(X,\theta_0)] = 0,$9,

$g(X,\theta)\in\mathbb R^q$0

The adjusted empirical likelihood becomes

$g(X,\theta)\in\mathbb R^q$1

with adjusted ratio

$g(X,\theta)\in\mathbb R^q$2

Because $g(X,\theta)\in\mathbb R^q$3 lies on the opposite side of the origin from $g(X,\theta)\in\mathbb R^q$4, the augmented vectors can always be arranged so that the origin lies in their convex hull, and the adjusted optimization problem is always feasible (Liu et al., 2010).

These constructions do not redefine the estimator in a wholly different way. In the extended version, the minimum of $g(X,\theta)\in\mathbb R^q$5 remains at the same center $g(X,\theta)\in\mathbb R^q$6, and maximizing the extended empirical likelihood does not produce a new estimator different from the MELE (Tsao et al., 2013). In the adjusted version, the core profile-likelihood logic remains the same, but feasibility is guaranteed (Liu et al., 2010).

3. Asymptotic theory, Wilks-type limits, and higher-order accuracy

The empirical-likelihood literature represented here repeatedly emphasizes that MELE inherits many asymptotic features of parametric likelihood under regularity conditions. For standard empirical likelihood, the empirical log-likelihood ratio at the true parameter has a chi-square approximation, and maximum empirical likelihood estimators are consistent and asymptotically normal under regularity (Mengersen et al., 2012). In the extended empirical likelihood setting, the main Wilks-type result is

$g(X,\theta)\in\mathbb R^q$7

so the extension preserves the first-order chi-square limit of the original empirical likelihood (Tsao et al., 2013).

The higher-order theory is particularly explicit in the literature on Bartlett correction and adjusted or extended variants. For original empirical likelihood,

$g(X,\theta)\in\mathbb R^q$8

and

$g(X,\theta)\in\mathbb R^q$9

has a $X_1,\dots,X_n$0 approximation with error $X_1,\dots,X_n$1, where $X_1,\dots,X_n$2 is the Bartlett correction factor (Tsao et al., 2013). In the scalar mean case, the factor is

$X_1,\dots,X_n$3

with $X_1,\dots,X_n$4 after standardization so that $X_1,\dots,X_n$5 (Liu et al., 2010).

Adjusted empirical likelihood shows that a specific adjustment level reproduces this higher-order behavior. If

$X_1,\dots,X_n$6

then, under the stated regularity assumptions, the adjusted ratio has an expansion of the same order, and when

$X_1,\dots,X_n$7

the adjusted empirical likelihood achieves the same second-order accuracy as Bartlett-corrected empirical likelihood (Liu et al., 2010). The mechanism appears in the asymptotic relation

$X_1,\dots,X_n$8

which leads to

$X_1,\dots,X_n$9

(Liu et al., 2010).

Extended empirical likelihood also admits a second-order construction in the just-determined case: $X$0 yielding

$X$1

The paper further gives the first-order expansion

$X$2

and states that this helps explain the strong finite-sample performance of the first-order extended empirical likelihood even without explicit Bartlett correction (Tsao et al., 2013).

The regularity conditions required for these expansions are not minimal. Extended empirical likelihood uses Conditions 1–3, including positive definiteness of $X$3, smoothness of $X$4, a Cramér-type condition, and the moment condition

$X$5

(Tsao et al., 2013). Adjusted empirical likelihood assumes Cramér’s condition, finite $X$6th moments,

$X$7

and nonsingular covariance, with additional smoothness in the over-identified case (Liu et al., 2010).

A common misconception is that empirical-likelihood refinements merely repair existence problems. The supplied papers show a broader point: domain correction and feasibility adjustment are tied to higher-order coverage behavior as well as definition on the parameter space (Tsao et al., 2013, Liu et al., 2010).

4. Global maximization, identifiability, and consistency of the empirical-likelihood maximizer

The distinction between a local empirical-likelihood maximizer and the global maximizer is a central issue in the modern theory of MELE. In the profile empirical-likelihood framework,

$X$8

and the MELE is defined as the global minimizer of $X$9 (Liang et al., 2023). The significance of global optimization is that, like parametric likelihood, the empirical-likelihood criterion can have multiple local or global extrema. The local asymptotic theory is not sufficient unless one knows that the extremum under consideration is the global one (Liang et al., 2023).

A major recent contribution is a strong global consistency theorem for the empirical-likelihood maximizer under Conditions C1–C5. These conditions require: uniqueness of the population root

$q=p$0

finite moments and positive definiteness of

$q=p$1

local Lipschitz continuity on compact sets, a closed parameter space, and a scaling vector $q=p$2 such that $q=p$3 behaves properly at infinity (Liang et al., 2023). Under these conditions,

$q=p$4

This is explicitly a theorem on strong global consistency, not merely local consistency near $q=p$5 (Liang et al., 2023).

The same work also introduces a global maximum test. Given a numerically found local maximizer $q=p$6, one rejects the claim that it is global when

$q=p$7

This is presented as a diagnostic for global optimality rather than a model-validity test (Liang et al., 2023).

A more structural remedy is to enlarge the estimating-function vector. If an initial $q=p$8 leads to multiple global maxima or fails C1–C5, the proposed remedy is to add unbiased estimating functions $q=p$9 and use

$q>p$0

so that the combined system satisfies C1–C5 (Liang et al., 2023). The paper illustrates this strategy in several examples. For the Cauchy model, the score-like function

$q>p$1

admits multiple roots and fails C5, but adding $q>p$2 yields a globally consistent MELE (Liang et al., 2023). In nonlinear regression with

$q>p$3

a one-dimensional estimating function may have three roots, whereas the overidentified system

$q>p$4

restores global consistency (Liang et al., 2023).

This body of results modifies an often implicit assumption in empirical-likelihood practice: solving the estimating equations or finding a numerically acceptable local optimum is not itself enough to justify the MELE. The paper’s position is that global identifiability and behavior at infinity must be checked, and, when necessary, additional estimating equations should be introduced (Liang et al., 2023).

5. Generalized empirical likelihood, maximum entropy, and Bayesian interpretation

MELE sits inside the broader class of generalized empirical likelihood (GEL) estimators. In moment-condition models, GEL is defined through divergence minimization. For a convex $q>p$5 with $q>p$6, the $q>p$7-divergence is

$q>p$8

The GEL estimator is

$q>p$9

where $\theta_0\in\Theta\subset\mathbb R^d$0 and $\theta_0\in\Theta\subset\mathbb R^d$1 (Rochet, 2012).

The paper on Bayesian interpretation of GEL shows that a large class of GEL estimators can be represented as maximum entropy on the mean (MEM) solutions, with priors on the weight vector $\theta_0\in\Theta\subset\mathbb R^d$2. In the i.i.d. prior case $\theta_0\in\Theta\subset\mathbb R^d$3, the estimator has saddle-point representation

$\theta_0\in\Theta\subset\mathbb R^d$4

where $\theta_0\in\Theta\subset\mathbb R^d$5 is the log-Laplace transform of the prior (Rochet, 2012).

Within this framework, empirical likelihood itself corresponds to an exponential prior

$\theta_0\in\Theta\subset\mathbb R^d$6

for which

$\theta_0\in\Theta\subset\mathbb R^d$7

Exponential tilting corresponds to a Poisson prior, and the continuous updating estimator corresponds to a Gaussian prior $\theta_0\in\Theta\subset\mathbb R^d$8 (Rochet, 2012). The paper therefore treats empirical likelihood not as an isolated construction but as one member of a maximum-entropy family of discrepancy-based estimators.

This viewpoint also supports robustness to approximate moment conditions. If $\theta_0\in\Theta\subset\mathbb R^d$9 is approximated by $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$00, the approximate GEL estimator becomes

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$01

and if A.1–A.9 hold,

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$02

If

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$03

then $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$04 is $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$05-consistent and asymptotically equivalent to the exact GEL estimator (Rochet, 2012).

The Bayesian strand continues in empirical-likelihood-based posterior approximation. In Bayesian computation via empirical likelihood, the intractable likelihood is replaced by an empirical likelihood $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$06, and the target posterior approximation is

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$07

The basic BC algorithm samples $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$08 and assigns weights

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$09

with approximation quality monitored by the effective sample size

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$10

An adaptive BC-AMIS variant updates multivariate Student proposals $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$11 and uses mixture-proposal weights (Mengersen et al., 2012).

These Bayesian and maximum-entropy formulations do not replace MELE in frequentist inference, but they show that empirical-likelihood maximization admits a probabilistic interpretation through priors on weights and can serve as a surrogate likelihood inside Bayesian samplers (Rochet, 2012, Mengersen et al., 2012).

6. Extensions to dependent data, high dimension, and many constraints

MELE has been extended well beyond the classical fixed-dimensional i.i.d. setting.

For dependent time series, frequency domain empirical likelihood (FDEL) replaces time-domain observations by periodogram ordinates at Fourier frequencies. With spectral estimating functions $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$12, the profile FDEL is

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$13

where

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$14

is the periodogram and $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$15 (0708.0197). The corresponding MELE is

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$16

and Theorem 2 gives a local maximizer $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$17 near $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$18 such that

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$19

Theorem 3 provides conditions under which the global maximizer exists with probability tending to $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$20 and is consistent (0708.0197). This extension covers short-range and long-range dependence and applies to autocorrelations, normalized spectral distribution quantities, Whittle estimation, and spectral goodness-of-fit (0708.0197).

For high-dimensional estimating-equation models, the challenge is that the parameter dimension $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$21 and the number of estimating equations $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$22 may grow exponentially. The cited work uses a penalized empirical likelihood estimator

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$23

with penalties on both $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$24 and $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$25 (Chang et al., 2018). For valid inference on a low-dimensional target subvector $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$26, the paper constructs transformed estimating equations

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$27

where $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$28 is chosen so that nuisance-gradient effects are asymptotically negligible. The resulting transformed empirical-likelihood ratio

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$29

has a Wilks-type limit under stated rate conditions, either $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$30 for fixed $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$31 or a normal approximation when $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$32 (Chang et al., 2018).

A distinct extension studies estimation of a linear functional

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$33

under side information characterized by infinitely many constraints. The “easy EL” estimator uses weights

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$34

producing

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$35

With estimated constraint functions and a growing number $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$36 of constraints, the paper establishes semiparametric efficiency in settings including known marginals, unknown but identical marginals, and distributional symmetry (Wang et al., 2023). The key expansion is

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$37

where $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$38 is the projection of $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$39 onto the closed linear span of the constraints (Wang et al., 2023). This suggests that empirical-likelihood weighting can be understood as projection-based variance reduction in semiparametric models.

Taken together, these extensions show that MELE is not confined to low-dimensional i.i.d. moment problems. It has been adapted to spectral inference for dependent data, sparse and overidentified high-dimensional models, and semiparametric estimation with infinitely many constraints (0708.0197, Chang et al., 2018, Wang et al., 2023).

7. Applications, computational roles, and practical interpretation

MELE and empirical-likelihood maximization appear in several distinct applied and computational roles.

In Bayesian computation via empirical likelihood, empirical likelihood replaces an intractable model likelihood. The method is positioned as an alternative to ABC because it does not simulate pseudo-data in the likelihood-approximation step, does not require a tolerance $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$40, and does not require ad hoc summary-statistic matching in the ABC sense (Mengersen et al., 2012). The paper illustrates this with standard distributions, time series, the $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$41-and-$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$42 family, ARCH(1), GARCH(1,1), and population-genetics models based on pairwise composite likelihood score constraints (Mengersen et al., 2012).

In rare-event probability estimation, the phrase Empirical Likelihood Maximization (ELM) is used for a Monte Carlo method that estimates unknown normalizing constants in a sequence of densities

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$43

one of which embeds the rare-event probability

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$44

as a normalizing constant (Huang et al., 2013). With pooled samples from the $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$45, the method reparameterizes via

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$46

and minimizes the convex objective

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$47

The target estimator is then recovered from

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$48

up to the chosen normalization (Huang et al., 2013). Although this usage differs from classical statistical MELE, the paper explicitly frames ELM as paralleling MLE logic: maximize an empirical likelihood to estimate unknown rare-event probabilities (Huang et al., 2013).

Practical recommendations and limitations recur across the literature. One recommendation in Bayesian computation via empirical likelihood is to keep the number of constraints equal to the dimension of $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$49, because overconstraining can degrade the fit (Mengersen et al., 2012). In classical EL inference, poor finite-sample behavior is linked to domain mismatch and infeasible constraints (Tsao et al., 2013, Liu et al., 2010). In the high-dimensional setting, too large a testing index set $\int \Phi(\theta_0,x)\,d\mu_0(x)=0$50 can inflate the critical value and reduce power (Chang et al., 2018). In the semiparametric many-constraint setting, efficiency depends on growth-rate conditions such as

$\int \Phi(\theta_0,x)\,d\mu_0(x)=0$51

depending on whether constraints are known or estimated (Wang et al., 2023).

A common misconception is that maximum empirical likelihood estimation is a single fixed estimator with one canonical form. The supplied papers instead present a family of closely related constructions. The classical MELE is the maximizer of a profile empirical likelihood under exact estimating constraints (Liang et al., 2023, Liu et al., 2010). Extended empirical likelihood leaves the estimator unchanged while expanding the domain (Tsao et al., 2013). Adjusted empirical likelihood guarantees feasibility and can match Bartlett-corrected accuracy (Liu et al., 2010). Generalized empirical likelihood embeds empirical likelihood inside a broader divergence and maximum-entropy class (Rochet, 2012). Frequency-domain, high-dimensional, and many-constraint formulations change the underlying empirical-likelihood problem to fit dependence, nuisance structure, or side information (0708.0197, Chang et al., 2018, Wang et al., 2023).

In that broader sense, MELE is best understood as a nonparametric likelihood principle for moment-defined parameters: choose the parameter value whose associated constraints yield the largest feasible empirical likelihood. The continuing research themes in the cited work are geometric correction of the likelihood domain, higher-order calibration, global rather than merely local maximization, robustness to approximate moments, exploitation of auxiliary constraints for efficiency, and adaptation to computational settings where ordinary likelihood is unavailable or inconvenient (Tsao et al., 2013, Liu et al., 2010, Liang et al., 2023, Rochet, 2012, Mengersen et al., 2012).