---
title: Semiparametric Full-Likelihood Estimators
url: https://www.emergentmind.com/topics/semiparametric-full-likelihood-estimators
type: topic
---

# Semiparametric Full-Likelihood Estimators

Semiparametric full-likelihood estimators are a class of statistical procedures that achieve efficient estimation of finite-dimensional parameters in models containing infinite-dimensional nuisance functions or nonparametric components, by leveraging the full likelihood structure and appropriate profiling or regularization strategies. These methods are rigorous extensions of classical likelihood techniques, designed to attain the semiparametric efficiency bound even when key aspects of the model, such as baseline densities or selection mechanisms, are modeled nonparametrically. They are fundamental in high-dimensional data analysis, missing data problems, density ratio models, regression with censored data, and complex dependencies where parametric assumptions for all model components are infeasible or undesirable.

## 1. Model Frameworks and Semiparametric Structure

Semiparametric full-likelihood estimators operate within models containing both finite-dimensional parameters of interest and infinite-dimensional nuisance functions. A canonical example is the semiparametric exponential family:

\[
f(x;\theta,g) = \exp\left\{ \theta^\top T(x) - A(\theta,g) + g(x) \right\}
\]

where $\theta \in \Theta \subset \mathbb{R}^d$ is the parameter of interest, $T(x)$ is a known vector of sufficient statistics, and $g(x)$ is a completely unspecified, nonparametric nuisance function. The partition function $A(\theta,g)$ ensures normalization. Identifiability is ensured by constraints such as $\int e^{g(x)} dx = 1$ and $A(\theta,g) < \infty$ for all $\theta$ in compact $\Theta$ [1711.11426].

Other generic models include proportional likelihood-ratio models [1906.00723], semiparametric mixture models with log-concave densities [1903.11200], selection models for non-ignorable missing data [1908.01260], sieve likelihood models for censored and truncated data [2504.07413], copula-based multivariate models [2401.17334], and regressions with flexible spline components [1311.1039].

Across all these models, the goal is to estimate $\theta$ efficiently without over-constraining $g$, while maintaining practical identifiability and robustness.

## 2. Construction of Semiparametric Full-Likelihood Estimators

The core methodology for semiparametric full-likelihood estimation involves four intertwined steps:

1. **Writing the Full Likelihood**: The joint log-likelihood $l(\theta,g)$ is written to include both $\theta$ and the full nonparametric $g$, for all observed data.

2. **Profiling or Sieve Approximation**: The infinite-dimensional nuisance function $g$ is profiled out via the least favorable curve approach, or approximated using sieve expansions (e.g., B-splines, Bernstein polynomials, empirical likelihood atoms) [1711.11426, 2103.00648, 1311.1039, 2504.07413]. The least favorable curve $g_\theta$ satisfies the functional score zero condition, leading to explicit or numerically tractable representations:

    \[
    e^{g_\theta(y)} = \left[ E_\theta \left\{ \exp(\theta^\top T(X)) \mid X = y \right\} \right]^{-1}
    \]

3. **Maximizing the Profile Likelihood**: The resulting profile log-likelihood $\ell_p(\theta) = l(\theta, g_\theta)$ is maximized with respect to $\theta$, often via efficient gradient-based algorithms.

4. **Efficient Score and Information**: Because $g_\theta$ solves the functional profile equations, the efficient score for $\theta$ is simply the gradient $\partial_\theta \ell(\theta, g_\theta)$ evaluated at the observed data. The semiparametric Fisher information is $I_\mathrm{eff}(\theta) = \mathrm{Var}_\theta \{ S_\mathrm{eff}(\theta;X) \}$ [1711.11426, 1906.00723, 2504.07413].

These steps generalize to the semiparametric proportional likelihood ratio [1906.00723], spline-sieve likelihoods [2504.07413], empirical likelihood schemes in mixture and semicontinuous models [2012.07092], and sieve copula estimators in multivariate models [2401.17334].

## 3. Algorithmic Implementation and Computational Aspects

Semiparametric full-likelihood algorithms routinely employ kernel or sieve smoothing for nonparametric components, EM-type iterations for mixtures or missing-data structures, and high-dimensional optimization routines. Key examples include:

- **Nadaraya–Watson estimator** for $m(y;\theta) = E_\theta[\exp(\theta^\top T(X)) | X=y]$ and its normalization for computing $g_\theta(y)$ [1711.11426].
- **Spline sieves**: Nonparametric functions approximated by $g(s) \approx \sum_{k} \gamma_k B_k(s)$, jointly optimized alongside $\theta$ [2504.07413, 1311.1039].
- **Bernstein polynomials**: Used for robust approximation of baseline densities in two-sample DRM and copula models [2103.00648, 2401.17334].
- **Empirical likelihood atomic masses**: Discrete $\{p_{ij}\}$ formulated via Lagrangian constraints for multimodal or semicontinuous populations [2012.07092].
- **Fractional imputation EM**: For nonignorability and missingness, missing $Y$ values are multiply imputed, weighted by the nonparametric response mechanism, and the full likelihood is maximized alternately over $\phi$ and $g$ [1809.03645].
- **Full semiparametric likelihood for missing data**: Empirical likelihood over the marginal $X$ distribution, alternating Lagrange-Multiplier and parameter updates [1908.01260].
- **Deep neural network integration**: Profiling out the nonparametric baseline hazard in frailty models and training via back-propagation with penalized profile h-likelihood [2307.06581].

The complexity per likelihood evaluation is generally $O(n)$ per gradient step, and EM/sieve algorithms converge rapidly when the sieve dimension and kernel parameters are properly chosen.

## 4. Asymptotic Theory and Semiparametric Efficiency

Semiparametric full-likelihood estimators achieve $\sqrt{n}$-consistency and asymptotic normality for $\theta$ under standard regularity conditions: compactness, identifiability, smoothness of $g$, and bounded covariate distributions.

The canonical limit result:

\[
\sqrt{n} (\hat{\theta} - \theta_0) \xrightarrow{d} N(0,\,I_\mathrm{eff}(\theta_0)^{-1})
\]

is established for the profile likelihood in the semiparametric exponential family [1711.11426], Z-estimator constructions in likelihood ratio models [1906.00723], sieve likelihood estimators for censored/truncated regression [2504.07413], and Bernstein–von Mises Bayes procedures under symmetric error [1510.05247]. The sieve MLE attains the semiparametric bound for any functional $\lambda^\top \beta$, and empirical likelihood approaches can be used to construct confidence intervals and perform hypothesis testing with $\chi^2$ limiting distributions for linear or smooth functionals [2012.07092].

Comparative simulation studies demonstrate that these methods uniformly attain or outperform classical conditional-likelihood estimators, pseudo-likelihood approaches, and fully parametric MLEs under misspecification, particularly in high-dimensional, nonstandard, or data-missing settings [1711.11426, 1906.00723, 1908.01260, 2401.17334, 2012.07092].

## 5. Applications, Robustness, and Empirical Performance

Semiparametric full-likelihood methods have demonstrated robustness and efficiency across a spectrum of data contexts, including:

- **Linear and generalized linear models with unknown error distribution or base measure**: negligible bias, robust variance control, full efficiency even under model deviations [1711.11426, 2504.07413, 2204.02170].
- **Handling non-ignorable missingness**: Properly designed full-likelihood estimators with empirical likelihood over $X$ provide identifiability and efficiency without auxiliary IVs or restrictive assumptions [1908.01260, 1809.03645].
- **Density-ratio and mixture models**: Bernstein polynomial EM estimators yield smooth, boundary-adaptive densities and efficient inference of mixture proportions and component functionals [2103.00648, 2012.07092, 1903.11200].
- **Mark-recapture and survival models with flexible baseline probability structures**: Penalized spline full-likelihood recovers nonlinear survival–covariate associations and enables data-driven smoothing parameter selection [1311.1039, 2504.07413].
- **Multilevel and frailty models**: Profiling-out nonparametric hazard functions combined with h-likelihood and deep networks enables scalable and unbiased inference for complex clustered event data [2307.06581].

Simulations and real-data analyses—e.g., wage studies with extensive missingness, insurance loss modeling, biological mixture identification, and multi-state disease progression—consistently demonstrate efficiency improvement, robust bias control, and proper coverage under the proposed full-likelihood semiparametric estimators [1711.11426, 1908.01260, 2401.17334, 2012.07092, 1311.1039].

## 6. Variants, Extensions, and Future Directions

Semiparametric full-likelihood techniques continue to expand into more complex modeling regimes, including:

- **High-dimensional settings and penalization**: Adaptive selection of smoothing, knot dimension, or sieve basis enables extension to settings with many covariates or response classes [1311.1039, 2504.07413].
- **Copula-based dependence modeling**: Sieve MLE for unspecified copulas achieves marginal efficiency and avoids propensity for bias seen in parametric specifications [2401.17334].
- **Nonstandard data structures**: Empirical likelihood and full-likelihood approaches extend naturally to interval-censored multi-state models, semicontinuous populations, and selection-biased sampling [2209.07708, 2012.07092].
- **Integration with machine learning**: Deep neural network integration, as in semiparametric frailty models, enables flexible nonlinear modeling while retaining full-likelihood efficiency properties [2307.06581].

Continued methodological research is focused on theoretical properties under ever weaker assumptions, efficient algorithms for ultra-high dimension and big datasets, and inferential procedures under various forms of missingness, censoring, and measurement error.

## 7. Summary Table of Representative Models and Methodologies

| Model/Context                                    | Full-Likelihood Approach                                   | Key Computational Strategy                         |
|--------------------------------------------------|------------------------------------------------------------|----------------------------------------------------|
| Semiparametric exponential family                | Profile likelihood via least favorable curve g↦g_θ         | Smoothing, closed-form profiling                  |
| Proportional likelihood ratio                    | Projection onto tangent space, Z-estimators                 | Neumann series, IPW weighting                     |
| Sieve-likelihood for truncated/censored data     | Spline basis expansion of baseline hazard                   | Newton–Raphson, spline sieve expansion            |
| Multivariate copula models                       | Sieve MLE with Bernstein–Kantorovich copula                 | Constrained optimization, simplex parameterization |
| Semiparametric mixture/log-concave models        | EM with nonparametric log-concave MLE                       | Active-set, EM iterations                         |
| Empirical likelihood for semicontinuous models   | EL with dual profile likelihood for functionals             | Lagrangian, atomic masses, Newton–Raphson         |
| Frailty models/deep learning                     | Negative profiled h-likelihood with profiled-out baseline   | Back-propagation, normalization, alternating minimization |

These strategies are foundational for semiparametric efficiency and practical applicability in complex statistical inference.

Source: https://www.emergentmind.com/topics/semiparametric-full-likelihood-estimators