Bernstein-Smoothed Feasible Estimator

• The Bernstein-Smoothed Feasible Estimator is a semiparametric approach that uses Bernstein polynomial bases to actively correct bias and ensure estimator feasibility.
• It leverages a Bernstein–von Mises expansion and functional Taylor series to secure asymptotic normality and optimal variance bounds even under low regularity.
• The estimator is effectively implemented in density estimation via random histograms and Gaussian process priors, unifying frequentist and Bayesian methods.

The Bernstein-Smoothed Feasible Estimator denotes a class of nonparametric or semiparametric estimators whose construction and theoretical justification rely fundamentally on the Bernstein–von Mises (BvM) phenomenon, functional expansions, and explicit convolution or smoothing using Bernstein polynomial bases. The term refers to estimators that incorporate (often explicit) bias corrections so as to achieve asymptotic normality and optimal variance bounds in settings where direct plug-in estimators are biased or non-feasible, especially for nonlinear functionals or under low regularity conditions. This concept emerged in the paper of Bayesian posterior distributions of semiparametric functionals, with particular focus on density estimation using random histograms and Gaussian process priors (Castillo et al., 2013).

1. The Semiparametric BvM Framework

The foundational theory analyzes a statistical experiment indexed by an infinite-dimensional parameter $\eta$ and a smooth functional $\psi : S \to \mathbb{R}$. The log-likelihood is locally expanded: $$\ell_n(\eta) - \ell_n(\eta_0) = -\frac{n}{2}\|\eta-\eta_0\|_L^2 + \sqrt{n}W_n(\eta-\eta_0) + R_n(\eta, \eta_0)$$ where $W_n$ is an asymptotically Gaussian process and $\|\cdot\|_L$ is the LAN (local asymptotic normality) norm. The functional $\psi$ is Taylor-expanded at $\eta_0$: $$\psi(\eta) = \psi(\eta_0) + \langle \psi_0^{(1)}, \eta-\eta_0\rangle_L + \frac{1}{2}\langle \psi_0^{(2)}(\eta-\eta_0), \eta-\eta_0\rangle_L + r(\eta, \eta_0)$$ with $\psi_0^{(1)}$ the first-order efficient influence function and $\psi_0^{(2)}$ a second-order operator capturing nonlinearity or low regularity effects.

2. Bernstein Smoothing and Bias Correction

A key methodological innovation is the introduction of an explicit change-of-parameter path $\eta \mapsto \eta_t$ to absorb and correct for semiparametric bias:

• First-order (linear functionals): $\eta_t = \eta - \frac{t}{\sqrt{n}}\psi_0^{(1)}$
• Second-order (nonlinear/low regularity): $$\eta_t = \eta - \frac{t}{\sqrt{n}}\psi_0^{(1)} - \frac{t}{2\sqrt{n}}\psi_0^{(2)}(\eta-\eta_0) - \frac{t}{2n}\psi_0^{(2)}w_n$$

The Laplace functional of the posterior over these shifted paths leads to a BvM expansion: $$E^\Pi[e^{t\sqrt{n}(\psi(\eta)-\hat{\psi})} | Y^n, A_n ] = \exp\Bigl\{ o_p(1) + \frac{t^2 V_{0,n}}{2} \Bigr\} e^{\mu_n t} (1+o_p(1))$$ yielding the central limit theorem for recentered plug-in estimators: $$n(\psi(\eta) - \hat{\psi} ) - \mu_n \xrightarrow{d} N(0, V_0)$$ A Bernstein-smoothed feasible estimator thus results from explicit bias correction (by shifting via $\mu_n$ and incorporating second-order effects as structured by the Bernstein polynomial representation), ensuring both computational tractability and optimal frequentist variance.

3. Implementation in Density Estimation

For density estimation on $[0, 1]$, two prior models are featured:

• Random Histograms: The density $f$ is piecewise constant on $k$ bins, with bin heights given a Dirichlet prior. The plug-in (projection) estimator $\hat{\psi}_k$ is generally biased, with corrections such as $-2K_n/n$ needed for quadratic functionals. Bernstein smoothing adapts the estimator by bias subtraction along the explicitly constructed path, leading to feasible and efficient estimation.
• Exponentiated Gaussian Processes: The density is modeled by $f(x) = \exp(W(x))/\int_0^1 \exp(W(u)) du$, with $W$ a Gaussian process over $[0, 1]$. Bernstein smoothing is employed by projecting the efficient influence function into the RKHS, and bias correction ensures that the difference $n\|\psi_n - \tilde\psi_{f_0}\|_\infty \to 0$, satisfying the Bernstein–von Mises condition.

A critical “no-bias” condition is required: for Bernstein-smoothed feasibility, the difference between the influence function and its Bernstein polynomial projection (or its variant for GP priors) must vanish fast enough so as not to disturb asymptotic normality.

4. Nonlinearity and Low Regularity Handling

When the functional $\psi$ is nonlinear (e.g., $\psi(f) = \int f^2$) or the density $f$ lacks regularity, plug-in estimators can fail to be first-order efficient. The methodology retains full quadratic expansion and path correction terms, capturing bias at the $1/\sqrt{n}$ level and providing an explicit formula for the Laplace transform. After recentering via the bias term $\mu_n$, Bernstein smoothing yields a feasible estimator with Gaussian posterior and efficient variance.

5. Synthesis and Properties

The general theory and implementation establish:

• Sufficient conditions (LAN, functional expansion, change-of-parameter) for a Bernstein–von Mises theorem for semiparametric functionals
• Explicit bias correction formulas and handling of nonlinear functionals/low regularity
• Bernstein–smoothed feasible estimators in density estimation under random histogram and GP prior models, both correcting discretization and projection biases
• Asymptotic normality (Gaussian posterior) and efficient variance
• Applicability to estimation and uncertainty quantification in broad nonparametric and semiparametric problems

Relevant formulas summarizing the Bernstein-smoothed feasible estimator construction include:

• LAN expansion:

$$\ell_n(\eta) - \ell_n(\eta_0) = -\frac{n}{2}\|\eta-\eta_0\|_L^2 + \sqrt{n} W_n(\eta-\eta_0) + R_n(\eta,\eta_0)$$

• Functional expansion:

$$\psi(\eta) = \psi(\eta_0) + \langle \psi_0^{(1)},\eta-\eta_0\rangle_L + \frac{1}{2}\langle \psi_0^{(2)}(\eta-\eta_0),\eta-\eta_0\rangle_L + r(\eta,\eta_0)$$

• Approximating path (second-order case):

$$\eta_t = \eta - \frac{t\,\psi_0^{(1)}}{\sqrt{n}} - \frac{t\,\psi_0^{(2)}(\eta-\eta_0)}{2\sqrt{n}} - \frac{t\,\psi_0^{(2)}w_n}{2n}$$

• Laplace transform for posterior:

$$E^\Pi\Bigl[ e^{t\sqrt{n}(\psi(\eta)-\hat{\psi})} \mid Y^n, A_n \Bigr] = \exp\Bigl\{ o_p(1) + \frac{t^2 V_{0,n}}{2} \Bigr\} \frac{\int_{A_n}e^{\ell_n(\eta_t)-\ell_n(\eta_0)}\,d\Pi(\eta)}{\int_{A_n}e^{\ell_n(\eta)-\ell_n(\eta_0)}\,d\Pi(\eta)}$$

• BvM convergence:

$$n\Bigl(\psi(\eta)-\hat{\psi}\Bigr) - \mu_n \xrightarrow{d} N(0, V_0)$$

6. Connections and Implications

The Bernstein–Smoothed Feasible Estimator is characterized by explicit construction and bias correction, typically achieved via a combination of Taylor expansions, Bernstein polynomial projections, and path shifting. In practice, this approach unifies frequentist and Bayesian perspectives—posterior distributions of smooth (even nonlinear) functionals become tractable, asymptotically normal, and attain the semiparametric information bound, even in the presence of infinite-dimensional nuisance parameters or low regularity.

A plausible implication is that Bernstein–smoothed feasible estimation provides a blueprint for constructing efficient estimators in semiparametric models where direct plug-in approaches are biased or infeasible. This extends to modern machine learning models with high-dimensional or functional targets as well.

7. Summary Table: Characteristic Features

Feature	Description	Implications
Smoothing mechanism	Bernstein polynomial basis, functional expansion, bias correction	Achieves efficient variance
Applicability	Density estimation, nonlinear functionals, low regularity, infinite-dimensional models	General nonparametrics
Bias correction	Explicit path shifting, second-order terms for nonlinearity/irregularity	Recovers asymptotic normality
Posterior behavior	Asymptotic Gaussian (Bernstein–von Mises), with variance matching efficiency bound	Valid inference

In conclusion, the Bernstein-Smoothed Feasible Estimator comprises plug-in estimators corrected by explicit smoothing and bias adjustment, grounded in the semiparametric BvM expansion. This guarantees both feasibility and optimality in terms of variance, even under model complexity, nonlinearity, and low regularity. The methodology has been extensively formalized for density estimation models using random histogram and Gaussian process priors (Castillo et al., 2013), and is applicable in broader semiparametric contexts where bias management and efficient uncertainty quantification are critical.