Higher-Order Corrections in Likelihood Inference

• Higher-order corrections in likelihood inference are advanced techniques that refine standard approximations by incorporating additional terms to reduce finite-sample errors.
• They include Barndorff–Nielsen adjustments and Bartlett corrections that enhance p-value accuracy and confidence interval coverage in small-sample or complex settings.
• These methods address challenges such as high-dimensional nuisance parameters and model misspecification through explicit expansions and analytical corrections.

Higher-order corrections in likelihood-based inference encompass a suite of methods that refine conventional likelihood asymptotics by incorporating additional terms or transformations—typically to attain enhanced accuracy in finite-sample inference, address model-dependent deviations, or handle complex settings such as high-dimensional nuisance parameters, nonparametric targets, and misspecified or non-Gaussian models. Classic first-order likelihood-based inference employs normal or chi-square approximations to the distribution of likelihood ratio statistics; higher-order techniques use expansions or adjustments to reduce error rates, delivering more precise p-values and confidence intervals, often with non-negligible impact in realistic data situations.

1. Foundations of Likelihood-based Inference and the Need for Higher-order Corrections

Classical likelihood-based inference for a parameter $\theta$ proceeds via the log-likelihood $\ell(\theta)$, the maximum likelihood estimator (MLE) $\hat\theta$, and likelihood ratio statistics. The signed root likelihood ratio statistic,

$$r(\theta_0) = \operatorname{sign}(\hat\theta - \theta_0) \sqrt{2 \left[ \ell(\hat\theta) - \ell(\theta_0) \right]},$$

is typically approximated as $N(0,1)$ up to $O(n^{-1/2})$ error under standard regularity assumptions, justifying Wald-type confidence intervals and $p$-values. However, this first-order approximation may be inaccurate when samples are small, when there are many nuisance parameters relative to information about $\theta$, or when the likelihood exhibits significant skewness or higher cumulant structure. Empirical and analytical studies indicate substantial undercoverage or anti-conservative inference in these cases, especially in meta-analysis, random-effects models, measurement error models, and semi/nonparametric settings (Guolo, 2018, DiCiccio et al., 2015, Canonero et al., 2023).

The primary objective of higher-order correction is to systematically reduce these approximation errors—typically to $O(n^{-1})$ (second order), $O(n^{-3/2})$ (third order), or beyond—by explicit expansion, transformation, or augmentation of the basic likelihood-based quantities.

2. Core Higher-order Adjustment Techniques

The two principal approaches for achieving higher-order accuracy in likelihood-based inference are:

(a) Barndorff–Nielsen–style Modified Likelihood Roots ($r^*, R^*$, or $p^*$ approximations)

The modified directed likelihood root for a scalar parameter is constructed as

$$r^*(\theta) = r(\theta) + \frac{1}{r(\theta)} \log \frac{q(\theta)}{r(\theta)},$$

where $q(\theta)$ is a correction factor involving observed-score derivatives, profile and observed Fisher information, and potentially determinant ratios from block matrices when nuisance parameters are present (Blasi et al., 2016, Tang et al., 2022, DiCiccio et al., 2015, Canonero et al., 2023). For instance, in models with nuisance parameters $\lambda$,

$$q(\theta) = \text{function}\left( \frac{|\text{profile/observed information}|}{|\text{full information}|}, \text{other derivatives} \right).$$

This adjustment results in $r^*(\theta)$ having a distribution approximating $N(0,1)$ up to $O(n^{-3/2})$, providing third-order accuracy for one-sided tail probabilities and delivering confidence intervals with higher-fidelity coverage.

Barndorff–Nielsen’s $p^*$ approximation refines the finite-sample density of the MLE as

$$p^*(\hat\theta;\theta) = \phi\left(r(\theta)\right) \sqrt{\frac{r(\theta)}{q(\theta)}} \exp\left\{ \frac{1}{12 r(\theta)}[r(\theta)^3 - q(\theta)^3] \right\} + O(n^{-3/2}).$$

(b) Bartlett Correction of the Likelihood Ratio Statistic

Bartlett’s correction addresses the mean bias of the likelihood ratio statistic $W(\theta) = 2[\ell(\hat\theta) - \ell(\theta)]$ by rescaling: $w_B(\theta) = \frac{W(\theta)}{1 + b(\theta)},$ where $b(\theta)$ is the Bartlett factor computed from cumulants or higher derivatives (Lawley’s formula) of the likelihood up to fourth order. This correction aligns the mean of $w_B(\theta)$ with its nominal $\chi^2$ distribution to $O(n^{-2})$, significantly improving the fidelity of $p$-values and intervals based on this approach (Canonero et al., 2023, Liu et al., 2010, DiCiccio et al., 2015).

(c) Median Bias Correction and Confidence Distributions

Confidence intervals derived by median bias correction consider inversion at the median function $b(\theta)$, where $G(b(\theta);\theta) = 0.5$ for the CDF of the MLE. The resulting (bias-corrected) deviance $w^*(\theta)$ yields a confidence distribution or corrected confidence curve with equal-tailed properties—achieving third-order coverage accuracy in regular one-parameter exponential families (Blasi et al., 2016).

3. Nuisance Parameters and Decomposition of Higher-order Adjustments

Presence of nuisance parameters substantially complicates higher-order refinement. The total higher-order adjustment (e.g. in $R^*(\theta)$) can be decomposed into two interpretable terms (DiCiccio et al., 2015, Tang et al., 2022):

• INF (Information) Adjustment: Captures intrinsic deviation from normality (skewness, kurtosis) inherent to the scalar parameter, independent of nuisance structure.
• NP (Nuisance Parameter) Adjustment: Quantifies distortion due to nuisance parameters—this term can dominate in high-dimensional settings and is identically zero if there are no nuisances.

These decomposed adjustments influence both the normal approximation to $R^*(\theta)$ and Bartlett’s correction, with the nuisance adjustment typically scaling linearly or quadratically with nuisance dimension. For example, in a Gaussian linear regression with $p$ predictors, the leading second-order mean adjustment is $n^{1/2}\mu_{NP} = (p-1)/\sqrt{2}$, implying rapid worsening as $p$ increases for fixed sample size (DiCiccio et al., 2015).

4. Empirical and Theoretical Performance Analysis

Extensive simulation studies and theoretical expansions confirm the practical impact of higher-order corrections:

• In control rate regression meta-analysis, Skovgaard’s statistic (a form of modified likelihood root) achieves near-nominal coverage for $n$ as small as 5 and for substantial between-paper heterogeneity, far outperforming both WLS and first-order likelihood-ratio statistics (Guolo, 2018).
• In models with unknown higher-order theoretical uncertainties, as in effective field theories, treating missing higher-order terms as zero-mean Gaussian random variables and integrating them into the covariance restores the exact $\chi^2$ distribution for the profiled test statistic—yielding rigorous $p$-values and coverage (Berthier et al., 2016).
• Adjusted empirical likelihood (AEL) with augmentation and optimal adjustment level achieves $O(n^{-2})$ coverage for confidence regions and eliminates solution-existence issues that plague standard empirical likelihood in small or high-dimensional samples (Liu et al., 2010).
• In complex settings with “errors on errors” (Gamma Variance Model), both $r^*$-based and Bartlett-corrected statistics yield confidence intervals and $p$-values that closely match fully simulated coverage for moderate samples and large error-on-error parameters (Canonero et al., 2023).
• Higher-order targeted maximum likelihood estimation (k-th order TMLE) for nonparametric functionals provides expansions valid up to $k+1$ order in the sample measure, with remainder terms that are negligible if the initial estimator converges slowly; this enables valid Wald-style inference without harsh undersmoothing (Laan et al., 2021).

5. Implementation Principles and Computational Considerations

Higher-order correction procedures typically require calculation of profile and observed Fisher information, score functions, and their derivatives (up to third or fourth order). This involves:

• Profile optimization of the likelihood for the parameter of interest, conditional on nuisance parameters.
• Numerical computation of derivatives or determinant ratios of Hessian blocks; in canonical models these often admit closed forms.
• Matrix operations (inverse, determinant) and gradient/Hessian evaluations; in most practical use (e.g., meta-analyses, $p<10$) all matrix dimensions are small and computation is rapid.
• For adjusted empirical likelihood, a pseudo-observation is constructed by augmenting the data, and root-finding is applied to solve the adjusted estimating equation for each $\theta$ (Liu et al., 2010).
• For missing higher-order corrections, estimation of variance parameters for unknown terms may be based on dimensional analysis or explicit power-counting logic in physical models (Berthier et al., 2016).

In practice, these calculations are amenable to implementation in R, Matlab, or other numerical languages, and codes for specific cases (e.g., control_rate_regression_LRTs.R) are directly available (Guolo, 2018).

6. Extensions, Generalizations, and Regularity Conditions

Higher-order methods generalize across:

• Scalar and vector parameters via Edgeworth and Cornish–Fisher expansion, law of large numbers for cumulants, and explicit polynomial expansions in the signed likelihood root.
• Nonparametric and semiparametric models, where k-th order targeting updates (TMLE) yield expansions with remainder terms of order $k+1$ (Laan et al., 2021).
• Arbitrary-order perturbation in theoretical models, where uncomputed higher-order effects are systematically accounted for through prior modeling and exact marginalization (Berthier et al., 2016).
• Regularity requirements include existence of derivatives of the log-likelihood to sufficient order, boundedness and nonsingularity of the Fisher information, absence of boundary or multiple maxima issues, and adequate moment conditions if using empirical likelihood or cumulant expansions (Blasi et al., 2016, Liu et al., 2010).

7. Limitations and Caveats

While higher-order corrections offer substantial improvements, caveats include:

• Assumptions about the distribution of uncomputed terms (e.g., Gaussianity) may not hold in all settings; heavy tails or outliers in the missing term distribution can lead to underestimation of tail probabilities (Berthier et al., 2016).
• For empirical likelihood and similar methods, in very high dimensions ($q \sim n$), root-$n$ expansions and moment estimates may lose accuracy, requiring empirical adjustment or bootstrapping (Liu et al., 2010).
• Bartlett correction and higher-order expansions rely critically on correct model specification and the availability of required higher-order derivatives or moments.
• The computational burden is generally moderate but may be nontrivial for grid-based profiling in multi-parameter settings or when high-order moments are unstable or computationally intensive to estimate.

In summary, higher-order corrections in likelihood-based inference constitute rigorously justified and practically validated modifications to classical likelihood inference. These approaches provide formal and computationally tractable means to address the inadequacies of first-order likelihood asymptotics, yielding improved accuracy for $p$-values and confidence sets across a substantial array of parametric, semi/nonparametric, and applied statistical models.