Minimum Rényi Pseudodistance Estimators

• Minimum Rényi pseudodistance estimators are a family of robust statistical estimators that interpolate between maximum likelihood and bounded influence alternatives using a tuning parameter α.
• They are constructed via explicit score-type estimating equations and iterative reweighting methods to yield asymptotically normal estimates and robust Wald-type tests.
• Extensive theoretical and simulation studies validate their performance in regression, multivariate models, and high-dimensional settings under contamination and model misspecification.

Minimum Rényi pseudodistance estimators (MRPEs) constitute a one-parameter family of robust statistical estimators and associated Wald-type statistical tests, constructed by minimizing the Rényi pseudodistance—a divergence measure parametrized by an order $\alpha > 0$—between the assumed parametric model and the empirical distribution. MRPEs interpolate smoothly between maximum likelihood estimators (MLE, recovered as $\alpha\to0$) and highly robust alternatives with bounded influence functions. Their applications include linear and logistic regression, normal mean and scale inference, bivariate normal models, multiple regression with fixed design, and high-dimensional scenarios, especially in the presence of contamination or model misspecification. MRPEs have explicit score-type estimating equations, asymptotic normality with tractable variance structure, and can be used as the foundation for robust Wald-type tests. Robustness/efficiency trade-offs and the choice of the tuning parameter $\alpha$ are supported by extensive theoretical and simulation studies (Castilla et al., 2021, Castilla, 2024, Jaenada et al., 2022, Broniatowski et al., 2011).

1. Rényi Pseudodistance: Definition and Properties

For two distributions $P$ and $Q$ on a common measurable space (with respective densities $p$ and $q$), the Rényi pseudodistance (also known as the Rényi pseudodivergence) of order $\alpha > 0$ is

$$D_\alpha(P,Q) = \frac{1}{1+\alpha} \log \int p(x)^{1+\alpha}\,dx - \frac{1}{\alpha}\log\int p(x)^\alpha q(x)\,dx.$$

For empirical distributions $Q = P_n$, the pseudodistance becomes

$\alpha\to0$0

As $\alpha\to0$1, $\alpha\to0$2 converges to the Kullback–Leibler divergence $\alpha\to0$3. Positivity and decomposability properties ensure suitability as an estimation criterion (Broniatowski et al., 2011, Castilla, 2024).

For independent non-identically distributed (i.n.i.d.) data $\alpha\to0$4 with corresponding model densities $\alpha\to0$5,

$\alpha\to0$6

serves as the objective function. The prefactor ensures the formal limiting connection to the log-likelihood as $\alpha\to0$7 (Castilla et al., 2021).

2. Minimum Rényi Pseudodistance Estimation

The MRPE is defined as the minimizer of the Rényi pseudodistance with respect to the parameter,

$\alpha\to0$8

Alternatively, maximizing $\alpha\to0$9 is also valid: $\alpha$0 For i.n.i.d. models,

$\alpha$1

with the empirical/robustification features controlled by $\alpha$2.

MRPEs can be explicitly formulated for normal location–scale inference, linear regression, polytomous logistic regression, and bivariate normal models (Castilla et al., 2021, Castilla, 2024, Jaenada et al., 2022, Broniatowski et al., 2011). Importantly, as $\alpha$3, MRPEs coincide exactly with MLEs.

3. Estimating Equations and Algorithms

Differentiation of the objective produces “score-type” estimating equations. For i.i.d. data: $\alpha$4 with $\alpha$5 the parameter gradient of the density.

For i.n.i.d. data, a compact expression for the estimating equation is: $\alpha$6 where $\alpha$7 (Castilla et al., 2021).

Iterative reweighting (IRLS) approaches are effective in models such as bivariate normals. The iterative weights and parameter updates are expressed directly in terms of current parameter estimates and evaluated density values (Jaenada et al., 2022).

4. Asymptotics and Influence Function

Under standard regularity, MRPEs are consistent and asymptotically normal. Specifically,

$\alpha$8

with

$\alpha$9

where $P$0, $P$1, and $P$2 as in the estimating equation (Castilla, 2024, Castilla et al., 2021, Broniatowski et al., 2011). At $P$3, both $P$4 and $P$5 equal the Fisher information.

The influence function (IF) for MRPE at $P$6 is

$P$7

which is bounded in $P$8 for $P$9. For MLE ($Q$0), the IF is unbounded, highlighting the robustifying effect of nonzero $Q$1. In i.n.i.d. models and regression, IF formulae are available in closed form, ensuring B-robustness under mild contamination (Castilla et al., 2021, Broniatowski et al., 2011).

5. Efficiency, Robustness, and the Choice of $Q$2

MRPEs furnish a continuum between the efficient but nonrobust MLE ($Q$3) and estimators with maximal outlier resistance for larger $Q$4. The loss in Fisher information (asymptotic efficiency) is typically minor for moderate $Q$5 but the gain in robustness—e.g., bounded influence, minuscule bias and mean squared error under contamination—is substantial in finite samples (Broniatowski et al., 2011, Castilla, 2024). The asymptotic relative efficiency (ARE) with respect to MLE is, for example, $Q$6 for $Q$7 and $Q$8 for $Q$9 in the polytomous logistic regression model (Castilla, 2024).

A practical selection for $p$0 lies in $p$1, with specific choices often guided by empirical MSE minimization (e.g., Warwick–Jones method), though this could be computationally costly and a fixed value suffices in many scenarios (Jaenada et al., 2022). Monte Carlo studies consistently validate the robustness–efficiency trade-off (Broniatowski et al., 2011).

6. Applications: Regression and Multivariate Models

In multiple linear regression ($p$2), the MRPE objective is: $p$3 with estimating equations: $p$4 where $p$5 (Castilla et al., 2021, Broniatowski et al., 2011).

In the bivariate normal setting with $p$6, IRLS algorithms allow efficient computation of MRPEs and operate by iteratively updating weighted means, variances, and covariances (Jaenada et al., 2022).

For polytomous logistic regression, MRPEs demonstrate high classification robustness and maintain low misclassification rates under response-label contamination for $p$7–$p$8 (Castilla, 2024).

7. Robust Wald-Type Tests

For hypotheses $p$9, Wald-type test statistics based on MRPEs follow: $q$0 Under $q$1, $q$2 is asymptotically $q$3, and for $q$4, test level remains near nominal under moderate contamination, while classical Wald tests with MLE break down (Castilla, 2024, Castilla et al., 2021, Jaenada et al., 2022). MRPE-based Wald tests are thus directly robustified by the properties of the estimators themselves.

---

References

• (Castilla et al., 2021) Estimation and testing on independent not identically distributed observations based on Rényi's pseudodistances
• (Castilla, 2024) A new robust approach for the polytomous logistic regression model based on Rényi's pseudodistances
• (Jaenada et al., 2022) Robust approach for comparing two dependent normal populations through Wald-type tests based on Rényi's pseudodistance estimators
• (Broniatowski et al., 2011) Decomposable Pseudodistances and Applications in Statistical Estimation