Bivariate Contaminated Normal Models

• Bivariate contaminated normal models are defined as mixtures of a dominant ‘good’ bivariate normal component and a minor, variance-inflated ‘bad’ component to capture outliers.
• They utilize ECM/EM algorithms and eigen-decomposition techniques for parameter estimation, ensuring both model identifiability and robust clustering performance.
• Applications in clustering, regression, and dependence analysis demonstrate enhanced estimation accuracy and automatic outlier detection under mild contamination.

A bivariate contaminated normal model is a probabilistic construct wherein the observed data are generated from a mixture of bivariate normal populations—typically a dominant “good” component and a minor “bad” component that accounts for mild outliers or impulsive contamination. This modeling approach is foundational in robust statistics, graphical modeling, clustering, and dependence analysis, as it directly encodes contamination via elliptical symmetry and variance inflation, and equips inference procedures with mechanisms for both robust estimation and outlier detection.

1. Model Formulation and Theoretical Properties

Let $(X, Y) \in \mathbb{R}^2$ represent bivariate observations. The contaminated normal model is specified as

$$(X, Y) \sim (1-\epsilon) \cdot N_2(\mu, \Sigma, \rho) + \epsilon \cdot N_2(\mu, \Lambda, \rho')$$

where $N_2(\mu, \Sigma, \rho)$ is a bivariate normal density with mean $\mu$, covariance $\Sigma$, and correlation $\rho$. The contamination fraction $0 < \epsilon \ll 1$ dictates the proportion of data from the “bad” or outlier component, which shares the mean but differs by a variance inflation ($\Lambda_{ii} = \lambda_i^2 \Sigma_{ii}$, $\lambda_i \gg 1$) and possibly altered correlation $\rho'$. This construction supports both symmetric (mild) and asymmetric (impulsive) contamination.

Closed-form expressions for expectations of rank-based correlation coefficients have been established. For example, for Spearman’s rho ($r_S$) and Kendall’s tau ($r_K$), the limiting expectations under contamination are

$$\begin{aligned} \lim_{\epsilon \to 0, \lambda_i \to \infty} E(r_K) &= \frac{2}{\pi} \left[(1 - 2\epsilon)\sin^{-1}\rho + 2\epsilon \sin^{-1}\rho'\right]\ \lim_{\epsilon \to 0, n \to \infty, \lambda_i \to \infty} E(r_S) &= \frac{6}{\pi} \left[(1 - 3\epsilon)\sin^{-1}(\rho/2) + \epsilon \sin^{-1}\rho'\right] \end{aligned}$$

These formulas quantitatively demonstrate how contamination—even at vanishing $\epsilon$—shifts correlation metrics towards the outlier component (Xu et al., 2010).

The model’s variance and covariance structure is nontrivial; for instance, the variance of Spearman’s rho incorporates non-elementary functions, requiring numerical tabulation (see Childs’s reduction formula and Table I in (Xu et al., 2010)). Covariance between $r_S$ and $r_K$ is also given by explicit integrals of orthant probabilities.

2. Parameter Estimation and Identifiability

Parameter estimation is commonly approached via expectation-conditional maximization (ECM) or variants of the expectation-maximization (EM) algorithm. In mixtures of contaminated normals for clustering, the likelihood is

$$f(x; \theta_g) = \alpha_g \, \phi(x; \mu_g, \Sigma_g) + (1-\alpha_g)\phi(x; \mu_g, \eta_g \Sigma_g)$$

with $\phi$ the multivariate normal. The mixture is parameterized by mixing proportions $\pi_g$, contamination probabilities $\alpha_g$, contamination inflation factors $\eta_g > 1$, and cluster means/covariances. Parameters $\alpha_g$, $\eta_g$ are estimated via closed-form updates or numerical maximization, not pre-specified, ensuring flexibility and model identifiability (Punzo et al., 2013, Punzo et al., 2016).

For selection models (e.g., bivariate Heckman contaminated normal), identifiability is established by natural constraints on contamination parameters (e.g., $\nu_2 \in (0,1)$) (Lim et al., 18 Sep 2024).

Eigen-decomposition of covariance matrices yields parsimonious models: $$\Sigma_g = \lambda_g \Gamma_g \Delta_g \Gamma_g^\top$$ with $\lambda_g$ (volume), $\Delta_g$ (shape), $\Gamma_g$ (orientation), facilitating model selection, identifiability, and reduction in free parameters.

3. Asymptotic Behavior and Robustness

In contaminated settings, the consistency of classical estimators (e.g., sample mean) hinges on the contamination fraction and the degree of variance inflation. Weak consistency of the mean persists if

$$\lim_{n \to \infty} \frac{1}{n^2} \sum_{k=1}^n p_k \sigma_k^2 = 0$$

where $p_k$ is contamination probability, $\sigma_k^2 \ge 1$ (Berckmoes et al., 2015). Asymptotic normality of normalized sample means is ensured under suitably decaying contamination, while more severe contamination results in only approximate normality, quantified by the Lindeberg index—see Kolmogorov distance bounds.

For Bayesian estimation, posterior robustness in contaminated regression models can be achieved under milder conditions when the heavy-tailed contaminating density $f_1$ is independent of regression parameters. The influence of outliers vanishes as their magnitude increases, provided $f_1$ has sufficiently heavy tails relative to the prior; even Student's t errors yield robustness in this mixed model (Hamura et al., 2023).

4. Practical Estimation and Outlier Detection

The ECM algorithm (and AECM in high dimensions) leverages two latent structures: cluster labels $z_{ig}$ and latent outlier indicators $v_{ig}$. Weighted updates for means and covariances depend on posterior probabilities: $$w_{ig}^{(r)} = v_{ig}^{(r)} + \frac{1-v_{ig}^{(r)}}{\eta_g^{(r)}}$$ which down-weight high Mahalanobis distance observations, providing automatic detection. The MAP classification $v_{ig} \leq 0.5$ flags outliers (Punzo et al., 2016).

Directional robustness is available in MSCN models, where each dimension is assigned its own contamination parameters $(\alpha_h, \eta_h)$, permitting detection of "bad" points per coordinate (Punzo et al., 2018).

In matrix variate models, posterior probabilities for each matrix observation are computed, yielding a two-step outlier flagging mechanism with enhanced recovery of underlying cluster structure under contamination (Tomarchio et al., 2020).

5. Statistical Inference, Testing, and Modal Structure

In contaminated bivariate normal models, inference for dependence (correlation) diverges for different estimators. Spearman’s $\rho$ and Kendall’s $\tau$ behave differently in terms of finite-sample bias, variance, MSE, and asymptotic relative efficiency (ARE):

• Biases: symmetric in $\rho$, vanish at $\rho=0, \pm1$; magnitudes differ, e.g., SR exhibits greater bias near $|\rho| \ne 0$.
• ARE: both below unity compared to Pearson's correlation; KT enjoys higher ARE, particularly for large $|\rho|$ (Xu et al., 2010).
• MSE: depends on true $\rho$ and sample size $n$; KT-based estimator typically outperforms in high correlation regimes.

Detection of sparse positive dependence under contamination is addressed by higher criticism tests constructed on pairwise differences or ranks. The adaptive HC statistic achieves the parametric detection boundary for contamination fraction $\epsilon$ and correlation $\rho$ (Arias-Castro et al., 2018). However, nonparametric, rank-based tests (HC-rank) lose power in the very sparse regime ($\beta > 3/4$), demonstrating intrinsic limitations.

Modal structure in bivariate normal mixtures, relevant for contaminated models, is governed by singularity theory. Classification via $\mathcal{A}$-equivalence yields three types, with modality bounded—general (non-codirectional) mixtures can have up to three modes, while codirectional and proportional cases (covariances aligned) are limited to two (Kabata et al., 1 Oct 2024).

6. Applications and Implications for Real Data Analysis

Simulation studies and real data examples consistently show that contaminated normal mixture models outperform classical normal and t-mixtures under mild outlier contamination. Applications cover artificial bivariate data (e.g., with injected uniform noise or high-leverage points), blue crabs and wine datasets, and benchmark econometric data (RAND Health Insurance, Mroz labor supply) (Punzo et al., 2013, Punzo et al., 2016, Lim et al., 18 Sep 2024).

Robust clustering is achieved via automatic down-weighting of atypical observations and information criteria-based model selection. In sample selection contexts, contaminated normal error models yield improved estimate stability, better fit diagnostics, and genuine outlier flagging compared to SLn and SLt alternatives.

Outlier detection via maximum a posteriori probabilities is a core feature, facilitating both robust estimation and interpretability. Directional detection allows fine-grained assessment in multivariate/bivariate settings. Parsimonious models promote computational tractability and avoid overfitting through eigen-decomposition constraints.

7. Extensions and Theoretical Perspectives

Approximate stochastic order under contamination generalizes rigid ordering assumptions by allowing two distributions to “almost” satisfy stochastic order, quantified by a contamination level $T_0$. Trimming fractions are derived, providing a statistical index for order deviation, with simulation studies verifying sensitivity and robustness in bivariate normal settings (Alvarez-Esteban et al., 2014).

Equi-dispersed bivariate normal conditionals introduce a related conceptual frame, constraining conditional means and variances to be equal. The resulting exponential family is flexible and, though not a contaminated model per se, the approach complements contamination-based robustness by allowing additional distributional flexibility (Arnold et al., 2022).

Summary Table: Key Features in Bivariate Contaminated Normal Models

Aspect	Technical Feature	Core References
Model formulation	Mixture of bivariate normals with variance inflation	(Punzo et al., 2013, Xu et al., 2010)
Estimation algorithm	ECM/AECM; closed-form updates; eigen-decomposition	(Punzo et al., 2016, Punzo et al., 2014)
Outlier detection	MAP probabilities; directionality per dimension	(Punzo et al., 2018, Tomarchio et al., 2020)
Inference (correlation, dependence)	Bias, MSE, ARE, higher criticism, modal bounds	(Xu et al., 2010, Arias-Castro et al., 2018, Kabata et al., 1 Oct 2024)
Robustness	Down-weighting, posterior insensitivity to outliers	(Hamura et al., 2023, Lim et al., 18 Sep 2024)

The bivariate contaminated normal model is a cornerstone in robust multivariate analysis, providing both theoretical guarantees and practical algorithms for clustering, regression, dependence detection, and outlier identification in the presence of mild to moderate contamination. The formalism is flexible, computationally accessible, and has been empirically validated across simulation and real datasets, with ongoing developments in model selection, modal analysis, and Bayesian robustness.