---
title: Minimum Absolute Distance Estimator
url: https://www.emergentmind.com/topics/minimum-absolute-distance-estimator
type: topic
---

# Minimum Absolute Distance Estimator

The minimum absolute distance estimator (also known as minimum L¹-distance estimator or MAD) refers, in its most general form, to the estimation of unknown model parameters—or probability measures—by minimizing the L¹ absolute distance between empirical and model distributions or functionals. This criterion is widely used for robust parameter estimation in parametric models, for robust location in multivariate statistics, for mixing measure recovery in mixture models, and for functional estimation of distributional divergence. Despite its conceptual simplicity, the minimum absolute distance approach encompasses a diverse range of estimation methodologies, each with distinct theoretical properties and computational challenges.

## 1. Fundamental Definitions and General Framework

The minimum absolute distance estimator is defined by selecting the parameter value (or measure) that minimizes the L¹ (absolute) distance between the empirical distribution (or statistic) and the model-implied counterpart.

- **General parametric CDF model**: Given data $X_1, \ldots, X_n$ and parametric CDF $F(x;\theta)$,
  $$
  \hat \theta_{\mathrm{MAD}} = \arg\min_{\theta \in \Theta} \int_{-\infty}^\infty |F_n(x) - F(x;\theta)| \, dx,
  $$
  where $F_n$ is the empirical CDF [2207.06086].

- **Non-normalized densities:** For families $p_\theta(x)$ (possibly unnormalized), $w(t)>0$, and characterization function $\eta_n(t, \theta)$,
  $$
  \widehat \theta_{n,1} = \arg\min_{\theta \in \Theta} \int_0^\infty \big| \eta_n(t, \theta) \big| w(t) \, dt,
  $$
  where
  $$
  \eta_n(t, \theta) = -\frac{1}{n} \sum_{j=1}^n \frac{p'_\theta(X_j)}{p_\theta(X_j)} \min\{X_j, t\} - \frac{1}{n} \sum_{j=1}^n \mathbf{1}\{X_j \leq t\}
  $$ 
  [1909.00002].

- **Mixture models and mixing measures:** For a class of functions $\Phi$ (e.g., $\|\phi\|_\infty \leq 1$), the $\Phi$-distance between mixing measures $G, G'$ is
  $$
  D_\Phi(G, G') = \sup_{\phi \in \Phi} \left| \int \phi \, dG - \int \phi \, dG' \right|.
  $$
  The minimum absolute (total variation) distance estimator for $k$-component mixtures is
  $$
  \hat G_n \in \arg\min_{G \in \mathcal{M}_k} \|P_n - P_G\|_{TV}
  $$
  [2304.10052].

- **Multivariate robust location:** The minimum (trimmed) absolute distance estimator for location in $\mathbb{R}^p$ is
  $$
  (\hat\theta, T^*) = \arg\min_{\theta \in \mathbb{R}^p,\, T \subseteq X_n,\, |T|=h} \sum_{x_i \in T} \|x_i - \theta\|_1
  $$
  [1511.04220].

## 2. Methodological Variants and Computational Strategies

The core MAD criterion admits multiple concrete implementations, depending on statistical context and model structure.

- **Untrimmed case:** For univariate location, the solution is the sample median. In higher dimensions, the LTAD approach (least trimmed absolute deviations) generalizes the median by trimming largest residuals and minimizing the L¹-loss [1511.04220].
- **Parametric density models:** Objective minimization often requires numerical integration over sample statistics or surrogate loss function evaluation, with optimization performed via L-BFGS-B, SLSQP, or BFGS in scientific computing environments [2207.06086, 1909.00002].
- **Mixture models:** The TV distance minimization forms a nonconvex program often approximated by discretization, global optimization for small $k$, or use of reproducing kernel Hilbert space surrogates (e.g., MMD distances) in the multivariate case [2304.10052].
- **LP/MILP formulations:** Robust location estimation via LTAD is rewritten as a mixed integer linear program (MILP) with auxiliary variables, or as a linear program (LP) via relaxation and data centering techniques [1511.04220].

| Application           | Objective Function                    | Numerical Approach         |
|-----------------------|--------------------------------------|---------------------------|
| Parametric CDF model  | $\int |F_n(x) - F(x;\theta)| dx$     | Grid or sample-based sum + BFGS or similar optimizer |
| Multivariate LTAD     | $\sum_{i\in T} \|x_i - \theta\|_1$   | MILP, LP, Subgradients    |
| Mixture Models        | $\|P_n - P_G\|_{TV}$                 | Grid search, global optimization, kernel methods |

## 3. Statistical Properties

The minimum absolute distance estimator possesses distinct statistical properties governed by both the L¹-norm structure and the trimming (if any).

- **Consistency:** Under standard identifiability and regularity conditions, MAD estimators are consistent:
  - For continuous parametric families, strong consistency holds under compactness and continuity of the loss function in the parameter [1909.00002, 2207.06086].
  - For mixture models, minimax rates in Wasserstein metrics are derived, matching lower bounds for both uniform and local settings [2304.10052].
- **Asymptotic Normality:** Asymptotic normality holds for MAD in both simple and some robust/truncated settings, with covariance matrices expressed via influence functions or explicit integrals involving the model CDF gradient and sign functions [2207.06086, 1511.04220].
- **Breakdown Robustness:** The LTAD achieves finite-sample breakdown points up to $50\%$ for maximal trimming, preserving robustness properties akin to the univariate median [1511.04220].
- **Minimax Properties:** When used to estimate functional divergences (e.g., $L_1$-distance between discrete distributions), minimax rate-optimality holds up to logarithmic factors, with MAD-type estimators achieving superior performance relative to naïve plug-in approaches in the large-alphabet regime [1705.00807].

## 4. Applications and Empirical Performance

Minimum absolute distance estimators are widely applied:

- **Lomax (Pareto-II) distribution:** For two-parameter modeling, MAD outperforms classical MLE and method-of-moments in small samples and remains competitive in moderate samples, with smallest mean squared error (MSE) observed for $n < 100$ [2207.06086].
- **Short and long-memory time series:** In ARFIMA models, bias-corrected minimum distance estimators (BCMDE) employing MAD-type criteria yield reduced bias and smallest MSE, especially when the sample mean is nonconstant and for larger memory parameters [1809.07415].
- **Mixture estimation:** In finite mixtures with compactly supported parameter spaces, minimum TV (L¹) estimators converge at optimal rates in Wasserstein ($W_1$) distance, matching minimax lower bounds and generalizing method of moments and Kolmogorov–Smirnov-type approaches [2304.10052].
- **High-dimensional robust statistics:** LP-based LTAD estimators offer computationally scalable, strongly robust location estimates with virtually zero bias, outperforming classical counterparts under contamination [1511.04220].

| Setting                      | Key Property                          | Empirical Outcome                 |
|------------------------------|---------------------------------------|-----------------------------------|
| Lomax model, $n < 100$         | MSE, low bias/variance                | MAD outperforms MLE, MoM          |
| ARFIMA time series           | Bias under unknown/nontrivial mean     | BCMDE variants reduce bias/MSE    |
| Finite mixtures              | Optimal $W_1$-rate, TV convergence     | MAD-type $\to$ uniform minimax    |
| Multivariate robust location | Breakdown, computational efficiency    | High breakdown, scalable          |

## 5. Variants and Extensions

Several extensions and specialization axes exist:

- **Minimum $L^q$-distance estimators:** For $q=1$, the estimator is the minimum absolute distance (MAD), but for $q>1$ or $q=\infty$, analogous estimators minimizing $L^q$ or supremal distances provide trade-offs between sensitivity and robustness; the $L^2$ version is closely related to Cramér–von Mises estimators [1909.00002, 2207.06086].
- **Mixture estimation via $\Phi$-distance:** MAD arises naturally as the special case where $\Phi$ is the unit ball in $L^\infty$, resulting in total-variation loss for mixing measures; kernel-based generalizations (MMD, etc.) further broaden the method’s applicability, especially in high dimensions or non-Euclidean sample spaces [2304.10052].
- **Trimmed and nontrimmed forms:** Trimming parameter $h$ in LTAD allows interpolation between the full-data MAD (median case) and highly robust estimators for contaminated data [1511.04220].

## 6. Limitations and Practical Considerations

Although minimum absolute distance estimators enjoy attractive robustness and minimax properties, several practical aspects warrant consideration.

- **Computation:** For high-dimensional or large-sample settings, exact minimization may be computationally intensive; the use of subgradient methods, data centering, and surrogates (RKHS/MMD) is essential for tractability [1511.04220, 2304.10052].
- **Bias-variance tradeoff:** MAD estimators may display small negative bias in extremely small samples, but compensate by significantly lower variance relative to classical estimators, especially under model misspecification or contamination [2207.06086].
- **Choice of metric and loss:** The $L^1$ criterion is less sensitive to outliers but may be less efficient than $L^2$ or KL divergences for well-specified, uncontaminated models in large samples. For optimal performance, hybrid approaches that adapt by sample size are recommended [2207.06086].
- **Model identifiability:** Strong identifiability conditions are required for mixture models to guarantee minimax convergence; in settings with low separation or high component overlap, performance may degrade and require regularization or penalized criteria [2304.10052].

## 7. Connections to Classical Estimators and Recommendations

The minimum absolute distance estimator is a cornerstone of robust statistics and minimum distance estimation.

- In the univariate setting, the MAD is the sample median, known for its 50% breakdown point and optimal robustness [1511.04220].
- In the context of finite mixture estimation and distributional functional estimation, MAD methods achieve optimal nonparametric risk, especially outperforming MLEs when the sample size is comparable to domain size [1705.00807, 2304.10052].
- For practical implementation, MAD (or its close relatives such as minimum squared deviation estimators) are recommended for small to moderate sample sizes or for contaminated data, with bias-adjusted MLEs becoming preferred as sample size increases [2207.06086].

A plausible implication is that the minimum absolute distance estimator provides an essential methodological and theoretical link between robust classical statistics, modern empirical process theory, and computational statistics, with continuing importance in high-dimensional inference, mixture learning, and time series analysis.

Source: https://www.emergentmind.com/topics/minimum-absolute-distance-estimator