Computationally tractable robust differentially private mean estimation

Abstract: We develop a new, differentially private mean estimator called the balloon mean. The main features of the balloon mean are that it is computationally tractable and enjoys robustness to outlying observations. It is based on an iterative clipping procedure over expanding Mahalanobis balls, or ``balloons.'' The method satisfies zero-concentrated differential privacy and depends on a small number of interpretable tuning parameters. We provide theoretical guarantees under heavy-tailed and contaminated elliptical models, characterizing its statistical performance and robustness to outliers. Extensive simulations demonstrate that the balloon mean is robust to heavy-tailed and contaminated data, and outperforms existing differentially private mean estimators in contaminated settings.

• The paper introduces the balloon mean, an iterative algorithm achieving robust and DP mean estimation for high-dimensional, contaminated data.
• It uses a noisy mean and adaptive Mahalanobis-ball projection to balance privacy and robustness while avoiding complex optimization or MCMC.
• Theoretical guarantees ensure minimax-optimal rates in heavy-tailed settings and near-optimal performance under adversarial contamination.

Computationally Tractable Robust Differentially Private Mean Estimation: The Balloon Mean

Introduction and Motivation

The intersection of robust statistics and differential privacy (DP) presents fundamental and algorithmic challenges in modern statistics and machine learning. High-dimensional mean estimation frequently encounters adversarial contamination and heavy-tailed data, while robust privacy guarantees such as zero-concentrated differential privacy (zCDP) are essential for applications involving sensitive data. Traditional robust DP estimators either rely on heavy computational resources (e.g., sum-of-squares, MCMC) or lack strong robustness or privacy guarantees. Addressing this, "Computationally tractable robust differentially private mean estimation" (2606.12654) proposes the balloon mean: a computationally feasible, robust, and strongly private iterative estimator for the mean of high-dimensional, potentially contaminated data.

The Balloon Mean Algorithm

The balloon mean is an iterative estimator that alternates between computing a noisy mean of Mahalanobis-ball-projected data and privately updating the ball (balloon) radius to contain a specified fraction of the data. The algorithm can be summarized in alternating steps:

1. Initialization: Start with an initial center $\tilde{\mu}_0$, radius $\tilde{R}_0$, and known (or differentially private) covariance $\Sigma$.
2. Iterative Process (for $m=1$ to $M$):

• Mean Update: Project data onto the current Mahalanobis ball and compute a Gaussian-noised mean of the projections, yielding $\tilde{\mu}_m$.
• Balloon Update: Using a private AboveThreshold mechanism, expand the ball centered at $\tilde{\mu}_m$ until a fraction $\tau_m$ of the data is enclosed, yielding a new radius $\tilde{R}_m$.

This routine efficiently achieves both robustness (via data exclusion at the tails) and zCDP. Notably, the method avoids convex optimization or MCMC and requires only basic linear-algebraic operations, yielding strong scalability in high dimensions.

Figure 1: Stepwise illustration of the balloon mean procedure, showing the cyclical updates of center and radius with successive noisy mean estimates.

Theoretical Guarantees

The authors present explicit finite-sample high-probability Mahalanobis error bounds for the balloon mean under general elliptical models, heavy tails, and $\eta$-proportion adversarial contamination. The main theoretical result is:

• For $\tilde{R}_0$0, the Mahalanobis error satisfies

$\tilde{R}_0$1

with probability at least $\tilde{R}_0$2 for a suitable privacy level $\tilde{R}_0$3, where $\tilde{R}_0$4 is the contamination fraction.

Crucially:

• Heavy-tailed setting ($\tilde{R}_0$5): The method achieves minimax-optimal rates (up to logarithmic factors), matching lower bounds for differentially private mean estimation.
• Adversarial contamination ($\tilde{R}_0$6): The error grows at rate $\tilde{R}_0$7, which is suboptimal relative to the non-private optimal $\tilde{R}_0$8 but is structural due to limitations of computational efficiency and strong privacy; no existing computationally efficient method is known to improve on this tradeoff for high-dimensional adversarial robustness.

Parameter dependence enters only logarithmically, making the estimator robust to tuning and practical for real-world datasets.

Algorithmic Insights and Interpretability

• Parameterization: The scheme depends on few, interpretable parameters: the fraction of data $\tilde{R}_0$9 to include (robustness), the grid parameter $\Sigma$0 (ball expansion granularity), the number of iterations $\Sigma$1, and the privacy allocations $\Sigma$2.
• Adaptation: The iterative structure enables recentering and updating based directly on the private data structure, improving over fixed-radius approaches (e.g., COINPRESS [Biswas et al., 2020]) and single-pass clipping (e.g., Huang et al., 2021).
• Robustness Lever: The parameter $\Sigma$3 directly controls the breakdown point — choosing smaller $\Sigma$4 excludes a higher fraction of tails/outliers, increasing robustness at a modest efficiency cost.
• Computation: Whitening and matrix decomposition are only performed once, and all subsequent computations are at most $\Sigma$5, ensuring tractability even for large $\Sigma$6.

Empirical Performance

Extensive simulations evaluate the balloon mean against state-of-the-art zCDP estimators (COINPRESS, private Huber $\Sigma$7-estimation, instance-optimal mean) across various distributions (Gaussian, contaminated Gaussian, heavy-tailed $\Sigma$8, banana-shaped), dimensions ($\Sigma$9 up to 128), and privacy levels ($m=1$0).

Key findings:

• Robustness: The low-$m=1$1 variant (stronger clipping) consistently outperforms all baselines under contamination and heavy tails, particularly in high dimensions and/or strong privacy regimes.
• Clean Data: Balloon mean performance matches (does not underperform) the best existing methods when the data is uncontaminated.
• Parameter Sensitivity: Performance is largely invariant to the choice of $m=1$2, $m=1$3, grid $m=1$4, or number of iterations $m=1$5 (after $m=1$6–$m=1$7).

  Figure 2: Mahalanobis error as a function of sample size $m=1$8 and dimension $m=1$9 for various contamination regimes and DP mean estimators.

  

  Figure 3: Breakdown performance under increasing contamination or heavy tails demonstrates greater robustness of the balloon mean with aggressive ($M$0) schedules.

  

  Figure 4: Insensitivity of Mahalanobis error to initial mean mis-specification, across distributions and dimensions.

  

  Figure 5: Empirical error landscape as a function of initial ball radius exhibits nearly flat performance across a wide range of $M$1.

Practical and Theoretical Implications

The balloon mean provides an effective tool for private mean estimation in scenarios where both data contamination and strong privacy are genuine concerns, e.g., health, finance, or genomics. Its computational simplicity and weak parameter sensitivity facilitate deployment by non-experts and in large-scale systems.

Theoretically, this work clarifies the attainable tradeoffs between robustness, privacy, and tractability. Notably:

• Logarithmic parameter sensitivity removes the need for fine hyperparameter search.
• The trade-off of error scaling as $M$2 under adversarial contamination persists as a structural barrier under zCDP and efficient algorithms.
• The iterative, privately adaptive clipping scheme is a conceptual advance over methods relying on fixed, non-adaptive radii.

Future Directions

Key open directions include:

• Optimal Adversarial Robustness: Bridging the gap to the $M$3 contamination rate in a tractable and strongly private estimator remains unresolved.
• Extension to Unknown Covariance: While the method presumes knowledge of $M$4, robust and private covariance estimation (e.g., [Kim & Jung, 2025]) can be composed for broader applicability.
• Generalization Beyond the Mean: The iterative private localization paradigm may be adaptable to other high-dimensional, robustly private estimation problems (medians, regression, covariance).

Conclusion

The balloon mean presents a computationally simple, robust, and strongly private approach to multivariate mean estimation under heavy-tailed or contaminated data. It achieves minimax-optimal rates in the heavy-tailed case and nearly optimal performance under adversarial contamination within the constraints of algorithmic efficiency and strong privacy. Through both theoretical guarantees and extensive empirical validation, the method demonstrates stability to parameter choice and superiority over existing approaches, supporting responsible use and analysis of sensitive, high-dimensional data in practice.