LDP Semi-Parametric Estimation

• The paper's main contribution is a framework for optimizing minimax risk in semi-parametric models under local differential privacy.
• It combines non-interactive and sequentially interactive protocols to achieve near-optimal estimation rates by leveraging adaptive noise calibration and synthetic data strategies.
• Adaptive plug-in estimators and modern privacy mechanisms enable versatile, multi-purpose analysis while effectively managing high-dimensional and nonparametric challenges.

Locally differentially private semi-parametric estimation concerns statistical functionals of a data-generating distribution that must be estimated based on data sanitized by local differential privacy (LDP) mechanisms—channels that perturb each individual's data prior to any central aggregation, providing privacy guarantees at data acquisition. This domain encompasses density functionals such as integrated squared density, entropy, more general integral/statistical functionals, high-dimensional parameter estimation, and general regression settings, all under stringent privacy. The landscape of research in this area combines minimax theory, privacy-aware algorithm design, technical analysis of optimal rates and adaptivity, and rigorous minimax risk characterization.

1. Formal Definitions and Minimax Theory

LDP requires that each data owner locally applies a randomized channel $Q$ to their record $x$, such that for all $x, x'$ and measurable sets $A$,

$Q(A|x) \leq e^\alpha Q(A|x')$

for privacy parameter $\alpha$. Given $n$ data owners and a functional parameter $\Lambda(f)$ of unknown density $f$, the minimax risk under LDP is typically defined as: $$\mathcal{M}_n(Q, \mathcal{F}) = \inf_{\hat\Lambda_n} \sup_{f\in\mathcal{F}} \mathbb{E}_{f,Q}[(\hat\Lambda_n - \Lambda(f))^2]$$ where $\mathcal{F}$ is a function class (e.g., Besov, Sobolev, Hölder), and the infimum is taken over all estimators based on privatized data $Z_1,\ldots,Z_n$. The inclusion of $Q$ in the formalism highlights that the privacy mechanism is not fixed a priori, and one must optimize over all mechanisms satisfying LDP.

A key feature is the "elbow" phenomenon: there exists a threshold in smoothness $s^*$ where the convergence rate of the minimax risk transitions from nonparametric to parametric. For different mechanisms and functionals, the elbow's location and the risk rate can differ drastically (see Table 1).

Estimation Setting	Minimax Risk Rate (up to log factors)	Elbow Smoothness $s^*$
Direct (no privacy)	$n^{-8s/(4s+1)}$	$s=\tfrac14$
Laplace-noise additive	$(n\alpha^2)^{-8s/(4s+9)}$	$s=\tfrac94$
LDP, non-interactive (quadratic functionals)	$(n\alpha^2)^{-8s/(4s+3)}$	$s=\tfrac34$
LDP, sequentially interactive (quadratic functionals)	$(n\alpha^2)^{-4s/(2s+1)}$	$s=\tfrac12$
Gaussian mean estimation (unit variance, LDP)	$(n\alpha^2)^{-1}$ (for small $\alpha$)	N/A

2. Architectures: Non-Interactive vs. Interactive Procedures

Non-interactive protocols compel each user to perturb their data independently with no subsequent rounds, often leading to inflated risk, especially for nonparametric functionals.

For integrated quadratic density functionals $D(f) = \int f^2$, the canonical non-interactive approach is to have each user release noisy empirical wavelet coefficients: $$Z_i = \psi_{jk}(X_i) + \frac{\sigma_j \, \sigma}{\alpha} W_{ijk}$$ where the $\psi_{jk}$ are, for instance, Haar wavelets, $W_{ijk}$ are i.i.d. Laplace noises, and $\sigma_j$ are regularization-dependent scalings. The estimator of $D(f)$ is then constructed as a U-statistic over all cross products of these privatized coefficients.

Sequentially interactive methods divide the sample (often into two halves): the first executes a non-interactive wavelet-based estimate of $f$. The second then conducts private estimation of a linear functional $L(f) = \int \hat{f}(x)f(x)dx$, incorporating prior knowledge from the first phase. Explicitly, in the second phase, sanitization depends on a function of the first-phase output, for example employing mechanisms for privatized inner products, leading to strictly improved minimax risk and a shifted elbow.

This strict improvement—impossible in many other nonparametric LDP settings—demonstrates that interaction (even limited and sequential) can drive statistical efficiency closer to the unprivatized regime (Butucea et al., 2020).

3. Adaptive and Plug-In Estimation

Adaptivity in LDP estimation is realized through penalized selector strategies (e.g., Goldenshluger–Lepski or penalized maximization) that choose the “resolution parameter” (wavelet scale $J$, kernel bandwidth $h$, or projection dimension $d$) data-adaptively, ensuring (up to log factors) minimax optimality even when smoothness $s$ is unknown.

More generally, multi-purpose plug-in estimation frameworks have emerged for broad classes of differentiable functionals. By projecting the density onto a spline or spline wavelet basis and releasing noisy coefficients (with level-dependent Laplace noise), these methods provide synthetic data that can be re-used by different analysts for arbitrary functionals, without further privacy budget consumption: $$\hat{f}_n(x) = \sum_{k=1}^{2^{j_n}+d} \bar{Z}^{(j_n)}_k B_{k,d,\xi^{(j_n)}}(x)$$ where the $Z_{i}^{(j)}$ are locally privatized vectors, and $B_{k,d,\xi^{(j_n)}}$ are spline basis functions. For smooth functionals $\Lambda(f)$, plug-in estimates $\Lambda(\hat{f}_n)$ achieve either parametric $n^{-1/2}$ or nonparametric minimax rates, depending on the functional's smoothness index (Randrianarisoa et al., 19 Aug 2025).

Crucially, the privacy mechanism and noise calibration are independent of the target $\Lambda$, enabling a single sanitized dataset to support many analysts and inference tasks, an advancement over prior function-specific LDP mechanisms.

4. Information-Theoretic and Statistical Foundations

LDP mechanisms induce a pronounced contraction of statistical divergence between distributions. For an $\epsilon$-LDP mechanism $Q$, contraction coefficients satisfy

$$\eta_f(Q) \leq \left(\frac{e^\epsilon-1}{e^\epsilon+1}\right)^2$$

for a large family of $f$-divergences (Asoodeh et al., 2022). This contraction directly limits the distinguishability of distributions (e.g., in $\chi^2$, KL-divergence, squared Hellinger distance), fundamentally increasing minimax risk.

Classical statistical lower bounds are accordingly modified:

• van Trees (Bayesian Cramér–Rao) and Fisher information-based bounds are scaled down by the maximal contraction coefficient.
• Testing-based bounds (Le Cam, Assouad) are upgraded to their "private" analogs, yielding sharper risk lower bounds in both parametric and nonparametric settings.

An important implication is that the effective sample size is reduced by the contraction, often behaving as $\Psi_\epsilon n$ with $\Psi_\epsilon \sim (e^\epsilon-1)^2$ or similar, so that the convergence rates seen in non-private settings degrade accordingly.

5. Specialized Scenarios and Functional Classes

Nonparametric functionals (e.g., entropy, goodness-of-fit, integral functionals): Many semi-parametric tasks (goodness-of-fit, estimation of entropy, or functionals involving derivatives or integrals of $f$) can be reduced to estimation of quadratic or generally integral functionals. The minimax rate for entropy estimation under LDP, for example, becomes

$$R^*(n, \Theta, \ell_2^2, Q)\gtrsim \min\left\{1, \frac{1}{n}\left(\frac{e^\epsilon+1}{e^\epsilon-1}\right)^2\right\}\log^2 k$$

in the discrete case (Asoodeh et al., 2022).

Regression and high-dimensional estimation: In sparse linear regression in high dimensions, user-level LDP with multiple samples per user circumvents the linear dependence on ambient dimension $d$ previously deemed unavoidable, reducing the minimax error rate to $O(s^{*2}/(nm\varepsilon^2))$ for $n$ users, each with $m$ local samples, and $s^*$-sparsity, instead of $O(ds^*/(nm\varepsilon^2))$ (Ma et al., 8 Aug 2024).

Bayesian LDP semi-parametric inference: By "baking in" the LDP mechanism into the generative model, Bayesian noise-aware inference integrates the privacy noise directly into the likelihood so that the posterior is automatically calibrated for uncertainty arising from both the data-generating process and LDP mechanisms. In models with sufficient statistics, privatized statistics are modeled as a noisy Gaussian, while in general models conjugate or approximate-conjugate priors are employed for tractability (Kulkarni et al., 2021).

6. Practical Implications and Adaptivity

These advances have several major consequences:

• Multi-purpose synthetic data releases (via noisy spline or wavelet coefficients) support repeated, functional-agnostic analysis, removing the need for functional-specific privacy mechanisms (Randrianarisoa et al., 19 Aug 2025).
• Adaptive estimation schemes, such as data-driven bandwidth selection (e.g., Goldenshluger–Lepski), allow estimators to achieve near-optimal rates across unknown analytic regularity (Schluttenhofer et al., 2022).
• User-configurable privacy for features: Semi-feature LDP models enable protection only of sensitive attributes, improving statistical efficiency and reducing privacy-induced risk for non-sensitive features (Ma et al., 22 May 2024).
• Improved accuracy via interaction or side-information: Sequentially interactive protocols, public features, or multiple samples per user can sharply improve estimation rates over conventional non-interactive or item-level LDP, moving the feasible region closer to the unprivatized setting (Butucea et al., 2020, Ma et al., 8 Aug 2024, Ma et al., 22 May 2024).

7. Future Directions

Several outstanding challenges motivate ongoing research:

• Extending plug-in synthetic data approaches to general regression and high-dimensional models, including heteroskedastic or structured settings.
• Refining adaptation techniques to further reduce the logarithmic penalty in the risk rates introduced by tuning and noise calibration.
• Characterizing tight non-asymptotic lower/upper bounds for broad classes of (possibly non-differentiable) functionals under LDP.
• Development of interactive or user-level LDP protocols in federated or distributed architectures with heterogeneous feature privacy requirements.
• Algorithmic refinements for high communication efficiency, real-time inference, and large-scale practical deployment.

In sum, locally differentially private semi-parametric estimation synthesizes modern privacy theory, minimax statistical analysis, and practical mechanism design, culminating in a roadmap for privacy-preserving, theoretically sound, and flexible statistical inference for infinite–dimensional and high-dimensional models. Key advances including functional-agnostic synthetic data generation, adaptive privacy-aware estimation, interaction, and tailored mechanism design have collaboratively advanced the attainable accuracy in various nonparametric and semi-parametric inferential domains under LDP, setting a standard for future private-data analytic frameworks.