Extend the positive utility-ordering regime to stronger privacy

Determine whether increasing model size or privacy budget can extend the positive source-ordering regime for data-free preconditioning to privacy levels with $\varepsilon \le 10$.

Background

The noise-cross-over experiments locate the regime in which matched probes outperform flat probes only under weak privacy and in a small-model training protocol. At stronger noise, learning itself collapses, so the preconditioning-induced source ordering is no longer observable.

The authors identify an unresolved scaling question: whether larger models or a different privacy budget can preserve the positive ordering at a substantially stronger privacy level, specifically at ε≤10\varepsilon \le 10.

References

The noise cross-over (Section~\ref{sec:negative}) is located at weak privacy in this small-model protocol, where learning survives only for $\varepsilon$ well above typical reporting thresholds; whether a larger model or budget pushes the positive-ordering regime into $\varepsilon \le 10$ is open.

— Spectrum Is Not Enough: A Random-Matrix Account of Data-Free Preconditioning for Differentially Private Learning  (2610.21312 - Zhang et al., 15 Oct 2026) in Section 10, subsection “Limitations”