Determine whether public data can replace the private spectral exponent

Determine whether a public proxy for the private activation-spectrum exponent $\beta_o$ suffices to select the probe exponent for data-free KFAC preconditioning.

Background

The exponent-matching rule selects a probe exponent by matching the probe activation-spectrum exponent to the private model's exponent βo\beta_o. Although this rule is presented as tuning-free, it still consumes one private statistic, so it is not fully data-free.

A practically deployable data-free method would need to infer or approximate βo\beta_o from public information. The paper explicitly identifies the sufficiency of such a public proxy as unresolved.

References

First, the rule consumes a private statistic: $\beta_o$ is the exponent of the private activation spectrum, the data covariance spectrum at the input layer and the trained activation spectrum at hidden layers, so the recipe is one-parameter and tuning-free but not fully data-free, and whether a public proxy for $\beta_o$ suffices is open.

— Spectrum Is Not Enough: A Random-Matrix Account of Data-Free Preconditioning for Differentially Private Learning  (2610.21312 - Zhang et al., 15 Oct 2026) in Section 5, subsection “What the probe family actually matches”

On the probe side, the exponent-matching rule consumes one private statistic ($\beta_o$) and lands within $0.15$ on $21$ of $43$ configurations, so a public proxy for $\beta_o$ and an exact rule are open.

— Spectrum Is Not Enough: A Random-Matrix Account of Data-Free Preconditioning for Differentially Private Learning  (2610.21312 - Zhang et al., 15 Oct 2026) in Section 10, subsection “Limitations”