Implications of PAC-learning separations for private distribution learning

Determine whether the known separations between non-private PAC learning and differentially private PAC learning of functions—under both (ε,0)-differential privacy and (ε,δ)-differential privacy, where approximate-DP learnability is characterized by the Littlestone dimension rather than the VC dimension—have any implications for the sample complexity or structural characterizations of privately learning distributions in total variation distance.

Background

The paper surveys strong separations established for PAC learning of functions under differential privacy. In particular, for approximate differential privacy, learnability is characterized by the Littlestone dimension rather than the VC dimension, highlighting a fundamental divergence from the non-private setting.

Because distribution learning (density estimation in total variation distance) differs substantially from PAC function learning, it is not immediate whether these PAC-learning separations transfer to, or inform, analogous results for private distribution learning. The authors explicitly note this uncertainty. Their main contribution is an explicit class of distributions that is learnable non-privately with a constant number of samples but not learnable under (ε,δ)-differential privacy with any finite number of samples; however, this result does not directly resolve whether previously known PAC-learning separations imply anything systematic for private distribution learning.