Leveraging Donsker–Varadhan for General Rate–Distortion Lower Bounds

Investigate and characterize to what extent the Donsker–Varadhan variational representation for Kullback–Leibler divergence can be leveraged to derive general rate–distortion lower bounds for the information-theoretic learning framework presented in the paper. Develop conditions and analytic techniques that yield broadly applicable lower bounds beyond the linear regression case.

Background

The paper’s framework connects estimation error to mutual information and rate–distortion theory, providing general upper bounds across diverse learning settings. Lower bounds, however, are largely developed only for linear regression.

In the conclusion, the authors emphasize the need for general rate–distortion lower bounds and point to the Donsker–Varadhan variational approach as a promising tool, explicitly leaving this direction open as future work.