2000 character limit reached
On change of measure inequalities for $f$-divergences (2202.05568v1)
Published 11 Feb 2022 in stat.ML, cs.IT, cs.LG, math.IT, math.PR, math.ST, and stat.TH
Abstract: We propose new change of measure inequalities based on $f$-divergences (of which the Kullback-Leibler divergence is a particular case). Our strategy relies on combining the Legendre transform of $f$-divergences and the Young-Fenchel inequality. By exploiting these new change of measure inequalities, we derive new PAC-Bayesian generalisation bounds with a complexity involving $f$-divergences, and holding in mostly unchartered settings (such as heavy-tailed losses). We instantiate our results for the most popular $f$-divergences.