Explaining the looseness of existing DNN generalization bounds

Identify and characterize the factors that make existing generalization bounds for deep neural networks loose in practice, and determine why these bounds are suboptimal relative to empirical performance.

Background

The paper surveys several families of generalization bounds for deep networks (e.g., path norms, product of layer norms, covering-number-based bounds) and notes that, empirically, these bounds are often very loose. Understanding the concrete reasons for this looseness could guide the development of tighter theoretical guarantees and connect to the proposed circuit-size perspective.