Improved approximation rates for deep norm-constrained networks

Determine whether deep path-norm-constrained ReLU neural networks can achieve approximation rates better than the rate established in the W^{1,p}-norm, namely a rate of order K^{-(n-1)/(d+d/p+1)} under the paper’s width and depth conditions.

Background

For deep path-norm-constrained networks, the paper proves a W^{1,p}-approximation error bound of order K^{-(n-1)/(d+d/p+1)} when the width grows sufficiently with the path-norm budget and the depth exceeds a logarithmic threshold.

The authors do not determine whether this rate is optimal. They explicitly leave unresolved whether deep norm-constrained networks can provide a strictly better approximation rate.