Finite-width uniform mixed-derivative bounds for random smooth networks

Establish a finite-width result for randomly initialized smooth neural networks that provides, on a single high-probability event, simultaneous and uniform bounds over the input domain for every non-empty square-free mixed input derivative, with explicit dependence on the derivative order, depth, width, input dimension, and failure probability.

Background

The paper places its contribution within the literature on derivatives of randomly initialized neural networks and finite-width Gaussian approximations. Existing results discussed by the authors address related quantities, including Jacobian stability, Gaussian-process limits, cumulant expansions, or derivative approximation at fixed depth and derivative order. The unresolved gap identified by the authors is a nonasymptotic finite-width theorem giving direct samplewise control of all square-free mixed input derivatives simultaneously on one high-probability event and uniformly over a continuum of input points, while also tracking the relevant order, depth, width, dimension, and failure-probability dependencies explicitly. The paper claims to provide precisely such an estimate, so this passage describes the prior unresolved result rather than an open problem left unanswered by the present paper.