Extension of gradient-descent learning lower bounds

Establish whether the gradient-descent learning lower bounds of Abbe, Bengio, Cornacchia, Kleinberg, Lotfi, Raghu, and Zhang extend to hierarchical functions under the general product-measure and non-separability framework considered here.

Background

The paper proves exponential noise-sensitivity and corresponding low-degree correlation bounds for hierarchical multilinear and, more generally, hierarchical non-separable functions under general product measures. Earlier work by Abbe, Bengio, Cornacchia, Kleinberg, Lotfi, Raghu, and Zhang established that noise sensitivity yields lower bounds on the number of iterations required by a specified version of gradient descent to learn functions in a fully connected deep-network setting with particular initialization assumptions.

The authors explicitly conjecture that these learning lower bounds should extend to the broader hierarchical-function and product-measure setting developed in the paper, but do not prove such an extension. The unresolved task is therefore to formulate the appropriate neural-network and initialization assumptions and establish the corresponding gradient-descent lower bounds.