Tractable nonlinear neuron oracle for infinite-dimensional greedy learning

Develop a tractable algorithm for solving the continuous nonlinear neuron-selection oracle used by the fully-corrective greedy method, with computational cost that remains manageable as the retained input resolution increases.

Background

The theoretical guarantees are uniform in the number of retained input coordinates because the normalized neural dictionary has resolution-independent statistical complexity. However, each greedy step requires optimizing a nonlinear objective over the neuron parameters, and the dimension of this parameter-search problem grows with the retained resolution.

The paper’s experiments avoid this issue by replacing the continuous oracle with a finite candidate dictionary. Consequently, establishing an efficient or provably approximate solver for the continuous neuron-selection problem remains unresolved and is necessary for computationally meaningful dimension-free claims.