Determine the optimal stopping-parameter value for the gradient-descent estimator

Determine the optimal value of the stopping constant controlling the cumulative gradient-descent step sizes for the proposed randomly pruned deep neural-network regression estimator in a given application, so that it can be selected without treating the constant as an externally tuned smoothing parameter.

Background

The proposed estimator uses a cumulative step-size threshold, denoted by the constant $\const[lambda]$, to determine when gradient descent stops. The simulation results show that this constant affects the estimator’s smoothing behavior: values that are too small or too large can increase the L2L_2 error, while moderate values perform better.

The paper explicitly states that the optimal value of this constant is unknown in applications. The authors therefore treat it, together with the input-weight scale parameter AA, as a smoothing parameter and select both by splitting the sample. The unresolved issue is thus how to determine the optimal stopping constant, rather than merely tune it empirically for a particular dataset.

References

Since we do not know what the optimal value for this constant is in an application, we consider it together with the value of $A$ as smoothing parameters (for $B$ we will use a moderately large fixed value in the sequel), and will choose both values by splitting of the sample.

— Learning of deep neural network regression estimates using gradient descent with pruning  (2608.25743 - Kohler et al., 26 Aug 2026) in Section 4, immediately after Table 1