Adaptive adjustments for non-interpolating stochastic line search

Ascertain whether adaptive step-size adjustments based on criterion-violation information are useful or applicable to stochastic line-search methods that resample function and gradient mini-batches in every loop cycle and do not rely on the interpolation property.

Background

The adaptive backtracking approach exploits violation information of line-search criteria to adjust step sizes and is naturally aligned with settings where the same sample or batch is used within the line search (e.g., interpolation-based stochastic line search).

For more general stochastic line search methods that repeatedly resample mini-batches and do not rely on the interpolation property, the relevance of violation information from one sample to another is questionable. The authors explicitly state it is unclear whether adaptive adjustments can be useful in this setting, framing a concrete open question.