Tuning‑free optimization in the finite‑sum setting with exact function evaluations

Investigate whether tuning‑free counterparts to optimally‑tuned stochastic gradient descent exist in the finite‑sum optimization setting when the algorithm can periodically compute exact function values; in particular, determine whether such periodic exact evaluations enable tuning‑free optimization that matches optimally‑tuned SGD up to polylogarithmic factors.

Background

After presenting several impossibility results for tuning‑free optimization under general stochastic oracles, the authors suggest that additional structure could change the landscape. In particular, the finite‑sum setting allows occasional exact evaluation of the objective, which might circumvent the barriers identified in the stochastic setting.

Clarifying whether such exact evaluations enable tuning‑free optimization would identify tractable regimes and inform algorithm design for practical finite‑sum problems.