Evaluate additional online optimizers

Determine whether memory-light, learning-rate-free, and non-stationary-online optimizers such as AdaFactor, Prodigy, and ObGD improve or otherwise alter the accuracy--adaptation-state-memory frontier established for SGD with momentum and Adam under the paper's leakage-free validation-only protocol.

Background

The experimental optimizer comparison is deliberately restricted to SGD with momentum and Adam. The paper therefore does not establish whether the reported resource and accuracy trade-offs generalize to optimizers designed to reduce memory, avoid learning-rate tuning, or handle non-stationary online learning.

The authors specifically name AdaFactor, Prodigy, and ObGD as candidates requiring separate evaluation under the same protocol, making the optimizer frontier an explicitly unresolved question.

References

Whether memory-light, learning-rate-free, and non-stationary-online optimizers (e.g., AdaFactor, Prodigy, and ObGD) move the frontier points of \S\ref{sec:frontier} requires a separate study under the same protocol.

— When Does Online Adaptation Pay on the Edge? A Leakage-Free Evaluation of Warmup, Learning-Rate Selection, and Resource Trade-offs for Time-Series Forecasting  (2609.01126 - Fujimoto et al., 1 Sep 2026) in Discussion and Limitations, Section 5