EoSS as a mechanism for optimizer‑induced functional divergence

Establish whether the functional divergence observed between predictors trained for S&P 500 volatility forecasting with different optimizers is caused by the constraint imposed by the Edge of (Stochastic) Stability (EoSS) on optimization trajectories, and characterize how optimizer‑specific interactions with the EoSS boundary drive the selection of distinct learned functions despite identical out‑of‑sample loss.

Background

The paper demonstrates that in one‑step‑ahead realized volatility forecasting for S&P 500 stocks, multiple architecture–optimizer pairs achieve statistically indistinguishable test NMSE yet learn qualitatively different input–output mappings. Functional diagnostics (impulse responses, optimizer‑difference surfaces, SHAP attributions) reveal that optimizers select different temporal dependencies and nonlinearities even when predictive accuracy ties.

To probe a mechanism, the authors analyze curvature and training dynamics and document Edge of (Stochastic) Stability (EoSS) behavior. They estimate that the stability horizon is crossed within the training budget and observe that SGD converges to flatter, simpler solutions while adaptive methods reach sharper, more nonlinear minima. Based on these observations, they explicitly conjecture that EoSS‑induced constraints on optimization trajectories may underlie the optimizer‑dependent functional divergence.