Determine whether differentiability improves the nonlinear TTSA MSE rate

Determine whether adding the twice-differentiability assumptions on the averaged dynamics to the hypotheses of the nonlinear constant-step-size TTSA MSE theorem improves its MSE bounds.

Background

The paper proves MSE bounds of order O(α+β²/α²) for nonlinear two-time-scale stochastic approximation with constant step-sizes under Markovian noise. In a numerical example whose averaged dynamics are differentiable, the slow-iterate MSE appears empirically closer to O(β) than the general theorem’s O(α) bound when β=α^{3/2}. The authors therefore identify strengthening the MSE theorem with the differentiability assumptions used in the bias analysis as an unresolved direction.