Verifying parameter conditions for two-timescale convergence

Develop practical, checkable criteria to determine whether the key assumption on the agent distribution (inequality (85) in the convergence analysis) holds for a given parameter set, thereby enabling verification of convergence for the two-timescale ODE approximation of the lin-lin coupling dynamics prior to execution.

Background

The paper presents a convergence analysis of an approximated lin-lin coupling model using a two-timescale ODE framework, where the endogenous cue dynamics (fast) and agent distribution dynamics (slow) interact through operators defined on the graph. The analysis proves convergence under assumptions including bounds on the agent distribution and parameter constraints.

However, the authors explicitly note a limitation: the central assumption about the distribution cannot be verified from parameters before running the dynamics. Making this assumption checkable would strengthen the applicability of the convergence theory and facilitate parameter selection and model validation in practice.

References

A limitation of our convergence result is that we cannot check if the assumption~eq:conv-assump-policy holds for given parameters before execution.

— Optimality theory of stigmergic collective information processing by chemotactic cells  (2407.15298 - Kato et al., 2024) in Appendix, Section “Convergence analysis result and discussion”

Condition (iv) remains an open theoretical question, though empirically the TD error converges to zero within approximately $150$ episodes (Fig.~\ref{fig_td_convergence}) and the learned parameters stabilize, suggesting well-behaved limiting dynamics for the present case study.

— Two-Timescale Reinforcement Learning for Real-Time Optimization and Economic NMPC: Experimental Validation  (2609.35141 - Adhau et al., 28 Sep 2026) in Remark 2? [Remark labeled “Convergence of the two-timescale scheme” in Section 3.2, subsection “Q-Learning for ENMPC”]