Asymptotic travel threshold and learning outcome

Determine whether the learned traveling bands possess a fixed threshold in the cooperator and defector learning rates and establish the asymptotic outcome of continued multi-agent Q-learning in the spatial public-goods model.

Background

The paper studies fixed populations of cooperators and defectors moving on a ring while independently learning stay, left, or right actions through tabular Q-learning. After finite training, certain learning-rate pairs produce coherent traveling bands supported by a shared directional preference. However, continued training can cause travel to appear, disappear, or reverse direction, and the observed boundaries shift with the training budget.

Consequently, the reported finite-training simulations do not establish either a rate threshold that persists asymptotically or convergence to a particular long-run behavioral or spatial state. Resolving this problem would require characterizing the dynamics of the coupled public-good environment and independently adapting Q-learners under substantially longer training or through an analytical convergence framework.