Comprehensive theory for UDRL, GCSL, and ODT across finite and asymptotic iterations

Establish a comprehensive, rigorous theoretical treatment of the behavior of Upside-Down Reinforcement Learning (UDRL), Goal-Conditioned Supervised Learning (GCSL), and Online Decision Transformers (ODT) for both a finite number of training iterations and in the asymptotic limit, including convergence and stability characterizations under general Markov decision process settings.

Background

The paper analyzes convergence and stability of algorithms that approach reinforcement learning via supervised learning or sequence modeling, focusing on UDRL, GCSL, and ODT. While prior work has examined the first iteration (especially in offline settings), a general theory covering multiple iterations and asymptotic behavior is lacking.

The authors explicitly note that existing results do not provide a complete understanding of these methods beyond the initial iteration, motivating the need for a comprehensive theoretical framework that addresses behavior across iterations and in the limit.