Applying Transformers to Reinforcement Learning

Develop robust techniques and training procedures that enable the transformer architecture to be effectively and stably applied to reinforcement learning tasks and environments.

Background

The work introduces TrMRL, a transformer-based meta-reinforcement learning agent, and reviews prior attempts to use transformers in reinforcement learning. These attempts often encountered instability or required architectural modifications or observation constraints, underscoring unresolved challenges in this area.

Motivated by this context, the authors explicitly state that applying transformers in reinforcement learning remains an open challenge, providing the rationale for their focus on optimization strategies (e.g., T-Fixup initialization) to stabilize training without architectural changes.