Applying Transformers to Reinforcement Learning

Develop robust techniques and training procedures that enable the transformer architecture to be effectively and stably applied to reinforcement learning tasks and environments.

Background

The work introduces TrMRL, a transformer-based meta-reinforcement learning agent, and reviews prior attempts to use transformers in reinforcement learning. These attempts often encountered instability or required architectural modifications or observation constraints, underscoring unresolved challenges in this area.

Motivated by this context, the authors explicitly state that applying transformers in reinforcement learning remains an open challenge, providing the rationale for their focus on optimization strategies (e.g., T-Fixup initialization) to stabilize training without architectural changes.

References

The application of the transformer architecture in the RL setting is still an open challenge.

— Transformers are Meta-Reinforcement Learners  (2206.06614 - Melo, 2022) in Section 2, Transformers for RL (Related Work)

These findings leave open the effectiveness of recommendation with DTs, control at the item level, learned control tokens, and canonical autoregressive DT rollouts.

— Auditing Return Conditioning as a Control Knob: An Offline Diagnostic for Decision Transformer Recommendation  (2608.24815 - Wang, 25 Aug 2026) in Section 4, Implications