Higher-order action supervision in offline reinforcement learning

Investigate higher-order action supervision for offline reinforcement learning, extending beyond policies learned solely in zeroth-order action spaces.

Background

The paper observes that existing offline reinforcement-learning methods primarily learn policies in zeroth-order action spaces, despite the control-instability and robustness problems associated with neglecting temporal action derivatives. Although the paper develops higher-order supervision mechanisms for several policy classes and evaluates them empirically, it identifies the broader incorporation of higher-order action supervision into offline reinforcement learning as an unexplored research area.

References

Incorporating higher-order action supervision in offline RL remains entirely unexplored.

— Higher-Order Action Supervision Makes A Strong Policy Class  (2610.11175 - Cheng et al., 8 Oct 2026) in Section 2, subsection “Offline RL and continuous-time RL”