Unbiased divide-and-conquer value learning in stochastic environments

Determine whether a divide-and-conquer value learning technique, such as Transitive RL’s triangle inequality-based update, can be applied to goal-conditioned reinforcement learning in stochastic environments to learn unbiased value functions.

Background

Transitive RL (TRL) is developed for offline goal-conditioned reinforcement learning and relies on the triangle inequality over temporal distances, an assumption that is exact in deterministic environments. While the paper notes related structures in stochastic settings, TRL’s current formulation targets deterministic dynamics, and unbiased value estimation under stochastic transitions remains unresolved.

The authors point to the successor temporal distance framework as a potential avenue, highlighting that establishing unbiased value learning via divide-and-conquer under stochastic dynamics would address a key limitation of approaches based on the vanilla triangle inequality.