Determine when relative achievement encoding improves action-abstraction learning

Determine under which task and state-abstraction conditions encoding Q-Shaped Options by the relative achievement pair comprising the current state and achieved target state improves downstream performance over encoding only the absolute achieved state.

Background

Q-Shaped Options can encode an option using either the absolute achieved state or the relative achievement represented by the current-state and achieved-state pair. The relative representation is motivated as a way to collapse context-dependent behaviors that share the same outcome relative to the current state.

The ablation shows a substantial benefit for relative encoding in antmaze-giant but little effect in cube-double. The authors offer a task-dependent explanation involving the extent to which absolute state-goal pairs can be collapsed, but leave that explanation as a hypothesis.

References

We hypothesise this to be because relative encoding provides the largest benefit when many absolute state-goal pairs $(s,g_s)$ can be collapsed into the same context-dependent achievement, which is the case for the locomotion task, but not necessarily for the manipulation one, where the above state-abstraction analysis suggests that the option retains much more fine-grained information for immediate control.

— Q-Shaped Options for Hierarchical Reinforcement Learning  (2610.12135 - Wibault et al., 8 Oct 2026) in Appendix H.5, paragraph beginning “How sensitive is QSO to option representation input?”