Transfer of RL-learned compositional skills across tasks contingent on atomic prerequisites

Establish whether compositional skills learned via reinforcement learning on a source task transfer to a distinct target task when the model has already acquired the target task’s atomic skills, thereby enabling composition of those atomic skills on the target.

Background

Collecting RL data in every domain is impractical, so the authors explore whether compositional skill learned on one task can act as a meta-skill that transfers to another task. They posit that transfer will succeed only if the model possesses the target’s atomic skills.

Using the Countdown task as a target, they report evidence supporting transfer from RL on a string-transformation source task, conditioned on prior acquisition of Countdown atomic skills, prompting further establishment of this conditional transfer principle across tasks.