Generalization of KD-to-RL Transition Dynamics to Complex Generation Tasks

Determine whether the transition dynamics between On-Policy Distillation and Group Relative Policy Optimization observed in vertical-domain few-shot classification extend to more complex generation tasks.

Background

PIVOT is evaluated on vertical-domain few-shot classification tasks, specifically Banking77 and HWU64, which use relatively structured outputs and explicit reward signals. The method routes samples from On-Policy Distillation to Group Relative Policy Optimization according to teacher-evaluated sequence perplexity, with the goal of matching the transition timing to each sample’s acquisition dynamics.

The paper explicitly identifies the applicability of these transition dynamics beyond structured classification as unresolved. Establishing whether similar dynamics occur in more complex generation tasks would clarify the scope and generality of perplexity-based, sample-level coordination between distillation and reinforcement learning.