Generalization of oursink to alternative learners and policies

Determine how the oursink performance-guided initial-state collection strategy performs with base reinforcement-learning methods and base policies other than the HiL-IDQL learner and Diffusion Policy actor evaluated in the paper, including generalist policies updated between supervised-deployment rounds.

Background

The paper evaluates oursink with two closely related learning configurations: HG-DAgger, which behavior-clones successful human-controlled data, and HiL-IDQL, which combines a Diffusion Policy actor with an implicit-Q-learning critic trained on all collected data and uses best-of-N action reranking. The strategy selects initial states for subsequent supervised-deployment rounds by promoting observed failures and filling remaining budget with underexplored states.

The authors identify the learner and policy-class coverage as a limitation of the evaluation. They leave unresolved whether the observed data-efficiency gains are specific to the tested actor–critic formulation or extend to other reinforcement-learning algorithms and other policy architectures, particularly generalist policies that are updated between rounds.

References

How the same collection strategy performs with other base RL methods and other base policies, including generalist policies updated between rounds, is open.

— Mulligan: Performance-Guided Data Collection for Efficient On-Robot Learning  (2610.05882 - Ankile et al., 5 Oct 2026) in Discussion, Section 6

Second, SGS samples from a given distribution but does not construct it: we rely on reset-diversity methods such as OmniReset~\citep{yin2026emergent} for broad coverage, and closing the loop on which configurations to make available remains an open problem.

— A Balanced Data Diet: Addressing the Exploration Bottleneck in Mega-Scale RL for Robot Control  (2610.12465 - Zhang et al., 8 Oct 2026) in Section 6, “Limitations”