Cross-expert robustness of recoverability-based selection

Establish whether the benefit of interventional recoverability for Vision-Language-Action recovery-data selection and the stability of recoverability frontiers persist across different recovery experts.

Background

The Kintsugi-VLA framework defines recoverability relative to a particular privileged recovery expert rather than as an intrinsic property of a simulator state. The paper’s audit finds substantial variation in state-wise recovery rankings among nominally competent teachers, with pairwise Spearman correlations ranging from 0.103 to 0.693. Because changing the expert may alter difficulty bins, selected states, and generated demonstrations, the robustness of the downstream VLA benefit and of stable recoverability frontiers across experts remains unresolved.

References

Cross-expert robustness of the VLA benefit and stable frontiers remains untested.

— Kintsugi-VLA: Turning Failed Robot Rollouts into Recovery Data through Interventional Recoverability  (2609.31048 - Snegirev et al., 25 Sep 2026) in Section LIMITATIONS, subsection “Dependence on the recovery expert”

Hardware/G1, release-and-place behavior, and post-success stability remain untested.

— Kintsugi-VLA: Turning Failed Robot Rollouts into Recovery Data through Interventional Recoverability  (2609.31048 - Snegirev et al., 25 Sep 2026) in Section LIMITATIONS, subsection “Measurement and evaluation scope”

Island-aware selection and joint optimization of recovery points and measurement budget remain open.

— Kintsugi-VLA: Turning Failed Robot Rollouts into Recovery Data through Interventional Recoverability  (2609.31048 - Snegirev et al., 25 Sep 2026) in Section LIMITATIONS, subsection “Selection, diagnosis, and total cost”

Reported data-efficiency results concern successful demonstrations and synchronized frames only; total savings remain unmeasured after including restoration, recoverability estimation, unsuccessful continuations, and training.

— Kintsugi-VLA: Turning Failed Robot Rollouts into Recovery Data through Interventional Recoverability  (2609.31048 - Snegirev et al., 25 Sep 2026) in Section LIMITATIONS, subsection “Selection, diagnosis, and total cost”

Open questions, blocked on further training runs. Left for future compute: the paired video-branch score of Sec.~\ref{sec:asym}; a systematic search for regimes where Flow-DPO hurts beyond the blind-margin finding above, in particular reward hacking on $\Phi$ that would only surface in closed loop; why $\beta$ did not behave as a lever once gradient-norm clipping was in place; the effect of the vision--language screen of Sec.~\ref{sec:screening}; and the scaling curves of Sec.~\ref{sec:unbounded} relating data-engine size to closed-loop success.

— AutodidactWAM: Cross-Modal Self-Distillation from Generated Video to Robot Actions  (2610.08119 - Kurchev et al., 6 Oct 2026) in Section Analysis, subsection “Open questions, blocked on further training runs”