Degraded behaviour in failed trajectories under perturbation

Determine whether input perturbations result in failed robot trajectories with meaningfully degraded behaviour in Vision-Language-Action model evaluations.

Background

The evaluation framework focuses on successful trajectories because it requires sufficient successful samples to compare behavioural metrics under baseline and perturbed conditions. Consequently, the effects of perturbations on failed trajectories were not analysed. The authors explicitly identify as a future-work question whether failures are associated with meaningful behavioural degradation, extending the framework beyond its current success-conditioned analysis.

References

Interesting questions for future work, include whether perturbations result in failed trajectories with meaningfully degraded behaviour.

— Is Success All You Need? Investigating the Impact of Input Perturbations on VLA Behaviour in Tabletop Manipulation Tasks  (2610.01351 - Higham et al., 1 Oct 2026) in Section LIMITATIONS AND FUTURE WORK