Scaling laws for 3D VLM pretraining and downstream robot control

Investigate and characterize the scaling laws that relate SPEAR-1’s downstream robot control performance to the quantity and quality of the 3D visual question answering pretraining data used to train SPEAR-VLM, in order to understand how data scale and data quality affect task performance.

Background

SPEAR-1 is built on SPEAR-VLM, a 3D-aware vision-LLM trained on non-robotic 2D images enriched with 3D annotations. The authors show that this pretraining improves downstream control but note that their experiments were constrained in data scale and diversity.

They explicitly state that the relationship between downstream performance and the quantity and quality of 3D pretraining data is not well understood, leaving open how increasing or curating such data would impact generalization and control accuracy.

References

While we have showed the benefits of 3D VLM pre-training on downstream robot control tasks, the scaling laws relating the latter to the quantity and quality of 3D pre-training data are still not well understood.

— SPEAR-1: Scaling Beyond Robot Demonstrations via 3D Understanding  (2511.17411 - Nikolov et al., 21 Nov 2025) in Section: Discussion and Limitations

The evidence supports joint improvement in perception and transfer, especially generalization, but not a monotonic law connecting grounding score to every action metric. Four settings without repeated-seed uncertainty estimates are insufficient to establish a scaling law or determine whether the final ID decrease is systematic.

— GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives  (2609.39601 - Yu et al., 30 Sep 2026) in Appendix, Section “Grounding and Action Scaling”

Open questions, blocked on further training runs. Left for future compute: the paired video-branch score of Sec.~\ref{sec:asym}; a systematic search for regimes where Flow-DPO hurts beyond the blind-margin finding above, in particular reward hacking on $\Phi$ that would only surface in closed loop; why $\beta$ did not behave as a lever once gradient-norm clipping was in place; the effect of the vision--language screen of Sec.~\ref{sec:screening}; and the scaling curves of Sec.~\ref{sec:unbounded} relating data-engine size to closed-loop success.

— AutodidactWAM: Cross-Modal Self-Distillation from Generated Video to Robot Actions  (2610.08119 - Kurchev et al., 6 Oct 2026) in Section Analysis, subsection “Open questions, blocked on further training runs”