Develop robust multimodal fusion for generalist robotic manipulation policies

Develop principled multimodal fusion techniques that reliably integrate visual, proprioceptive, and linguistic inputs to improve performance and generalization of generalist robotic manipulation policies.

Background

LBMs and related VLA approaches rely on combining multiple sensory modalities. While the paper fixes an architecture for controlled evaluation, it notes that multimodal fusion remains an open research challenge for generalist policies.

Improved fusion could enhance language grounding, perception-action coupling, and robustness under distribution shift.