- The paper introduces GeoFuse-MV3D to deliver conservative geometry-prior-guided 3D object reconstructions that reduce pose ambiguity and enhance photometric consistency.
- The methodology incorporates a governed KnowledgeBank that curates past manipulation experiences using precision gates and explicit quality annotations for reliable recall.
- Quantitative benchmarks demonstrate improved task success and efficiency in both simulations and real-world evaluations, establishing the framework's effectiveness for long-horizon robot planning.
Overview of GeneralVLA-2: Geometry-Aware Reconstruction and Governed Memory for Robot Planning
GeneralVLA-2 introduces significant advancements to hierarchical vision-language-action (VLA) robotics frameworks by augmenting object-centric 3D reconstruction and long-term manipulation memory systems. The core contributions are the GeoFuse-MV3D branch, which delivers conservative geometry-prior-guided multi-view 3D object representations, and a governed KnowledgeBank module, which manages reusable manipulation experience with explicit quality and conflict handling. These innovations directly target two critical limitations impeding robust robotic planning: hallucinated geometry from monocular observation and unreliable experience reuse conditioned solely on semantic similarity.
Technical Contributions
GeoFuse-MV3D: Geometry-Prior-Guided Multi-View 3D Reconstruction
GeoFuse-MV3D addresses the vulnerability of SAM3D- and MV-SAM3D-style methods to pose ambiguity and the generation of implausible unseen geometry from single images. When calibrated, multi-view RGB-D observations are available, GeoFuse-MV3D:
- Integrates external geometry-prior providers into the reconstruction protocol, validating their support against input-view masks and applying appearance calibration for alignment.
- Employs a soft visual hull mechanism for mask-driven geometry regularization without destructive deletion, enabling only small, conservative corrections to object structure.
- Utilizes an axis-compensation branch that performs low-rank, provider-free axis-wise refinement, enhancing consistency with input views while remaining agnostic to external priors.
- Fuses geometry updates conservatively, preserving non-geometric appearance attributes to prevent color, opacity, and other photometric drifts.
This branch operates under a provider-agnostic protocol and explicit mask verification, decoupling appearance preservation from geometry scaffolding. Evaluation on GSO-30, with fixed input views and masks, demonstrates that GeoFuse-MV3D reduces Chamfer Distance (CD) and LPIPS by 2.20% and 2.02% while improving PSNR and SSIM by 2.36% and 1.03%, respectively, over the MV-SAM3D baseline.
Governed KnowledgeBank: Memory Governance and Precision Retrieval
The original GeneralVLA KnowledgeBank appends natural-language memories from past trajectories, relying primarily on semantic similarity for retrieval. This approach is susceptible to retrieving inactive, stale, or contextually inappropriate manipulations, which can compromise planning safety and scene relevance. The governed KnowledgeBank in GeneralVLA-2 introduces:
- Extended record schema containing confidence scores, lifecycle state (provisional, active, summary, archive), verifier-derived quality annotations, conflict/supersession links, and verifier metadata.
- Admission, verifier, and consolidation gates for memory curation: only sufficiently verified memories are promoted, merged, or summarized, while stale or conflicting items are suppressed or archived.
- Precision-oriented retrieval, influencing 3DAgent planning with relevant procedural hints and explicit failure-avoidance constraints, instead of treating all past experience as equally applicable.
Empirical evaluation on Terminal-Bench 2.0 and SWE-Bench Verified establishes that the governed KnowledgeBank outperforms ReasoningBank by 4.53% on Terminal-Bench success rate and 3.73% on SWE-Bench resolve rate, while also reducing average steps (AS) by 4.95% and 5.65%. Component ablations confirm that governance, conflict handling, and failure-aware filtering are necessary for memory-augmented agent reliability.
Experimental Results
Simulation-Based Task Coverage
Experiments on RLBench with a Franka Panda platform and 14 tasks reveal that GeneralVLA-2 achieves coverage on all tasks, outscoring CAP, Hamster, and VoxPoser baselines. Ablation without KnowledgeBank guidance led to consistent reductions in task success, confirming the benefit of governed experience reuse for long-horizon planning.
Real-World Demonstrations
In real-robot evaluations, GeneralVLA-2 successfully executes diverse tabletop manipulation tasks ("move_spray bottle", "open_drawer", etc.) with higher success rates than CAP and RoboPoint. For instance, GeneralVLA-2 obtains success rates of 63.33%, 40.00%, 53.33%, and 83.33% across four representative tasks without training, demonstrating the practical benefit of its improved evidence pipeline.
Despite added computational overhead from verifier and governance calls, the shorter, more accurate task executions of the governed KnowledgeBank architecture result in reduced total tokens and latency compared to less principled memory architectures.
Limitations and Implications
GeneralVLA-2's methodology assumes reliable input masks, camera calibrations, and verifier competence. Errors in these upstream modules propagate to planning, while the conservative design intentionally prioritizes stability over aggressive geometric correction, leading to incremental—but consistent—gains over previous methods. Long-horizon mobile manipulation, heavy occlusion, and tasks involving deformable objects remain open challenges.
From a practical perspective, GeneralVLA-2 solidifies the value of explicit memory governance in experience-augmented robotics: only high-quality, contextually verified experience should influence future plans. Theoretically, the work motivates future directions in (1) robustifying multi-view 3D reconstruction under variable observation quality and (2) scaling governed memory to open-world, online-adaptive agents. The role of persistent verifier frameworks and metadata-rich agent memory is likely to be central for further advances in the reliability and generalizability of VLA systems.
Conclusion
GeneralVLA-2 systematically improves robot manipulation planning by (1) introducing GeoFuse-MV3D for better object-centric geometry under multi-view input and (2) instantiating a governed KnowledgeBank for precise, reliable experience reuse. Quantitative and qualitative benchmarks confirm stronger 3D evidence and improved memory-augmented planning, establishing a clear methodological foundation for VLA systems prioritizing verifiable geometric evidence and trusted, conflict-aware experiential guidance (2606.17480).
