CLASP: Closed-loop Asynchronous Spatial Perception for Open-vocabulary Desktop Object Grasping

Abstract: Robot grasping of desktop object is widely used in intelligent manufacturing, logistics, and agriculture.Although vision-LLMs (VLMs) show strong potential for robotic manipulation, their deployment in low-level grasping faces key challenges: scarce high-quality multimodal demonstrations, spatial hallucination caused by weak geometric grounding, and the fragility of open-loop execution in dynamic environments. To address these challenges, we propose Closed-Loop Asynchronous Spatial Perception(CLASP), a novel asynchronous closed-loop framework that integrates multimodal perception, logical reasoning, and state-reflective feedback. First, we design a Dual-Pathway Hierarchical Perception module that decouples high-level semantic intent from geometric grounding. The design guides the output of the inference model and the definite action tuples, reducing spatial illusions. Second, an Asynchronous Closed-Loop Evaluator is implemented to compare pre- and post-execution states, providing text-based diagnostic feedback to establish a robust error-correction loop and improving the vulnerability of traditional open-loop execution in dynamic environments. Finally, we design a scalable multi-modal data engine that automatically synthesizes high-quality spatial annotations and reasoning templates from real and synthetic scenes without human teleoperation. Extensive experiments demonstrate that our approach significantly outperforms existing baselines, achieving an 87.0% overall success rate. Notably, the proposed framework exhibits remarkable generalization across diverse objects, bridging the sim-to-real gap and providing exceptional robustness in geometrically challenging categories and cluttered scenarios.

• The paper presents CLASP, a closed-loop framework that decouples semantic intent and geometric grounding to mitigate spatial hallucinations in object grasping.
• It employs VLM-driven grasp reasoning and asynchronous corrective feedback, achieving an 87% success rate and reducing latency by ~40% in cluttered settings.
• The use of a scalable multimodal dataset, with over 500k synthetic and real scenes, enhances generalization in diverse, dynamic desktop environments.

Closed-Loop Asynchronous Spatial Perception for Open-vocabulary Desktop Object Grasping

Introduction

The proliferation of vision-LLMs (VLMs) has enabled semantic understanding for robotic manipulation, but their deployment in low-level grasping is fundamentally limited by a scarcity of high-quality multimodal demonstrations, poor geometric grounding, and the fragility of open-loop execution in dynamic, cluttered environments. The paper "CLASP: Closed-loop Asynchronous Spatial Perception for Open-vocabulary Desktop Object Grasping" (2604.11320) presents the CLASP framework, an integrated, asynchronous closed-loop architecture that addresses these challenges via decoupled hierarchical perception, VLM-based constrained reasoning, and text-based asynchronous corrective feedback.

Framework Architecture

CLASP decomposes the open-vocabulary grasping problem into three primary modular components: Dual-Pathway Hierarchical Perception, VLM-driven Structured Grasp Reasoning, and a Closed-Loop Asynchronous Evaluator. The system initiates from natural language instructions and RGB scene observations and outputs pixel-precise 2D grasp configurations parameterized by contact points and orientation, which are mapped to 7-DoF manipulator controls after collision and spatial consistency checks.

Figure 1: The CLASP framework depicts input processing, dual-pathway semantic/geometric awareness, heuristic VLM reasoning, and closed-loop correction modules for robust instruction-driven grasping.

The Dual-Pathway Hierarchical Perception module decouples semantic intent extraction and geometric grounding. The semantic pathway performs dense instruction-based representation learning, while the geometric pathway generates open-vocabulary bounding boxes, instance masks, and pose descriptors. This architecture explicitly mitigates spatial hallucination, as the geometric constraints enforce physically valid grasp candidates regardless of semantic uncertainty.

VLM-based reasoning is structured as a deterministic parsing task: JSON-formatted grasp parameters are output, incorporating spatial and collision constraints to ensure action feasibility. The output is not only semantically valid but also fully specified in the robot’s operational space.

The asynchronous evaluator operates by comparing pre- and post-execution world states with a text-driven Judger module. In the event of failure modes—misgrasp, incorrect placement, or environmental interference—a heuristic diagnostic is issued (e.g., shift grasp points towards the mass center), closing the adaptation loop to reduce failure rates in non-stationary settings.

Dataset Construction and Synthetic Augmentation

To achieve high annotation density and task diversity, the authors construct a scalable multimodal data engine integrating over 500k synthetic and real desktop scenes, blending RGB-D images, dense masks, and precise 3D model information. Scenes are sampled from highly cluttered everyday contexts, and affordance annotation is provided for both spatial and functional regions. Negative samples (infeasible/unsafe grasps) are explicitly synthesized, promoting robust boundary learning within the reasoning model.

Quantitative Results

Experimental evaluations are conducted using the ManiSkill simulator with the WidowX 250S platform as well as on physical robot deployment. Without any in-the-loop fine-tuning, CLASP achieves an 87.0% overall pick success rate, outperforming strong VLM (Qwen3-VL, Doubao-Seed) and affordance-based (AffordanceNet, AnyGrasp) baselines. Notably, CLASP also performs robustly under real-world implementation with generalization to previously unseen objects and significant resilience in tool- and clutter-heavy scenarios, where geometric and semantic ambiguities are most pronounced.

Figure 2: Success rates for the pick task across Toys, Blocks, and Tools, demonstrating consistent superiority of CLASP under varied object categories and attempt budgets.

Figure 3: WidowX 250S manipulator executing categorization and placement tasks for tools, toys, and blocks in a real-world unstructured environment.

Among evaluated object classes, block-type objects are the easiest, achieving 89.0% with three attempts; tool-type objects exhibit the lowest pick rates (33.0%), reflecting their geometric complexity. The closed-loop corrective feedback implementation, particularly the Judger and Heuristic modules, raises success rates for challenging objects by up to 20 percentage points relative to ablative baselines. For the categorization task, CLASP attains 83.33% success, significantly outperforming Qwen3-VL (63.33%) and Doubao-Seed (46.67%).

System Latency and Computational Throughput

Execution efficiency in robotics is paramount. The asynchronous decoupling of the Reasoner and Judger modules reduces total task completion time by ~40% and lowers temporal variance by almost 60% compared to classical streaming execution policies. This advance is non-trivial in deployed settings, as it minimizes idle time and maximizes throughput without sacrificing sample efficiency or final grasp performance.

Qualitative Analysis of Grasp Precision

CLASP demonstrates significantly improved grasp point selection in complex arrangements. Baseline VLM models (Qwen3-VL) and segmentation-based methods frequently select non-optimal or invalid contact points (e.g., grasping at the edge or on non-affordant surfaces), leading to increased slippage and failed placements. In contrast, CLASP’s closed-loop and geometry-informed reasoning localizes antipodal grasp points, notably on complex tools (e.g., wrenches, scissors), enhancing both grasp stability and semantic correctness.

Figure 4: CLASP selects optimal opposing grasp locations even on challenging geometries; baselines fail due to mislocalization or invalid action proposals.

Theoretical and Practical Implications

On the theoretical front, the explicit disentanglement of semantic and geometric reasoning constrains the VLM’s action space, addressing the spatial hallucination problem and enabling robust grasp parameterization for open-vocabulary instructions. Closed-loop asynchronous feedback introduces error-corrective adaptation—closing the physical-symbolic gap that compromises pure open-loop VLM deployment.

Practically, CLASP’s scalable data synthesis pipeline and real-to-sim transfer offer a viable path for generalized manipulation in resource-constrained or long-tail task distributions, as it avoids the heavy human annotation and model fine-tuning typical of prior VLA paradigms. The computational advances in asynchronous execution ensure readiness for on-hardware deployment, with direct implications for industrial, logistics, and service robot contexts where low-latency, high-fidelity grasping is mission-critical.

Future Directions

Possible extensions of CLASP include scaling to longer-horizon, multi-step manipulation sequences, handling occlusions and dynamic objects, and integrating more expressive affordance priors. There is significant potential for leveraging additional feedback channels (e.g., tactile, force-torque) within the closed-loop Judger module, and for improving compositional reasoning in environments with adversarial clutter or novel object categories.

Conclusion

CLASP sets a new state of the art in open-vocabulary robot grasping by combining decoupled semantic-geometric grounding, scalable structured data generation, VLM-constrained action formulation, and closed-loop asynchronous feedback. Strong empirical performance documented across simulation and real-world settings validates its architectural choices, particularly in high-diversity and cluttered scenes. Its generalization and execution efficiency establish a foundation for more advanced spatial reasoning and robust manipulation under ambiguous, instruction-driven task specifications.