NoTVLA: Sparse Trajectory for VLA Robotics

• NoTVLA is a paradigm in Vision-Language-Action robotics that reduces dense action sequences to sparse, semantically important keyframes for efficient manipulation.
• It employs automatic keyframe extraction and spline interpolation to reconstruct smooth trajectories while significantly lowering computational costs.
• The approach preserves high-level language reasoning and supports robust multi-task generalization across diverse robotic platforms.

Narrowing of Trajectory VLA (NoTVLA) is a paradigm in Vision-Language-Action (VLA) robotics and control that focuses on reducing dense action sequences to semantically important, sparse trajectories. This approach, introduced in the context of generalizable robot manipulation, addresses critical issues such as catastrophic forgetting, inefficient compute usage, and poor task generalization by emphasizing keyframe-based supervision, anchor-based spatial reasoning, and spline-based trajectory reconstruction. NoTVLA supports robust multi-task scenarios, enhances zero-shot generalization, and maintains intrinsic language understanding, thereby facilitating unified deployment across diverse robot platforms without dependency on dense data streams or specialized sensors (Huang et al., 4 Oct 2025).

1. Motivations and Challenges in Trajectory Narrowing

Standard VLA models typically rely on dense, continuous action trajectories or action chunks that create task-specific data silos. This leads to catastrophic forgetting, where the addition of new task demonstrations can cause loss of previously acquired skills. Furthermore, dense trajectory fine-tuning restricts model adaptability and increases computational cost. NoTVLA was developed to mitigate these problems by narrowing training and inference to sparse, critical keyframes, decoupling high-level semantic reasoning from low-level motor execution. The approach avoids the creation of isolated data silos, maintains broader task generalization, and reduces hardware requirements by obviating the need for wrist-mounted cameras.

2. Sparse Trajectory Planning and Temporal Compression

NoTVLA’s trajectory planning reorganizes the traditional object-centric dense rollout into a sparse format focused on the robot’s end-effector. Keyframes are automatically extracted based on kinematic criteria: a time index $t_k$ is marked as a keyframe if either the acceleration $\|\ddot{T}(t_k)\|_2$ exceeds a threshold $\alpha$, or there is a discontinuous change in the gripper state. When keyframes are chosen, uniform sampling is applied between consecutive keyframes to maintain temporal coherence, and additional sub-keyframes may be inserted as needed. During execution, sparse keyframe tokens are interpolated via cubic splines for position and SLERP for rotation, ensuring a smooth, closed-loop trajectory suitable for robotic manipulation.

3. Training Process with Sparse Supervision

In contrast to dense demonstration annotation, NoTVLA employs a sparse supervision strategy. Training data comprises RGB images, natural language instructions, predicted 2D anchor locations $(u_a, v_a)$, and associated depth $d_a$ from external sources. The Anchor-Conditioned Token Generation (ACTG) module serializes each trajectory as a sequence:

$$S = [\text{CLS}, \text{IMG}, \text{TXT}, \text{ANCHOR}(u_a,v_a, d_a), B_1,\ldots, B_N, \text{EOS}]$$

where each block $B_i = (D_i, (u_i, v_i), G_i, R_i)$ encodes depth, image-plane coordinates, gripper state, and orientation. This reduced annotation frequency diminishes overfitting risks, sustains cross-task performance, and significantly lowers hardware demands.

4. Performance, Generalization, and Robustness

NoTVLA demonstrates superior performance in multi-task evaluations, including RoboTwin and AGIBOT benchmarks, where zero-shot generalization and robustness to adversarial instructions are critical. Success rates are consistently high on tasks such as “click bell,” “grab roller,” and “handover mic,” often surpassing dense baselines like $\pi_0$. Trajectory reconstruction via the spline-based detokenizer yields reduced dynamic time warping distances, lower endpoint errors, and lower Fréchet distances, reflecting superior spatial accuracy and execution smoothness. The sparse approach also preserves semantic model features necessary for cross-platform deployment.

Model/Benchmark	Compute Requirement	Sensor Dependency
NoTVLA	$<$10\% of $\pi_0$	Head-mounted RGB-D only
$\pi_0$	Standard	Wrist camera needed
Expert (ACT)	High, single-task	Varies

NoTVLA’s compute requirements are over an order of magnitude lower than $\pi_0$, and it operates without specialized wrist sensors.

5. Integration of Anchor-Based Depth and Spline Detokenization

NoTVLA infers 3D spatial actions through anchor-based depth prediction: the model conditions its pose estimation on language and vision, then predicts an anchor $(u_a, v_a)$ in the image, fused with external depth $d_a$. This enables spatial reasoning even under varying viewpoints or occlusions. The detokenization step reconstructs smooth, physically-plausible end-effector motion from sparse waypoints via spline methods, yielding robust closed-loop trajectories suitable for a variety of manipulation platforms.

6. Language Reasoning Preservation and Zero-Shot Adaptation

A distinctive advantage of NoTVLA is its preservation of high-level language understanding capabilities. By decoupling vision-language reasoning from motor control, the model remains responsive to diverse, complex instructions and supports zero-shot adaptation. These language features are retained despite the shift to sparse trajectory supervision, as demonstrated by successful exploitation of large-scale pre-trained models (e.g., Qwen VL 2.5 7B). NoTVLA is thus applicable across novel perspectives and manipulation tasks without per-task fine-tuning.

7. Deployment Across Platforms and Future Implications

NoTVLA is validated on multiple robot platforms including Franka Reach, Aloha–AgileX, PiPER, and AGIBOT G1, where unified deployment is achieved without hardware-specific retuning. Key benefits include robust performance under sensor limitations, resilience to camera viewpoint changes, and streamlined adaptation to new environments. This suggests a scalable pathway toward foundational models for embodied AI, where cost-effective, semantically-relevant trajectory narrowing fosters broad generalization and practical applicability.

In conclusion, NoTVLA represents an effective strategy to address catastrophic forgetting and compute inefficiency in VLA systems. By leveraging sparse keyframes, anchor-driven spatial inference, spline-based action synthesis, and modular language integration, it achieves strong task generalization, low computational overhead, and high operational accuracy, supporting a unified paradigm for generalizable robot manipulation (Huang et al., 4 Oct 2025).