AutoSpeed: Annotation-Free Stage-Adaptive Motion Speed Learning for Robot Manipulation

Abstract: Different stages of manipulation tasks exhibit varying levels of difficulty, suggesting stage-dependent motion speeds and temporal prediction horizons. However, existing IL-based visuomotor policies typically imitate the execution speed of expert demonstrations and operate with a fixed temporal prediction horizon, limiting flexibility and overall task throughput. In this paper, we introduce AutoSpeed, a model-agnostic learning framework that enables existing visuomotor policies to predict trajectories with stage-adaptive motion speeds, without requiring speed or stage annotations. We treat future trajectories at different speeds as candidate optimization targets, evaluate each candidate using a composite cost that trades off prediction error against prediction horizon, and optimize the policy toward the minimum-cost candidate. With a fixed-length action sequence, speed modulation adjusts the effective temporal prediction horizon: simple stages are executed faster with a longer prediction horizon, whereas complex stages are executed more slowly with a shorter prediction horizon. Specifically, we implement speed modulation in the frequency domain via the discrete cosine transform (DCT), which enables smooth, non-integer speed scaling and thus preserves motion continuity. Extensive evaluations show that AutoSpeed substantially reduces task execution time while also improving success rates. Under the AutoSpeed framework, the inferred motion speeds exhibit a strong correspondence with task stages.

• The paper introduces a model-agnostic method that learns stage-adaptive motion speeds without relying on explicit annotations.
• It employs DCT-based frequency scaling and a cost-aware multi-target optimization to balance speed and prediction accuracy across task stages.
• Experimental results on simulated and real-world tasks demonstrate improved efficiency, with up to a 1.78x reduction in execution time and maintained or enhanced success rates.

AutoSpeed: Annotation-Free Stage-Adaptive Motion Speed Learning for Robot Manipulation

Motivation and Problem Formulation

Imitation learning-based visuomotor policies for robot manipulation typically replicate the motion speed and temporal horizons present in expert demonstrations, which are often suboptimal and not tailored to the difficulty profile of individual task stages. The absence of stage-aware motion adaptation restricts policy efficiency and can degrade real-world task throughput. Numerous tasks, as observed in both human motor control literature and robotic practice, demand dynamic adjustment of both speed and action prediction horizon: simple, free-space phases permit rapid execution and long-horizon action chunking, while challenging, contact-rich stages necessitate fine-grained control and frequent feedback. However, current approaches do not adapt dynamically to these requirements and generally rely on fixed-rate imitation, often constrained by annotation bottlenecks and lack of stage signal generalizability.

Figure 1: Stage-aware motion speed adaptation; AutoSpeed infers stage-adaptive speed without requiring explicit annotations.

Methodology: Model-Agnostic Stage-Adaptive Speed Modulation

AutoSpeed advances the field by proposing a model-agnostic framework enabling policies to infer and execute stage-adaptive motion speeds end-to-end without any explicit speed or stage annotations. The central mechanism operates via cost-aware multi-target selective optimization: given a demonstrated trajectory, candidate future action chunks are generated at multiple retimed speeds by applying DCT-based scaling in the frequency domain, permitting smooth non-integer speed modulation and preservation of critical high-frequency action details. For each candidate, a composite cost function $J$ jointly accounts for the mean-squared prediction error and adapts the penalty based on temporal prediction horizon. The policy loss is computed w.r.t. the candidate that minimizes this cost, thus implicitly shaping the speed selection signal to be both feasible (low prediction error) and efficient (longer horizons where possible).

Figure 2: AutoSpeed overview illustrating the multi-target selective optimization and contrasting generative-model training with/without stage-adaptive speed selection.

Notably, AutoSpeed is compatible with both non-generative (e.g., standard MLP or transformer decoders) and generative (diffusion- or flow-matching-based) policy architectures. For generative models, candidate-specific denoising is parallelized, and a three-stage training schedule stabilizes convergence: uniform averaging in the early stage, cost-based selection in the mid-stage, and direct optimization via a lightweight Ratio Head that infers the optimal speed ratio from latent observations. The Nonlinear Temporal Aggregation (NTA) procedure provides smooth inference-time aggregation across overlapping action chunks with variable horizons.

Benchmark Evaluation and Quantitative Analysis

Simulation Tasks

Extensive benchmarking was conducted across 62 tasks drawn from ALOHA, Meta-World, and LIBERO-10, with both single- and multi-task paradigms.

Figure 3: Simulation tasks coverage encompasses ALOHA, LIBERO-10, and Meta-World with diverse manipulation challenges.

On the challenging ALOHA Transfer Cube task, AutoSpeed reduces mean episode length from 272 to 160 time steps (a 1.7x speedup) with only a minor decrease in vanilla ACT's success rate (from 72% to 64%), and pairing AutoSpeed with a high-gain controller not only maintains but improves the success rate to 78%. For the contact-rich Insertion task, AutoSpeed increases success from 22% to 24% while also shortening episodes. These findings establish that task efficiency and robustness can be simultaneously improved via adaptive speed/horizon learning.

Multi-Task and Real-World Results

In multi-task settings, e.g., Meta-World, AutoSpeed provides a substantial lift: BAKU-DiT (AutoSpeed) raises success rate from 43% to 62% (+19%) while reducing the average episode to 70 steps. Similar trends occur across other model variants and LIBERO-10 tasks, indicating broad applicability and model-agnostic generalization. Significantly, acceleration does not sacrifice precision; in real-world deployments, the effect is accentuated.

Real-World Deployments

Policies integrated with AutoSpeed and NTA on a bimanual Agilex Piper platform yield an average 1.78x reduction in task execution time relative to the baseline, coupled with consistent improvements or maintenance of task success rates across all four tested tasks. For the challenging “Place the Toy” scenario, AutoSpeed delivers a 10% absolute improvement in success rate, reinforcing its ability to infer semantically appropriate speed modulation policies from unannotated expert data.

Figure 4: Speed ratio curves in real-world tasks qualitatively track underlying task phases, confirming inferred ratios correlate with true manipulation difficulty.

Analysis: Phase-Awareness and Systematic Ablations

Ablation studies confirm the robustness and phase-responsiveness of AutoSpeed's policy inference over a variety of speed range bounds and loss weighting parameters $w$. Regardless of range, the learned speed profile consistently slows during interaction-critical phases and accelerates in unconstrained motion, validating the optimization-driven emergence of meaningful stage-adaptive behavior. Adjusting $w$ provides an effective means of biasing the resultant motion profile toward either more aggressive or safer execution, subject to user requirements.

Figure 5: Phase-aware deceleration and acceleration appear consistently across different speed range bounds; success rates and efficiency both outperform the vanilla baseline.

Figure 6: Variation in the length penalty coefficient $w$ demonstrates impact on motion style but maintains underlying phase awareness.

Theoretical and Practical Implications

AutoSpeed’s formulation unifies motion speed modulation and prediction horizon adaptation into a single objective function, streamlined into the policy’s end-to-end learning dynamics. This construct eliminates the need for costly or unreliable annotation, external proxy policies, or ad hoc heuristics, thus enabling direct, self-supervised discovery of locally optimal execution profiles. The DCT-based frequency scaling ensures that the retimed trajectory candidates do not sacrifice critical action details, which is essential for contact- and precision-dominated manipulation stages.

Practically, the method supports straightforward integration into state-of-the-art VLA and transformer-based policy stacks without architectural disruption. Its flexibility across both non-generative and modern generative robot control frameworks positions it as a scalable solution for industrial, multi-task, and real-world manipulation scenarios where efficiency and robustness are paramount.

Conclusion

AutoSpeed delivers a robust, annotation-free approach for learning stage-adaptive motion speeds and prediction horizons in robotic manipulation. The data-driven, end-to-end multi-target optimization directly imbues visuomotor policies with the ability to reason about both speed and temporal context at a per-stage level, rather than operating under suboptimal fixed-rate constraints. Theoretical contributions (e.g., frequency-domain retiming and cost-driven selection) are matched by substantial empirical gains in both task efficiency and reliability across simulated and physical platforms. Future research should consider extending AutoSpeed to settings with even greater task diversity and exploring integration with more sophisticated feedback models for fine-grained adaptivity.

(2607.01051)