SmoothVLA: RL Fine-Tuning for Smooth Robotic Manipulation

• SmoothVLA is a reinforcement learning fine-tuning framework that explicitly incorporates a jerk-based smoothness term to achieve stable robotic manipulation.
• It leverages Group Relative Policy Optimization to select the smoothest successful trajectory from multiple rollouts, reducing erratic motions in vision-language-action policies.
• Empirical results on LIBERO benchmarks demonstrate improved success rates and lower average jerk, enhancing robustness and generalization under various perturbations.

SmoothVLA is a reinforcement learning fine-tuning framework for vision-language-action (VLA) models that aligns robotic control with physical constraints through intrinsic smoothness optimization. Introduced for robotic manipulation with an OpenVLA backbone, it combines sparse binary task success with a continuous dense term derived from trajectory jerk, computed directly from policy rollouts rather than from environment-specific reward engineering. In this formulation, smoothness is not treated as an emergent by-product of imitation quality or post hoc trajectory filtering, but as an explicit optimization prior within Group Relative Policy Optimization (GRPO), with the stated aim of reducing erratic, jittery trajectories while preserving or improving task success and generalization (Li et al., 14 Mar 2026).

1. Definition, scope, and naming

In its primary and most specific usage, SmoothVLA denotes the framework presented in "SmoothVLA: Aligning Vision-Language-Action Models with Physical Constraints via Intrinsic Smoothness Optimization" (Li et al., 14 Mar 2026). The method addresses post-training of VLA policies for manipulation, particularly the tension between the smooth but distribution-bound behavior of supervised fine-tuning (SFT) and the exploratory but often unstable behavior of standard reinforcement learning.

The framework is defined by three coupled elements: a pretrained VLA model, online RL fine-tuning with GRPO, and a physics-informed hybrid reward in which binary task success is modulated by a dense jerk-based smoothness term. The reward is described as intrinsic because it is computed directly from policy rollouts through differential kinematics and proprioception, without requiring extrinsic environment-specific shaping.

The term also appears in adjacent but distinct senses in contemporaneous literature. In "Smooth Operator," SmoothVLA refers to a training framework for activating 3D spatial reasoning in vision-LLMs through Smooth Numerical Reward Activation (SNRA) and Absolute-Preserving GRPO (AP-GRPO), with no architectural modification to the base VLM (Jiao et al., 12 Jan 2026). In "MotionVLA," by contrast, the term does not appear in the paper; it is only used as an interpretation for a VLA system whose executions are smoother and more direct because motion-consistent memory is retrieved and grounded for control (Yuan et al., 6 Jun 2026). In "VLA-RAIL," the relevant contribution is an execution-layer infrastructure for smooth, continuous, high-speed action-chunk deployment rather than a post-training alignment method (Zhao et al., 31 Dec 2025). These distinctions matter because the robotics literature uses “smoothness” to refer to at least three different loci: policy optimization, representation and memory, and trajectory execution.

2. Problem setting and motivation

SmoothVLA is motivated by what the paper characterizes as an exploration–stability paradox in VLA post-training. SFT can learn globally coherent and smooth motion patterns from expert demonstrations, but its generalization is constrained by the demonstration distribution; it struggles under out-of-distribution perturbations such as novel language instructions, lighting, backgrounds, and layouts. Standard RL, in contrast, improves exploration and generalization through online interaction, yet often produces erratic trajectories with detours, backtracking, and high-frequency oscillations, especially near grasp and placement events (Li et al., 14 Mar 2026).

The paper frames these behaviors as violations of physical constraints such as actuator bandwidth, and links them to practical consequences including torque spikes, sensor noise, degraded success, and unstable execution. SmoothVLA therefore sets a dual objective: preserve or improve task success while explicitly optimizing motion smoothness.

A central design choice is to use jerk as the smoothness variable. Jerk is defined as

$j(t) = \frac{d^3 x(t)}{dt^3},$

the time derivative of acceleration. Within the method’s rationale, jerk is preferable to lower-order quantities such as velocity or acceleration because it directly penalizes rapid changes in acceleration, which are more tightly associated with torque spikes and residual vibrations. The paper explicitly connects this choice to minimum-jerk theory and prior control literature on dynamic stability. This suggests that SmoothVLA treats smoothness not merely as aesthetic regularity of the path but as a physically meaningful proxy for feasible and stable control.

3. Physics-informed intrinsic smoothness optimization

The policy is written as $\pi_\theta$ over multimodal input

$$s = (o_{\mathrm{vis}}, o_{\mathrm{prop}}, l_{\mathrm{task}}),$$

where $o_{\mathrm{vis}}$ denotes visual observations such as RGB-D, $o_{\mathrm{prop}}$ denotes proprioceptive state such as joint positions $q$, and $l_{\mathrm{task}}$ denotes the language instruction. Actions are continuous end-effector control, either target pose or velocity, and the model may output an action chunk $a_{t:t+k} = \{a_t,\dots,a_{t+k-1}\}$ for temporal coherence (Li et al., 14 Mar 2026).

The smoothness term is computed from rollout trajectories mapped to joint space through inverse kinematics. The paper gives the operational-space kinematic relations

$V = J(q)\dot{q},$

$\dot{A} = J(q,\dot{q})\dot{q} + J(q)\ddot{q},$

and

$\pi_\theta$0

In practice, jerk is estimated discretely along rollout timesteps from action sequences and proprioceptive feedback, with acceleration and jerk obtained by finite differences of velocity and acceleration.

The core reward is the trajectory-level hybrid objective

$\pi_\theta$1

where $\pi_\theta$2 is the binary task-success indicator and $\pi_\theta$3 is the smoothness weight, set to $\pi_\theta$4 via grid search. The jerk term uses the squared magnitude $\pi_\theta$5, aggregated across degrees of freedom and averaged over timesteps by the factor $\pi_\theta$6.

Several consequences follow directly from this construction. First, smoothness only contributes when the rollout is successful, because the reward is multiplied by $\pi_\theta$7. Second, among successful rollouts for the same instruction, lower-jerk trajectories receive higher reward. Third, no environment-specific shaping terms are needed: the method computes smoothness intrinsically from the policy’s own trajectories and proprioceptive traces. The paper argues that this suppresses torque spikes and residual vibrations while yielding more human-like, energy-efficient motion.

4. GRPO optimization and training pipeline

SmoothVLA uses GRPO as an actor-only, trajectory-level RL method without a learned critic (Li et al., 14 Mar 2026). For a given instruction $\pi_\theta$8, the algorithm samples a group of $\pi_\theta$9 trajectories $$s = (o_{\mathrm{vis}}, o_{\mathrm{prop}}, l_{\mathrm{task}}),$$0, computes each trajectory reward $$s = (o_{\mathrm{vis}}, o_{\mathrm{prop}}, l_{\mathrm{task}}),$$1, and forms the group-normalized advantage

$$s = (o_{\mathrm{vis}}, o_{\mathrm{prop}}, l_{\mathrm{task}}),$$2

The paper gives the GRPO objective as

$$s = (o_{\mathrm{vis}}, o_{\mathrm{prop}}, l_{\mathrm{task}}),$$3

Within SmoothVLA, this means that smoothness acts as an optimization prior inside intra-group competition: GRPO prefers the smoothest successful trajectory among multiple attempts for the same language instruction. The clipped objective and KL regularization are intended to stabilize updates while preserving exploration.

The reported implementation uses OpenVLA as the backbone policy with LoRA fine-tuning and AdamW. Supervised fine-tuning uses learning rate $$s = (o_{\mathrm{vis}}, o_{\mathrm{prop}}, l_{\mathrm{task}}),$$4 and batch size 16; preference/RL fine-tuning uses learning rate $$s = (o_{\mathrm{vis}}, o_{\mathrm{prop}}, l_{\mathrm{task}}),$$5 and batch size 16. Benchmarks are LIBERO and LIBERO-Plus. Observation space is $$s = (o_{\mathrm{vis}}, o_{\mathrm{prop}}, l_{\mathrm{task}}),$$6 with RGB-D, joint states, and text instruction. The paper states that other RL hyperparameters, including $$s = (o_{\mathrm{vis}}, o_{\mathrm{prop}}, l_{\mathrm{task}}),$$7, $$s = (o_{\mathrm{vis}}, o_{\mathrm{prop}}, l_{\mathrm{task}}),$$8, group size $$s = (o_{\mathrm{vis}}, o_{\mathrm{prop}}, l_{\mathrm{task}}),$$9, rollout horizon, and related quantities, are not specified.

A concise view of the training loop is as follows. For each instruction, the policy collects a group of rollouts, records actions and proprioception, determines task success, computes jerk through differential kinematics or finite differences, forms the hybrid reward, normalizes advantages within the group, and updates the policy by gradient ascent on the GRPO objective. This procedural coupling of success and smoothness is the distinctive mechanism of the framework.

5. Empirical results on smoothness, success, and robustness

The evaluation centers on LIBERO and LIBERO-Plus, with success rate and average jerk of end-effector trajectories as principal metrics (Li et al., 14 Mar 2026). Jerk is reported in $o_{\mathrm{vis}}$0 and averaged across the LIBERO suites Spatial, Object, Goal, and Long.

For smoothness, the reported baselines are OpenVLA-SFT with average jerk $o_{\mathrm{vis}}$1 and OpenVLA-RL with average jerk $o_{\mathrm{vis}}$2. SmoothVLA is reported to improve motion smoothness by $o_{\mathrm{vis}}$3 versus SFT and by $o_{\mathrm{vis}}$4 versus standard RL. The qualitative analysis attributes these gains to the elimination of backtracking and local jitter near objects, more stable posture during placement, and avoidance of collisions.

For in-domain task success, SmoothVLA-GRPO achieves $o_{\mathrm{vis}}$5 average success, a $o_{\mathrm{vis}}$6 point improvement over Octo at $o_{\mathrm{vis}}$7. The paper also reports that applying SmoothVLA as a general plugin to DPO- and GRPO-based RL fine-tuning produces an additional $o_{\mathrm{vis}}$8 average gain.

Robustness is evaluated on LIBERO-Plus under rewritten instructions, changed illumination and shadows, background changes, and layout perturbations. The reported averages are:

Method	Original	Average
OpenVLA-SFT	82.1	22.0
OpenVLA-DPO	84.2	39.2
SmoothVLA-DPO	85.2	44.8
OpenVLA-GRPO	84.5	39.2
SmoothVLA-GRPO	85.6	46.2

The detailed breakdown shows SmoothVLA-GRPO at 43.3 on Language, 16.1 on Light, 61.2 on Background, and 64.4 on Layout. The paper states that SmoothVLA variants lead supervised fine-tuning by $o_{\mathrm{vis}}$9 in average success under perturbations and outperform DPO/GRPO baselines by $o_{\mathrm{prop}}$0–$o_{\mathrm{prop}}$1 on average, with smaller performance degradation out of distribution.

Ablations on LIBERO-Spatial compare $o_{\mathrm{prop}}$2, $o_{\mathrm{prop}}$3, and $o_{\mathrm{prop}}$4. The averages are 48.2, 46.5, and 54.1, respectively, with $o_{\mathrm{prop}}$5 also improving each perturbation category. This suggests that the jerk-based term is not merely regularizing trajectory appearance; it is also functioning as a robustness prior during policy adaptation.

6. Interpretation, related directions, and limitations

SmoothVLA belongs to a broader family of attempts to make VLA systems smoother, but it operates at the RL alignment layer rather than the representation or execution layers. MotionVLA improves long-horizon manipulation by replacing sparse historical context with motion-consistent, time-continuous trajectory-field tokens and reports smoother, more direct executions without any explicit acceleration or jerk penalties (Yuan et al., 6 Jun 2026). VLA-RAIL improves smoothness after inference through cubic intra-chunk fitting, dual-quintic chunk fusion, and an asynchronous inference-control architecture that guarantees continuity of position, velocity, and acceleration across action-chunk boundaries (Zhao et al., 31 Dec 2025). By contrast, SmoothVLA changes the policy’s optimization target itself.

The distinction from "Smooth Operator" is equally important. There, SmoothVLA refers to an RL framework for vision-LLMs in which SNRA converts verifiable numerical error into dense reward and AP-GRPO preserves absolute scalar information during optimization, targeting 3D spatial reasoning rather than robotic manipulation (Jiao et al., 12 Jan 2026). The shared terminology can obscure the fact that the two methods solve different problems with different reward constructions.

The limitations reported for SmoothVLA are mainly methodological. Accurate jerk estimation depends on reliable derivatives of velocity and acceleration and on Jacobian time derivatives; simulator fidelity and state estimation therefore matter. The method is currently demonstrated with end-effector kinematics, and extension to direct torque control or contact-rich manipulation may require explicit modeling of contact dynamics and actuator constraints. The paper also notes that numerical details for key RL hyperparameters such as $o_{\mathrm{prop}}$6, $o_{\mathrm{prop}}$7, and group size $o_{\mathrm{prop}}$8 are not reported, and that broader sensitivity analysis is needed (Li et al., 14 Mar 2026).

Future directions identified in the paper include scaling to real-world platforms and longer-horizon tasks, learning adaptive task-dependent smoothness objectives, and integrating the approach with model-based control or world-model learning. A plausible implication is that SmoothVLA is best understood not as a replacement for perception, memory, or execution-layer smoothing, but as a policy-alignment mechanism that can in principle be combined with those components when physical feasibility must be enforced across the full VLA stack.