LiteVLA-Edge: Compact VLA for Embedded Robots

• The paper demonstrates a deployment-oriented VLA system that uses FP32 supervised fine-tuning combined with 4-bit GGUF quantization for on-device inference.
• LiteVLA-Edge employs a 42-layer multimodal transformer with LoRA adapters to fuse visual and language inputs into discrete actions for ROS 2 control loops.
• It achieves an end-to-end latency of approximately 150 ms on Jetson Orin hardware, establishing a reproducible baseline for edge robotics applications.

LiteVLA-Edge is a deployment-oriented Vision-Language-Action (VLA) pipeline for fully on-device inference on Jetson Orin-class hardware. It combines supervised image-to-action fine-tuning in FP32 with post-training 4-bit GGUF quantization and GPU-accelerated inference through the llama.cpp runtime, and it is embedded in a ROS 2 perception–reasoning–action stack that operates entirely offline (Williams et al., 3 Mar 2026). Within the Lite VLA line of work, it extends an earlier CPU-bound edge-robotics program that demonstrated compact Vision-LLMs (VLMs) for concurrent reasoning and mobility on a TurtleBot 4 under strict computational constraints (Williams et al., 7 Nov 2025).

1. Conceptual lineage and problem setting

LiteVLA-Edge belongs to a line of research motivated by the need to deploy AI models at the edge for autonomous robots operating in GPS-denied environments, where local, resource-efficient reasoning is essential. The earlier Lite VLA system was framed around simultaneous movement and reasoning in dynamic environments using only on-board hardware, rather than a separation between perception and mobility, and was presented as one of the first successful deployments of small VLMs for concurrent reasoning and mobility at the edge (Williams et al., 7 Nov 2025).

The LiteVLA-Edge paper narrows this agenda to a practical systems problem: executing compact multimodal control models locally on embedded hardware while preserving modular interfaces between perception, reasoning, and actuation. It explicitly does not introduce a new policy objective; its stated contribution is a practical systems path for fully on-device inference and a reproducible baseline for future task-level evaluation of on-device VLAs in robotics (Williams et al., 3 Mar 2026).

A related but distinct line is EdgeVLA, which targeted efficient Vision-Language-Action models by eliminating the autoregressive requirement for end-effector position prediction and by leveraging Small LLMs (SLMs). That work emphasized a non-causal “one-shot” end-effector predictor and reported substantial inference gains relative to OpenVLA (Budzianowski et al., 18 Jul 2025). LiteVLA-Edge differs in emphasis: it is centered on deployment configuration, quantized inference, and ROS 2 integration rather than a change in policy head formulation (Williams et al., 3 Mar 2026).

2. Multimodal model and action interface

The LiteVLA-Edge model is a compact multimodal transformer distilled from SmolVLM with approximately 256 M parameters and 42 layers. Its vision encoder patch-embeds $H \times W \times 3$ RGB frames into a sequence of visual tokens. Its language encoder uses standard tokenization plus positional embeddings of the goal instruction $g$. Multimodal fusion is implemented as concatenation of visual and language tokens, followed by 42 transformer layers consisting of multi-head self-attention and MLPs. The decoder head performs next-token prediction over a discrete action vocabulary, and generated tokens $a_1 \ldots a_n$ are mapped via a fixed parser to continuous control commands, specifically linear velocity $v$ and angular velocity $\omega$ (Williams et al., 3 Mar 2026).

Supervised fine-tuning in FP32 uses LoRA adapters injected at each attention and feed-forward layer with rank $r = 8$ and scaling factor $\alpha = 8$. The training objective is negative log-likelihood over ground-truth action sequences,

$$L_{sft} = - \sum_{i=1}^{n} \log P(a_i \mid a_{(<i)}, I_t, g; \theta).$$

Optimization uses AdamW, although the paper does not specify learning rate or batch size. The paper states that full-precision weight updates ensure high-fidelity mapping from pixels+language to actions (Williams et al., 3 Mar 2026).

The earlier Lite VLA system provides a more explicit example of the control interface. There, the model decoded a one-token action string of the form forward_0.2_3.0s, drawn from { forward, backward, turn_left, turn_right, stop } with parameters encoded as _<linear_vel>_<duration>. A string parser extracted linear and angular velocities and duration, populated a geometry_msgs/Twist, and timestamped the end of motion (Williams et al., 7 Nov 2025). This suggests a shared design principle across the Lite VLA line: preserving an explicit parser layer between high-level token generation and low-level motor control.

3. Quantization and numerical compression

LiteVLA-Edge applies post-training quantization to shrink the 256 M-parameter model for on-device deployment. The chosen format is GGUF with 4-bit kernels, specifically Q4_K_M. The quantization procedure is described step by step. Trained FP32 weights $W \in \mathbb{R}^{m \times n}$ are exported to an intermediate format, then each row or column of $W$ is partitioned into contiguous groups of size $g$0 such as $g$1 elements. For each group $g$2, the paper computes

$g$3

then

$g$4

and

$g$5

Each weight is quantized as

$g$6

and stored as a 4-bit integer together with one $g$7 pair per group. Dequantization at inference time reconstructs

$g$8

This is the core numerical compression mechanism used for deployment on Jetson Orin (Williams et al., 3 Mar 2026).

The paper reports that 4-bit quantization yields approximately a $g$9 reduction in model memory and approximately a $a_1 \ldots a_n$0 speed-up over 8-bit, with negligible action drift. It also notes that truncating the context window below 512 tokens would further reduce latency at the cost of shorter reasoning horizons, while LoRA fine-tuning avoids full-model updates and preserves the original VLM weights for multi-task reusability (Williams et al., 3 Mar 2026).

The earlier CPU-bound Lite VLA system used a different quantization regime: NF4 quantization applied to all transformer weights except the final projection head, which remained in FP32 as a hybrid-precision design. That work reported a net effect of approximately 75% reduction in memory footprint versus an FP32 baseline and highlighted instability under full 4-bit quantization, including hallucinated commands approximately 5% of the time (Williams et al., 7 Nov 2025). The contrast is informative: LiteVLA-Edge adopts GGUF Q4_K_M in a GPU-accelerated setting, whereas the earlier platform emphasized hybrid precision for CPU-bound stability.

4. Runtime stack and ROS 2 deployment

LiteVLA-Edge uses llama.cpp v2.x compiled with CUDA support. All 42 transformer layers with int4 kernels are offloaded to the Orin GPU via CUDA. The configuration uses a context window of n_ctx = 512, a maximum of 12 generated tokens per inference, and deterministic greedy decoding with temperature $a_1 \ldots a_n$1 (Williams et al., 3 Mar 2026).

The low-level optimizations reported for the runtime are specific to llama.cpp and its CUDA backend: fused int4 GEMM kernels for Q4_K_M format, double-buffered KV caches in GPU memory to hide memory latency, and asynchronous CUDA streams for overlapping softmax, layernorm, and token-embedding lookups (Williams et al., 3 Mar 2026). These are system-level optimizations rather than algorithmic changes to the control policy.

The end-to-end pipeline is organized as a modular perception $a_1 \ldots a_n$2 reasoning $a_1 \ldots a_n$3 actuation loop under ROS 2. The paper gives the following mean latency breakdown:

Component	Mean latency (ms)
Image capture + resize + norm	15
Tokenization (vision+text)	5
Model inference (int4)	120
Action parsing + ROS 2 publish	10
Total end-to-end	150.5

The VLA node runs asynchronously at approximately 6.6 Hz, while a separate low-level controller enforces a 100 Hz publication heartbeat for safety and smooth motion. The paper further reports model-inference jitter $a_1 \ldots a_n$4 ms, which it presents as sufficient to ensure stable control loops under ROS 2 (Williams et al., 3 Mar 2026).

This asynchronous structure has a clear antecedent in the earlier Lite VLA implementation, where ROS 2 Action Chunking allowed the low-level controller to continue executing the last Twist command at 10 Hz while the VLM processed the current frame. That earlier design was explicitly introduced to mask inference latency and close the loop between high-level reasoning and low-level actuation when model response times were much slower (Williams et al., 7 Nov 2025).

5. Hardware profile and empirical performance

LiteVLA-Edge is reported on an NVIDIA Jetson AGX Orin with 64 GB LPDDR5 unified memory, 2048 CUDA cores, 64 Tensor cores, a GPU clock of approximately 1600 MHz, a 12-core ARM Cortex-A78AE CPU at approximately 2.2 GHz, and a 40 W power mode typical for AGX Orin deployments (Williams et al., 3 Mar 2026).

The benchmark summary is based on 300 runs after warm-up. Reported figures are a mean latency of 150.5 ms, standard deviation of 0.13 ms, reasoning frequency of 6.64 Hz, model size of approximately 128 MB on disk and approximately 200 MB resident in memory, and system power draw of 35–40 W under sustained inference load (Williams et al., 3 Mar 2026). These figures define the paper’s central claim of timing feasibility for reactive language-conditioned control on embedded hardware.

A frequent point of confusion is whether LiteVLA-Edge also reports extensive task-level robotic success metrics. It does not. The paper frames itself as a reproducible baseline for future task-level evaluation and emphasizes latency, runtime, and modular deployment (Williams et al., 3 Mar 2026). By contrast, task-level results were reported in the earlier CPU-bound Lite VLA study. That predecessor used a TurtleBot 4 for indoor navigation in office corridors and trained on 15,083 RGB–action pairs collected via teleoperation, with 13,000 used for training and an 85%/15% train/validation split. On the held-out 15% validation set, it reported correct move prediction accuracy of approximately 92.3% for the FP32 model and approximately 90.1% for the hybrid-quantized model. In 50 runs of a 5 m corridor traversal, the hybrid-quantized LiteVLA achieved an 86% success rate, compared with 94% for the FP32 baseline (Williams et al., 7 Nov 2025).

Those earlier results also quantified the trade-off introduced by compression: quantization reduced memory by 75% and improved latency up to $a_1 \ldots a_n$5, at the cost of an approximately 2% drop in prediction accuracy (Williams et al., 7 Nov 2025). In that sense, LiteVLA-Edge should be read as shifting the edge-VLA discussion from bare feasibility under CPU-only constraints to substantially lower end-to-end latency on embedded GPU hardware.

6. Reproducibility, deployment heuristics, and significance

LiteVLA-Edge includes explicit reproducibility provisions. The paper states that code and conversion scripts for FP32-to-GGUF Q4_K_M are packaged in the authors’ GitHub, and example ROS 2 launch files configure camera topics, llama.cpp parameters such as --n_ctx 512, --n_predict 12, --threads, and --gpu-layers=all, as well as power modes. A documented warm-up procedure of 50 dummy inferences is used to ensure consistent latency measurements (Williams et al., 3 Mar 2026).

The paper also outlines future directions: agentic extensions with multi-step planning, natural-language failure explanations, and retry logic; swarm robotics in bandwidth-constrained settings using the low power footprint of Q4 inference; and task-level benchmarking on grasp and assembly tasks under dynamic object shifts to evaluate closed-loop performance beyond latency (Williams et al., 3 Mar 2026).

The earlier Lite VLA study provides complementary deployment guidelines that remain relevant for edge multimodal control. It recommends choosing a model no larger than 300 M parameters when targeting less than 1 GB RAM, preserving the final projection head in FP32 if sub-second stability is required, using LoRA or QLoRA for task-specific fine-tuning rather than full-model updates, exploiting operator fusion and vector instructions such as NEON on ARM, and employing an asynchronous action-chunking or buffer mechanism so that low-level controllers never stall waiting for the model (Williams et al., 7 Nov 2025). These recommendations were derived from CPU-bound experiments, but they suggest a broader engineering doctrine for small VLAs at the edge.

Taken together, LiteVLA-Edge and its immediate predecessors delineate a specific subfield of edge robotics: compact multimodal transformers, aggressive quantization, explicit parser layers, and ROS 2 modularity for local perception–reasoning–action loops. The earlier work positioned this trajectory as a foundation for scalable, assured autonomy in service robotics, disaster response, and defense operations (Williams et al., 7 Nov 2025), while LiteVLA-Edge refines the same trajectory into a deployment-oriented, offline, Jetson-based baseline with quantified end-to-end timing performance (Williams et al., 3 Mar 2026).