GeoAware-VLA: Implicit Geometry for VLA

• The model integrates a frozen, pretrained geometric backbone to improve zero-shot generalization on novel viewpoints without requiring explicit 3D data.
• GeoAware-VLA uses a lightweight projection layer and a GPT-style policy decoder to adapt multi-view features into robust 3D-aware representations for control.
• The approach significantly enhances success rates on the LIBERO benchmark and transfers gains to real-robot deployments across both continuous and discrete action spaces.

GeoAware-VLA is a Vision-Language-Action model designed to improve generalization to novel camera viewpoints by injecting an implicit geometric prior into the visual pathway rather than requiring the policy to infer 3D consistency from 2D images alone. The model replaces the usual trainable image encoder with a frozen, pretrained geometric backbone and a lightweight trainable projection layer, then feeds the resulting representation to a GPT-style transformer policy decoder. In the reported formulation, this design yields substantial zero-shot gains on held-out camera poses in the LIBERO benchmark and transfers those gains to a real robot, while remaining effective for both continuous and discrete action spaces (Abouzeid et al., 17 Sep 2025).

1. Conceptual basis and problem formulation

The motivating claim is that many Vision-Language-Action models fail to generalize to novel camera viewpoints because they struggle to infer robust 3D geometry from 2D images. GeoAware-VLA addresses this by integrating strong geometric priors into the vision backbone, but without training a new visual encoder and without relying on explicit 3D data (Abouzeid et al., 17 Sep 2025).

A common misconception is that viewpoint robustness in this setting requires explicit 3D reconstruction, depth supervision, or dedicated geometric regularizers. The reported formulation does not do so. Instead, it uses a frozen, pretrained geometric vision model as a feature extractor, and the trainable projection layer adapts those geometrically rich features for the policy decoder. No additional explicit geometric consistency or view-equivariance regularizers are used; the frozen backbone is intended to enforce 3D structure implicitly (Abouzeid et al., 17 Sep 2025).

This design places GeoAware-VLA in the class of implicit geometry-aware VLAs. The geometry prior is not introduced as an explicit point cloud, Gaussian scene representation, or depth map input to the policy. Rather, it is embedded in the latent features of a geometric foundation model and then exposed to the action model through a small learned interface. This suggests that the central design choice is not merely feature reuse, but the reassignment of geometric reasoning from the downstream policy network to a pretrained backbone.

2. Architecture

GeoAware-VLA operates on multi-view RGB observations, proprioceptive state, and a natural-language instruction. The multi-view observation at time $t$ is denoted by

$\mathbf I_t = \{I_t^{(1)}, \dots, I_t^{(V)}\},$

the robot state by $s_t$, and the instruction by $l_t$. The policy is written as

$$a_t = \pi_\phi\bigl(P_\theta(\Phi_{\mathrm{geo}}(\mathbf I_t)),\; E_{\mathrm{lang}}(l_t),\; E_{\mathrm{proprio}}(s_t)\bigr),$$

where $\Phi_{\mathrm{geo}}$ is the frozen geometric backbone, $P_\theta$ is the trainable projection layer, $E_{\mathrm{lang}}$ and $E_{\mathrm{proprio}}$ are lightweight encoders, and $\pi_\phi$ is a GPT-style transformer policy decoder plus an action head (Abouzeid et al., 17 Sep 2025).

The geometric backbone is the Visual Geometry Grounded Transformer (VGGT). In the reported system, VGGT is a 24-layer transformer trained to predict multi-view depth, camera parameters, and dense point clouds. Each image is passed through VGGT to obtain intermediate feature tokens from multiple layers, and the model selects $\mathbf I_t = \{I_t^{(1)}, \dots, I_t^{(V)}\},$0 evenly spaced layers $\mathbf I_t = \{I_t^{(1)}, \dots, I_t^{(V)}\},$1. At layer $\mathbf I_t = \{I_t^{(1)}, \dots, I_t^{(V)}\},$2, the features are

$\mathbf I_t = \{I_t^{(1)}, \dots, I_t^{(V)}\},$3

with $\mathbf I_t = \{I_t^{(1)}, \dots, I_t^{(V)}\},$4 the visual token sequence length. Because VGGT is frozen, these tokens are treated as already encoding rich 3D structure and viewpoint-consistent geometry (Abouzeid et al., 17 Sep 2025).

The projection layer adapts these features to the policy decoder. For each selected layer and each view, GeoAware-VLA applies a small 1D-convolutional encoder followed by adaptive pooling and a final MLP:

$\mathbf I_t = \{I_t^{(1)}, \dots, I_t^{(V)}\},$5

Across $\mathbf I_t = \{I_t^{(1)}, \dots, I_t^{(V)}\},$6 views, this yields $\mathbf I_t = \{I_t^{(1)}, \dots, I_t^{(V)}\},$7, each in $\mathbf I_t = \{I_t^{(1)}, \dots, I_t^{(V)}\},$8. This projection layer is the only trainable component upstream of the policy decoder (Abouzeid et al., 17 Sep 2025).

The policy decoder is a causal, decoder-only transformer that ingests a single token sequence composed of visual tokens from all views, a language token, a proprioceptive token, and a learned action token. The language token is produced by projecting a sentence-BERT embedding of $\mathbf I_t = \{I_t^{(1)}, \dots, I_t^{(V)}\},$9, and the proprioceptive token by a small MLP on $s_t$0. The output at the action token feeds one of two action heads: a 2-layer MLP head for continuous control with $s_t$1, or a VQ-BeT head for discrete, multimodal action modeling that predicts a codebook index together with a small continuous offset (Abouzeid et al., 17 Sep 2025).

3. Training objective and optimization

GeoAware-VLA is trained by behavior cloning on an expert demonstration dataset

$s_t$2

For continuous control, the primary objective is mean-squared error on the action:

$s_t$3

Standard weight decay is added on all trainable parameters:

$s_t$4

No explicit geometric consistency loss is introduced; the intended source of geometry is the frozen VGGT representation itself (Abouzeid et al., 17 Sep 2025).

This training strategy is notable because it keeps the upstream geometric module fixed. The trainable burden is concentrated in the projection layer, the lightweight language and proprioception encoders, and the policy decoder. A plausible implication is that the method aims to preserve the inductive bias of the geometric foundation model while minimizing the risk that end-to-end finetuning erodes viewpoint-consistent structure.

4. Benchmark evaluation and zero-shot viewpoint generalization

Evaluation is conducted on the four LIBERO benchmark subsets: Spatial, Object, Goal, and Long. Policies are trained on the original, in-distribution camera poses and then tested zero-shot on held-out novel viewpoints, including small, medium, and large deviations. The metric is success rate in percent (Abouzeid et al., 17 Sep 2025).

In simulation, the reported gains are large. Averaged across all LIBERO subsets, baseline BAKU achieves approximately $s_t$5 success on novel viewpoints, GeoAware BAKU achieves approximately $s_t$6, and GeoAware VQ-BeT achieves approximately $s_t$7. The reported change corresponds to an absolute improvement of about $s_t$8 percentage points over BAKU and a greater than $s_t$9 increase over the next strongest baseline, OpenVLA-OFT at $l_t$0 (Abouzeid et al., 17 Sep 2025).

These results are used to support the claim that robust geometric grounding is a key component for more generalizable robotic agents. Importantly, the reported effect is not confined to one action parameterization. The system is described as effective across both continuous and discrete action spaces, which addresses a second common misconception: that geometry-aware gains in VLA models are specific to one decoder family or one action head design (Abouzeid et al., 17 Sep 2025).

5. Real-robot deployment and ablation analysis

The real-world experiments deploy GeoAware-VLA on a Realman 65B arm in a tabletop environment. Two fixed cameras, one wrist-mounted and one top-down, provide the two-view input used during training, while a third diagonally mounted viewpoint is used for zero-shot evaluation. Five language-guided pick-and-place tasks of increasing complexity are tested, including nested and multi-object placements and delicate stacking (Abouzeid et al., 17 Sep 2025).

Both continuous and discrete action heads are evaluated. After $l_t$1 K steps of behavior cloning on tele-operated demonstrations, GeoAware BAKU exhibits approximately $l_t$2–$l_t$3 percentage points higher success rates on the unseen view than baseline BAKU, matching the simulation trend. Reported sample efficiency, measured as the number of behavior-cloning steps needed to reach greater than $l_t$4 success on known views, is effectively unchanged, indicating that the frozen VGGT does not introduce a training slowdown (Abouzeid et al., 17 Sep 2025).

The layer-selection ablation isolates which VGGT features matter most. Three projection-layer configurations are compared on LIBERO-Long:

Configuration	Seen success	Novel success
All 24 VGGT layers	90.0%	74.3%
Default (4 evenly spaced)	93.0%	72.7%
Last 4 layers only	60.0%	34.3%

Using all layers yields similar novel-view gains but slightly lower in-distribution performance, whereas the default four-layer design gives the best overall trade-off. The last-four-layers-only variant performs substantially worse, although even that weakest variant still doubles the next-best baseline of $l_t$5 on novel views (Abouzeid et al., 17 Sep 2025).

This pattern suggests that viewpoint robustness is not concentrated only in the deepest backbone activations. Intermediate geometric features appear to be important for the downstream policy, and a sparse multi-layer readout can retain most of the novel-view benefit without sacrificing seen-view performance.

6. Relation to adjacent geometry-aware VLA research and open directions

GeoAware-VLA belongs to a broader line of work that attempts to strengthen spatial grounding in VLA systems, but it does so through an implicit mechanism. Later work explores more explicit geometric representations. GST-VLA converts frozen dense depth and frozen semantic patch features into $l_t$6 anisotropic 3D Gaussian primitives, supervises four structured intermediate spatial thoughts through 3D Depth-Aware Chain-of-Thought, and uses a flow-matching action expert; it reports $l_t$7 on LIBERO and $l_t$8 on SimplerEnv (Sarowar et al., 10 Mar 2026). Lift3D-VLA, by contrast, injects explicit 3D point-cloud tokens into a lifted 2D VLA encoder, uses Geometry-Centric Masked Autoencoding for static structure and future geometry prediction, and reports higher mean success rates on MetaWorld, RLBench, and real-world manipulation tasks (Liu et al., 7 Jul 2026).

Within that landscape, GeoAware-VLA is distinguished by its minimal intervention: a frozen VGGT backbone, a small projection module, and no explicit 3D reconstruction or depth input to the policy (Abouzeid et al., 17 Sep 2025). This makes it conceptually distinct from approaches that introduce structured 3D tokens, point clouds, or additional geometry-specific supervision.

The phrase “GeoAware-VLA” also appears in a different sense in geospatial and Earth-vision contexts. EarthVL proposes a progressive Earth vision-language framework with land-cover segmentation, object-guided attention, VQA, and a numerical difference loss for city-planning applications (Wang et al., 6 Jan 2026). Separately, a UAV-VLA extension described as “GeoAware-VLA” augments a vision-language-action pipeline with satellite imagery metadata, geo-referencing, and coordinate transforms for UAV mission generation, reporting a mean point error of $l_t$9 m under K-Nearest Neighbors matching and a trajectory-length difference of about $$a_t = \pi_\phi\bigl(P_\theta(\Phi_{\mathrm{geo}}(\mathbf I_t)),\; E_{\mathrm{lang}}(l_t),\; E_{\mathrm{proprio}}(s_t)\bigr),$$0 relative to human plans (Sautenkov et al., 9 Jan 2025). This suggests that the term has acquired a broader usage spanning both robotic 3D geometry awareness and geospatial awareness.

The limitations and future directions explicitly identified for GeoAware-VLA are threefold: exploring finetuning of parts of VGGT to adapt its geometry representations to the robotic domain, integrating alternative geometric foundation models such as MUST3R and DUSt3R, and extending beyond tabletop tasks to more dynamic scenes, deformable objects, and mobile robots (Abouzeid et al., 17 Sep 2025). These directions follow directly from the central hypothesis of the model: that robust geometric grounding, even when introduced implicitly, is a primary lever for viewpoint-robust robotic behavior.