GemDepth: Geometry-Embedded Video Depth

• GemDepth is a polysemous concept that denotes a geometry-embedded video depth framework, a width–depth-aware scaling law in transformer analysis, and a measure of meteor penetration in atmospheric studies.
• In video depth estimation, GemDepth employs a Geometry-Embedding Module (GEM) and an Alternating Spatio-Temporal Transformer (ASTT) to integrate explicit camera motion and global scale into a multi-scale feature hierarchy.
• Its effectiveness is demonstrated by robust zero-shot performance across benchmarks using metrics like AbsRel and Temporal Alignment Error, while emphasizing efficient, geometrically conditioned learning.

GemDepth most directly denotes the 2026 video-depth framework “GemDepth: Geometry-Embedded Features for 3D-Consistent Video Depth,” which injects explicit camera motion and global scale into a feature hierarchy through a Geometry-Embedding Module (GEM) and an Alternating Spatio-Temporal Transformer (ASTT) (Cheng et al., 11 May 2026). The same label also appears in adjacent literatures with different meanings: in Gemstones it names the width–depth-aware scaling-law facet that studies how transformer depth interacts with width, parameter count, and training tokens (McLeish et al., 7 Feb 2025); in monocular driving depth, “GemDepth” is best understood as referring to GEDepth or to the Ground Embedding Module for depth, since there is no separate algorithm named “GemDepth” in that paper (Yang et al., 2023); and in meteor physics it denotes the terminal altitude reached by Geminid meteors at the end of luminous flight (Spurný et al., 18 Feb 2026).

1. Terminological scope and disambiguation

The term has therefore become polysemous across computer vision, scaling-law analysis, and astronomy. In computer vision, the exact title “GemDepth” refers to video depth estimation with explicit geometric conditioning (Cheng et al., 11 May 2026). In large-language-model scaling, GemDepth is a component of the Gemstones model suite rather than a standalone model (McLeish et al., 7 Feb 2025). In autonomous-driving monocular depth, the closest matching term is GEDepth: Ground Embedding for Monocular Depth Estimation; the accompanying clarification states that if “GemDepth” appears in that context, it is best understood as GEDepth or as the Ground Embedding Module, not as a separate method (Yang et al., 2023). In meteor studies, GemDepth is a descriptive label for how deeply Geminid meteors penetrate into the atmosphere (Spurný et al., 18 Feb 2026).

Usage of “GemDepth”	Domain	Defining idea
GemDepth	Video depth estimation	Geometry-embedded features for 3D-consistent video depth
GemDepth	Scaling laws	Width–depth-aware analysis within Gemstones
“GemDepth” as GEDepth	Monocular depth estimation	Ground embedding from camera parameters
GemDepth	Meteor physics	Terminal altitude of Geminid meteors

This distribution of meanings suggests that the term functions less as a single established concept than as a recurring shorthand for depth problems in which geometry, model depth, or penetration depth is made explicit.

2. GemDepth as geometry-embedded video depth

In its primary contemporary usage, GemDepth addresses video depth estimation, where the target is a temporally coherent sequence of per-frame depth maps that preserves spatial accuracy while enforcing strict 3D geometric consistency across time (Cheng et al., 11 May 2026). The motivating claim is that existing methods frequently rely on temporal smoothing over 2D frame features and therefore struggle under camera rotations and drastic view changes. GemDepth responds by making camera motion a first-class conditioning signal rather than an implicit latent variable.

The pipeline begins with a frozen DINOv2 ViT encoder operating on non-overlapping patches. Given video input $$X \in \mathbb{R}^{B \times N \times C \times H \times W}$$, time is collapsed into the batch dimension, and multi-scale features $F_j$ are extracted from layers $j \in \{5,12,18,24\}$. GEM then injects a learnable camera token into the highest-level feature map and uses a lightweight EfficientPoseNet with a 4-layer alternating-attention transformer to predict inter-frame motion and a global scale. The pose head outputs a quaternion $Q$, translation $\tilde{T}$, and scale $Z$, which are encoded by geometric MLPs into a camera feature $F_{\text{cam}}$. This feature is fused with visual features to produce metric-aware enhanced representations.

ASTT then alternates between two regimes. In the temporal phase, tokens are reorganized so that identical spatial positions across frames can be grouped, and GEM-guided motion priors are used to aggregate features along trajectories and establish latent point-level correspondences. In the spatial phase, intra-frame spatial attention and inter-frame spatial attention refine local layout and long-range cross-frame structure, improving sharpness in high-frequency regions. A DPT decoder converts the refined multi-scale features into per-frame depth maps.

The central geometric equations are explicit. For a pixel $\tilde{u} = [u,v,1]^\top$ at time $t$ with depth $D_t(u)$ and intrinsics $F_j$0,

$F_j$1

Under the inter-frame pose $F_j$2,

$F_j$3

and reprojection proceeds as

$F_j$4

These relations are used both in the architecture’s geometric conditioning and in temporal evaluation metrics.

3. Geometry embeddings, losses, and training regime

GEM is supervised directly on scale-normalized pose targets in a canonical frame (Cheng et al., 11 May 2026). Rotation is represented by a quaternion, translation is normalized by the learned global scale, and the camera loss is a Huber-penalized average of rotation and translation errors:

$F_j$5

Depth supervision combines three terms: the scale- and shift-invariant loss $F_j$6, the multi-scale gradient matching loss $F_j$7, and the temporal geometric consistency loss $F_j$8. The total objective is

$F_j$9

with $j \in \{5,12,18,24\}$0, $j \in \{5,12,18,24\}$1, and $j \in \{5,12,18,24\}$2.

A notable design choice is the absence of photometric consistency loss in training. Temporal consistency is instead enforced through $j \in \{5,12,18,24\}$3 and geometry-aware conditioning. The paper also states that a commonly referenced edge-aware smoothness term is not part of GemDepth; sharpness is attributed to ASTT’s spatial refinement and gradient matching.

Training is explicitly data-efficient and proceeds in two stages. Stage 1, termed Geometric Optimization, jointly supervises GEM and depth on pose-annotated datasets: Virtual KITTI 2, TartanAir, PointOdyssey, MVS-Synth, and Dynamic Replica, totaling approximately 690K frames. Stage 2, termed Depth Refinement, freezes GEM and fine-tunes ASTT and the decoder on approximately 250K pose-free frames from IRS and diverse in-the-wild sequences. Multi-resolution training uses a base shorter edge of 518, random center cropping, and clip length $j \in \{5,12,18,24\}$4. Optimization uses AdamW with discriminative learning rates: $j \in \{5,12,18,24\}$5 for ASTT and GEM, and $j \in \{5,12,18,24\}$6 for pre-trained components, with a global batch size of 16 on 16 NVIDIA A800 GPUs. Each stage converges in approximately 3 days.

This training protocol suggests a division of labor: Stage 1 teaches the system explicit geometric reasoning, while Stage 2 makes the temporal module resilient to imperfect motion priors.

4. Empirical performance, ablations, and efficiency

GemDepth is evaluated zero-shot on Sintel, KITTI, ScanNet, and Bonn with affine-invariant alignment, using spatial and temporal metrics including Abs Rel, $j \in \{5,12,18,24\}$7, Temporal Alignment Error (TAE), Temporal Chamfer Distance (TCD), F1 score of reconstructed point clouds, and ATE (Cheng et al., 11 May 2026). The reported zero-shot depth results are strong across all four benchmarks. GemDepth-VDA achieves AbsRel/$j \in \{5,12,18,24\}$8 of $j \in \{5,12,18,24\}$9 on Sintel, $Q$0 on Bonn, $Q$1 on ScanNet, and $Q$2 on KITTI. GemDepth-DAv2 also improves substantially over VDA, while remaining weaker than GemDepth-VDA on the same benchmarks.

Temporal consistency is likewise improved. On ScanNet TAE over the first 20 sequences of 110 frames each, DepthAnything V2 records $Q$3, RollingDepth $Q$4, VDA $Q$5, GemDepth-DAv2 $Q$6, and GemDepth-VDA $Q$7. For 3D geometric accuracy with DA3/VGGT under 32 consecutive views, GemDepth reports $Q$8B parameters, compared with $Q$9B for VGGT and $\tilde{T}$0B for DA3; on ScanNet it reaches ATE $\tilde{T}$1, F1 $\tilde{T}$2, AbsRel $\tilde{T}$3, and TAE $\tilde{T}$4.

Ablation results attribute gains to both ASTT and GEM. Relative to the VDA baseline, adding spatial attention lowers AbsRel on KITTI/Sintel from $\tilde{T}$5 to $\tilde{T}$6, while adding temporal attention yields $\tilde{T}$7 and reduces ScanNet TAE from $\tilde{T}$8 to $\tilde{T}$9. ASTT with both branches further improves to $Z$0 and TAE $Z$1, and the full system with GEM reaches $Z$2 and TAE $Z$3. Pose-noise experiments show graceful degradation: on ScanNet, AbsRel/TAE move from $Z$4 at $Z$5 injected noise to $Z$6 at $Z$7, $Z$8 at $Z$9, and $F_{\text{cam}}$0 at $F_{\text{cam}}$1.

Placement of ASTT is important. Early insertion immediately after feature extraction is best: Position 1 yields KITTI AbsRel $F_{\text{cam}}$2, versus $F_{\text{cam}}$3 and $F_{\text{cam}}$4 for mid and late placement; on Sintel the corresponding values are $F_{\text{cam}}$5, $F_{\text{cam}}$6, and $F_{\text{cam}}$7; on ScanNet TAE they are $F_{\text{cam}}$8, $F_{\text{cam}}$9, and $\tilde{u} = [u,v,1]^\top$0.

The efficiency profile is comparatively modest. Relative to VDA, ASTT adds about 10 ms and GEM about 5 ms, reaching 99 ms total at $\tilde{u} = [u,v,1]^\top$1 on a single NVIDIA A800. FLOPs increase by approximately 40G, and the paper states this is only 12.6% of the FLOPs of large generative methods like RollingDepth. No post-processing is required beyond the standard affine alignment used for evaluation.

5. Ground-plane and camera-parameter embeddings in monocular depth

In monocular depth estimation for autonomous driving, the closest precursor to the name is GEDepth, whose clarification explicitly states that there is no separate algorithm named “GemDepth” in that paper (Yang et al., 2023). GEDepth addresses the ill-posedness of monocular depth by decoupling camera parameters from pictorial cues. Given camera intrinsics $\tilde{u} = [u,v,1]^\top$2, extrinsics $\tilde{u} = [u,v,1]^\top$3, and a nominal planar ground height $\tilde{u} = [u,v,1]^\top$4, its ground embedding module computes a per-pixel ground depth $\tilde{u} = [u,v,1]^\top$5, stacks it with the input image as $\tilde{u} = [u,v,1]^\top$6, and predicts both a residual depth $\tilde{u} = [u,v,1]^\top$7 and a ground attention map $\tilde{u} = [u,v,1]^\top$8. The final depth is

$\tilde{u} = [u,v,1]^\top$9

For a planar ground $t$0, the ray–plane intersection yields

$t$1

with pixels above the vanishing line set to $t$2. An adaptive version introduces a learned ground slope $t$3 and uses

$t$4

The module is described as lightweight, plug-and-play, and integrated with BTS, DepthFormer, PixelFormer, and BinsFormer. On KITTI, for example, DepthFormer improves from AbsRel $t$5 to $t$6 with GE-Vanilla and to $t$7 with GE-Adaptive; in cross-dataset transfer, KITTI$t$8DDAD improves from AbsRel $t$9 to $D_t(u)$0.

GenDepth extends the same general line of attack by explicitly learning equivariance to camera-parameter changes using synthetic diversity and transferring it to real data through adversarial alignment (Koledić et al., 2023). It computes a per-pixel ground-plane intersection depth

$D_t(u)$1

or, under standard driving assumptions, the row-wise closed form

$D_t(u)$2

This ground-plane depth is normalized to $D_t(u)$3, Fourier-encoded as

$D_t(u)$4

with $D_t(u)$5, and fused into content features after adversarial feature alignment. Zero-shot evaluation reports AbsRel $D_t(u)$6 on DDAD, $D_t(u)$7 on Argoverse, $D_t(u)$8 on Waymo, and $D_t(u)$9 on nuScenes. The ablation sequence CARLA-only $F_j$00 CARLA+ground-plane embedding $F_j$01 CKGD $F_j$02 CKGDS shows DDAD AbsRel improving from $F_j$03 to $F_j$04 to $F_j$05 to $F_j$06.

A plausible implication is that the shared conceptual core of these systems is not the name itself but the use of analytic geometry to decouple metric depth from camera-specific appearance priors.

6. Other uses: scaling laws and Geminid penetration depth

Within Gemstones, GemDepth refers to the suite’s facet for width–depth-aware scaling laws rather than to visual depth estimation (McLeish et al., 7 Feb 2025). Its core formulation explicitly introduces width $F_j$07 and depth $F_j$08 into the loss model:

$F_j$09

The experiments span 22 transformer models in parameter bands near 50M, 100M, 500M, 1B, and 2B, with width ranging from 256 to 3072 and depth from 3 to 80. The principal empirical claim is that compute-optimal aspect ratio increases with FLOPs but slowly: as budgets grow, prescriptions become relatively wider and shallower rather than significantly deeper. Time-efficiency favors width over depth in the authors’ tensor-parallel setting; the least efficient shapes overspend approximately 50% FLOPs but can overspend more than 200% GPU hours versus pareto-frontier shapes. The paper emphasizes sensitivity to learning-rate schedule, cooldown, token regime, and checkpoint selection, and recommends avoiding very skinny models and avoiding fits based on narrow width slices or the sub-100B-token regime.

In meteor physics, GemDepth denotes the minimum or terminal altitude $F_j$10 reached by Geminid meteors at the end of luminous flight, used as a proxy for penetration depth and ablation resistance (Spurný et al., 18 Feb 2026). Across the European Fireball Network, Prairie Fireball Network, and MORP data, Geminids with entry mass greater than approximately 10 grams exhibit a terminal-altitude plateau near 38 km. The exceptional event EN131212_041259 reached a terminal height of $F_j$11 km, about 6 km below that plateau, with terminal speed $F_j$12 km s$F_j$13 after entering at $F_j$14 km s$F_j$15. The analysis uses the meteor-physics convention

$F_j$16

with fragmentation beginning near 58 km at approximately 0.4 MPa and a maximum dynamic pressure of about 1.9 MPa near 40 km. The event is interpreted as the result of strong material, early fragmentation into lower-ballistic-coefficient fragments, and rapid deceleration that still allowed a likely surviving terminal mass of about 3.4–3.7 g. In this literature, GemDepth is not an algorithmic module but an observational depth-of-penetration quantity.

Taken together, these usages show that “GemDepth” is not a single canonical term. In current machine-vision usage it is most specifically the geometry-embedded video-depth framework of 2026; in neighboring literatures it names a depth–width scaling facet, a mistaken reference to GEDepth’s ground embedding, or an atmospheric penetration measure for Geminid meteors.