GeoMotion: Geometry-Guided Motion Modeling

• GeoMotion is a geometry-centered approach that models temporal variation through explicit spatial structures rather than opaque sequence patterns.
• It is applied across diverse domains such as motion segmentation, camera-motion understanding, motion-language alignment, and wearable sensing, yielding improved accuracy and efficiency.
• The framework emphasizes end-to-end decoding from latent geometric features, avoiding multi-stage pipelines and enabling robust, interpretable motion inference.

Searching arXiv for the named GeoMotion paper and closely related geometry-guided motion work. GeoMotion denotes a geometry-centered treatment of motion in which temporal variation is modeled through explicit spatial structure rather than as an opaque sequence pattern. In current arXiv usage, the term names a specific fully learning-based motion segmentation framework that decodes moving objects from latent 4D geometry (He et al., 25 Feb 2026), while closely related work uses the same underlying principle for camera-motion understanding in VideoLLMs (Feng et al., 13 Mar 2026), motion-language alignment (Ye et al., 12 Jan 2026), setup-agnostic wearable sensing (Chen et al., 21 May 2026), and other domains. Across these settings, motion is represented through camera extrinsics, latent 4D scene priors, codebook geometry, body-surface geometry, optical flow, or geodesic manifolds, depending on the task. This suggests that “GeoMotion” functions less as a single standardized formalism than as a recurrent research program: motion understanding improves when the representation is matched to the geometry that generates the signal.

1. Conceptual scope and recurring formulations

The literature associates GeoMotion with several distinct but structurally related problem settings.

Domain	Representative geometric carrier	Source
Motion segmentation	Latent 4D geometry, flow, camera pose	(He et al., 25 Feb 2026)
Camera-motion understanding	3D extrinsics and constrained primitives	(Feng et al., 13 Mar 2026)
Motion-language modeling	Orthogonal codebooks and embedding geometry	(Ye et al., 12 Jan 2026)
Wearable motion understanding	Body-surface placement geometry	(Chen et al., 21 May 2026)

The common pattern is that motion is not treated as a purely statistical temporal trace. In camera-motion understanding, the relevant structure is the change in 3D extrinsics—rotation and translation—over fixed within-shot clips (Feng et al., 13 Mar 2026). In motion segmentation, the decisive signal is a fused latent representation of 4D geometry, optical flow, and camera pose features, decoded directly by self-attention without explicit correspondence estimation (He et al., 25 Feb 2026). In motion-language systems, the geometry lies in the organization of discrete motion codes and the embedding spaces into which they are projected (Ye et al., 12 Jan 2026). In wearable sensing, geometry appears in dense body-surface placements, local sensor frames derived from surface normals and tangents, and graph-structured body topology (Chen et al., 21 May 2026).

A second recurrent property is a refusal of brittle intermediate pipelines. GeoMotion for segmentation explicitly criticizes multi-stage systems that estimate optical flow, trajectories, camera pose, and motion masks iteratively, and instead replaces them with end-to-end feed-forward decoding from latent features (He et al., 25 Feb 2026). The camera-motion work for VideoLLMs similarly argues that failures are not merely linguistic but geometric, and therefore diagnoses internal representations before injecting external motion cues back into prompting (Feng et al., 13 Mar 2026). In both cases, geometry is used as a compact intermediate representation that is structured enough to preserve interpretability and physical consistency.

2. Camera motion as a constraint-aware geometric signal

A particularly explicit GeoMotion formulation appears in camera-motion understanding for VideoLLMs, where camera motion is defined as a geometry-grounded, temporally structured signal rather than as a vague visual style cue (Feng et al., 13 Mar 2026). The work organizes its contribution as a three-part framework of benchmarking, diagnosis, and injection.

The benchmarking component introduces CameraMotionDataset and CameraMotionVQA, built from synthetic videos with known camera parameters and segmented into fixed 1-second within-shot clips. The motion taxonomy instantiates a compact set of $K=15$ atomic motion primitives: pan left, pan right, tilt up, tilt down, roll cw, roll ccw, truck left, truck right, crane up, crane down, dolly in, dolly out, arc cw, arc ccw, and static (Feng et al., 13 Mar 2026). Because primitives can co-occur across axes, the task is formulated as constrained multi-label recognition with a symmetric incompatibility matrix

$\mathbf{M} \in \{0,1\}^{K \times K},$

where opposing directions on the same axis are mutually exclusive and static is incompatible with any non-static primitive. Canonicalization removes duplicates, discards invalid label sets, sorts labels in a fixed global order, and restricts each segment to at least 1 and at most 3 primitives.

The annotation pipeline maps camera extrinsics

$$\{ \mathbf{E}_t \}_{t=1}^{T}, \quad \mathbf{E}_t = [\mathbf{R}_t \mid \mathbf{t}_t] \in \mathbb{R}^{3 \times 4},$$

with $T=15$ frames per segment, into labels using net translation in the initial camera frame and accumulated inter-frame rotation (Feng et al., 13 Mar 2026). Relative rotation is computed as $$\mathbf{R}_{\Delta,t}=\mathbf{R}_{t}\mathbf{R}_{t-1}^\top$$, converted to axis-angle form, and accumulated into signed yaw, pitch, and roll changes. For translation-dominant clips, net camera-frame displacement is computed as $$\Delta \mathbf{t}_{\text{cam}}=\mathbf{R}_1^\top(\mathbf{t}_T-\mathbf{t}_1)$$. Curvature is quantified by

$$\kappa=\frac{\sum_t \|\mathbf{f}_{t}-\mathbf{f}_{t-1}\|_2}{\sum_t \|\mathbf{t}_{t}-\mathbf{t}_{t-1}\|_2+\epsilon},$$

and if $\kappa>9{\times}10^{-4}$, the segment is labeled as arc cw or arc ccw depending on pan sign. Validation on 720 sampled segments reached 93% agreement. The full automatic conversion yielded 542,504 1-second segments, then rebalancing capped each unique canonical label set at 200 samples, producing 12,274 segments (Feng et al., 13 Mar 2026).

Evaluation is standardized through 4-way multiple-choice VQA. Distractors are drawn from the same pool of incompatibility-valid canonical label sets and stratified by label cardinality, preventing shortcuts based on invalidity or cardinality mismatch. Diagnosis then probes frozen VideoLLM vision features, especially Qwen2.5-VL, and finds that camera-motion cues are only partially recoverable. Supplementary results report Macro-F1 / Weighted-F1 = 0.87 / 0.92 for a VGGT classifier, 0.83 / 0.87 for VGGT--Q-Former, and 0.69 / 0.74 for Q-Former probing; static is consistently the hardest class, while dolly out is over-predicted as a default positive class (Feng et al., 13 Mar 2026).

The injection stage avoids retraining the VideoLLM. A 3D motion backbone, instantiated with VGGT in the reported setup, produces per-second canonicalized motion labels such as static or pan left + tilt up, and these are prepended to a filmmaker-style instruction through structured prompting. The reported effect is improved directional correctness, terminological consistency, temporal grounding, and reduced hallucinated motion. In GeoMotion terms, the key result is that explicit geometric motion cues can be exposed to a LLM in a compact temporal format without modifying model weights (Feng et al., 13 Mar 2026).

3. Latent 4D geometry and geometric inference

The paper “GeoMotion: Rethinking Motion Segmentation via Latent 4D Geometry” defines GeoMotion narrowly as a motion segmentation framework for dynamic scenes that predicts moving-object masks by decoding latent 4D geometry rather than by estimating correspondences, camera motion, or iterative refinements explicitly (He et al., 25 Feb 2026). Its central premise is that motion segmentation should be reframed as feed-forward representation learning.

The architecture has two principal modules. A feature aggregation module fuses latent 4D geometry features from a pretrained visual geometry backbone, camera pose features, and optical flow features according to

$$\mathbf{F}_{\rm fuse} = \mathrm{MLP}([\mathbf{F}_{\rm geo}; \mathbf{F}_{\rm flow}; \mathbf{F}_{\rm cam}]).$$

The appendix specifies the use of DINOv2 for per-image features, alternating-attention layers from VGGT and $\pi^3$ for geometry features, RAFT for optical flow, and a final fused representation of shape $\mathbf{M} \in \{0,1\}^{K \times K},$0 (He et al., 25 Feb 2026). A lightweight motion decoder with 5 self-attention layers and an MLP head predicts a per-pixel motion mask $\mathbf{M} \in \{0,1\}^{K \times K},$1.

Training uses binary motion masks with a combined focal and Dice loss,

$\mathbf{M} \in \{0,1\}^{K \times K},$2

with $\mathbf{M} \in \{0,1\}^{K \times K},$3. The pretrained geometry backbone is frozen; training runs for 15 epochs with Adam, learning rate $\mathbf{M} \in \{0,1\}^{K \times K},$4, 4 NVIDIA RTX 5090 GPUs, batch size 16 frames, and input resolution $\mathbf{M} \in \{0,1\}^{K \times K},$5 (He et al., 25 Feb 2026).

The main empirical claim is that eliminating explicit correspondence estimation and iterative refinement yields both accuracy and efficiency. Reported benchmark performance includes 83.9 $\mathbf{M} \in \{0,1\}^{K \times K},$6 on DAVIS2016-M, 84.7 $\mathbf{M} \in \{0,1\}^{K \times K},$7 on DAVIS2016, 81.1 $\mathbf{M} \in \{0,1\}^{K \times K},$8 on DAVIS2017, and 77.3 $\mathbf{M} \in \{0,1\}^{K \times K},$9 on SegTrackV2, with runtime 0.31 s/frame compared with 6.44 s/frame for SegAnyMotion and 8.34 s/frame for RoMo (He et al., 25 Feb 2026). Ablations on DAVIS2017 show incremental benefit from each modality: baseline 67.9, + Cam 74.2, + Flow 74.7, + Shallow 72.4, + Cam + Flow 80.2, and All 81.4.

Related geometric reasoning appears in earlier and adjacent work. VGfM uses camera motion across multiple views to estimate 3D ellipsoidal object geometry and then reasons over object relations with a tri-partite graph whose geometric nodes encode 3D centers and extremal points (Gay et al., 2018). GeoMoE treats two-view correspondence filtering as heterogeneous motion-field modeling: it decomposes the motion field into sub-fields, routes them through a Mixture-of-Experts rectifier, and reports 34.21 / 56.25 / 73.53 AUC at $$\{ \mathbf{E}_t \}_{t=1}^{T}, \quad \mathbf{E}_t = [\mathbf{R}_t \mid \mathbf{t}_t] \in \mathbb{R}^{3 \times 4},$$0 on YFCC100M with weighted eight-point estimation (Le et al., 1 Aug 2025). These formulations differ in task, but all treat geometry not as auxiliary metadata but as the organizing substrate for motion inference.

4. Motion tokens, embedding geometry, and LLMs

One major GeoMotion strand concerns the geometry of discrete motion representations used by LLMs. GeoMotionGPT argues that motion understanding improves when the discrete motion tokenizer and the LLM embedding space share a unified geometric basis, specifically orthogonality (Ye et al., 12 Jan 2026). Instead of freezing a quantizer and mapping token IDs into the LLM through a free dense adapter, it imposes orthogonality on both the motion codebook

$$\{ \mathbf{E}_t \}_{t=1}^{T}, \quad \mathbf{E}_t = [\mathbf{R}_t \mid \mathbf{t}_t] \in \mathbb{R}^{3 \times 4},$$1

and the motion-token embeddings inside the LLM. The tokenizer is a decoder-only quantizer with Gumbel-Softmax and utilization regularization,

$$\{ \mathbf{E}_t \}_{t=1}^{T}, \quad \mathbf{E}_t = [\mathbf{R}_t \mid \mathbf{t}_t] \in \mathbb{R}^{3 \times 4},$$2

followed by a sparse projection that preserves orthogonality when motion codes enter the LLM (Ye et al., 12 Jan 2026). On HumanML3D, the strongest reported model—GPT-2 full fine-tuning—achieves 53.48 average versus 43.71 for MotionGPT3, a reported 22.4% improvement over the strongest baseline.

MoGeFlow addresses a complementary question: whether motion-token codebooks should be modeled as categorical vocabularies at all. It reports that PartVQ codebooks exhibit measurable, non-random, and decoder-causal geometry (Fang et al., 10 Jun 2026). Distances between code embeddings and distances between local motion prototypes show strong positive Spearman alignment across six groups—root 0.959, upper arms 0.837, right leg 0.761, upper neck 0.881, left leg 0.692, head 0.797—with mean correlation $$\{ \mathbf{E}_t \}_{t=1}^{T}, \quad \mathbf{E}_t = [\mathbf{R}_t \mid \mathbf{t}_t] \in \mathbb{R}^{3 \times 4},$$3, whereas a shuffled control collapses to $$\{ \mathbf{E}_t \}_{t=1}^{T}, \quad \mathbf{E}_t = [\mathbf{R}_t \mid \mathbf{t}_t] \in \mathbb{R}^{3 \times 4},$$4 (Fang et al., 10 Jun 2026). MoGeFlow therefore generates in continuous codebook space through conditional flow matching and projects terminal states back to valid codes only at the end. Reported results include R-Precision Top-1/Top-2/Top-3 = 0.592/0.783/0.873, FID 0.058, and MultiModal Distance 2.599 on HumanML3D, together with the best reported MotionMillion R@1, R@2, R@3, and FID 28.1 under the benchmark protocol (Fang et al., 10 Jun 2026).

A third variant appears in AnyMo, which moves GeoMotion into wearable sensing (Chen et al., 21 May 2026). Here the central problem is setup dependence: body location, mounting position, orientation, hardware, and sampling protocol all alter the IMU signal. AnyMo addresses this by physics-grounded simulation over dense body-surface placements, pretraining an ST-GCN body graph encoder with paired synthetic placement views and masked partial observations, tokenizing multi-position IMU into full-body motion tokens, and aligning those tokens with Qwen2.5-0.5B. The simulation uses surface normals, anatomical tangents, and local sensor frames; the pretraining objective is a masked cross-view predictive InfoNCE loss; and the tokenizer is a product-quantized VAE (Chen et al., 21 May 2026). Across 14 unseen downstream datasets, reported zero-shot gains are +11.7% Accuracy, +11.6% macro-F1, and +22.6% Recall@2 on HAR, with additional zero-shot retrieval improvements of 15.9% IMU-to-text MRR and 28.6% text-to-IMU MRR, and 18.8% higher zero-shot captioning BERT-F1 (Chen et al., 21 May 2026).

Taken together, these systems shift the GeoMotion question from “how to tokenize motion” to “what geometry the tokenization should preserve.” One line enforces a shared orthogonal basis (Ye et al., 12 Jan 2026); another generates directly through codebook geometry (Fang et al., 10 Jun 2026); a third builds tokens from simulated sensor geometry and body topology (Chen et al., 21 May 2026).

5. Geometry-guided generation, transfer, and controllable motion

GeoMotion also appears in generative settings where the aim is not only to recognize motion but to synthesize or transfer it under spatial and semantic constraints. SemGeoMo treats dynamic contextual human motion generation as a problem of joint semantic and geometric guidance (Cong et al., 3 Mar 2025). The input dynamic environment is a sequential point cloud $$\{ \mathbf{E}_t \}_{t=1}^{T}, \quad \mathbf{E}_t = [\mathbf{R}_t \mid \mathbf{t}_t] \in \mathbb{R}^{3 \times 4},$$5 with $$\{ \mathbf{E}_t \}_{t=1}^{T}, \quad \mathbf{E}_t = [\mathbf{R}_t \mid \mathbf{t}_t] \in \mathbb{R}^{3 \times 4},$$6 points per frame, encoded through Basis Point Set features $$\{ \mathbf{E}_t \}_{t=1}^{T}, \quad \mathbf{E}_t = [\mathbf{R}_t \mid \mathbf{t}_t] \in \mathbb{R}^{3 \times 4},$$7. The pipeline consists of an LLM Annotator based on a pre-trained LLaMA model with LoRA finetuning, a first-stage conditional diffusion model with a dual-branch transformer that predicts hand/joint positions $$\{ \mathbf{E}_t \}_{t=1}^{T}, \quad \mathbf{E}_t = [\mathbf{R}_t \mid \mathbf{t}_t] \in \mathbb{R}^{3 \times 4},$$8 and affordance maps $$\{ \mathbf{E}_t \}_{t=1}^{T}, \quad \mathbf{E}_t = [\mathbf{R}_t \mid \mathbf{t}_t] \in \mathbb{R}^{3 \times 4},$$9, and a second-stage Motion ControlNet on a frozen MDM backbone (Cong et al., 3 Mar 2025). On FullBodyManipulation, reported best numbers with ground-truth text include HandJPE 27.84, MPJPE 16.62, F1 0.77, FID 1.17, R-score 0.66, and FS 0.57.

MotionGrounder addresses multi-object motion transfer in video generation through two inference-time modules: Flow-based Motion Signal (FMS) and Object-Caption Alignment Loss (OCAL) (Teodoro et al., 1 Apr 2026). FMS extracts optical-flow-based patch trajectories from the source video using GMFlow, while OCAL aligns object-specific text attention with object masks across the generated video. The paper introduces Object Grounding Score (OGS) as the product of IoU-based localization and local CLIP similarity, averaged across objects and frames (Teodoro et al., 1 Apr 2026). On the overall “All” setting of its 52-video benchmark, MotionGrounder reports MF 0.6813, IoU 0.2371, LTA 0.2677, and OGS 0.0671, and a human study with 49 participants gives the best average ranks for Motion Adherence (2.99), Global Textual Faithfulness (2.97), and Object Grounding (2.83).

A camera-control analogue appears in GimbalDiffusion, which grounds generated camera motion in a gravity-aligned absolute coordinate system rather than a relative frame-to-frame representation (Fortier-Chouinard et al., 9 Dec 2025). The core absolute pose is

$T=15$0

so that pitch and roll are defined absolutely while yaw remains unconstrained. Training data come from WEB360 panoramic videos; field of view is sampled between 35° and 100°; and null-pitch conditioning is applied to 50% of samples to reduce conflicts between text content and requested camera pose (Fortier-Chouinard et al., 9 Dec 2025). On SpatialVID-extreme, the method reports PitchErr 23.79, GravityErr 27.06, RotErr 14.25, TransErr 0.75, CLIP 21.35, FID 110.71, and FVD 896.84, with an ablation showing improved pitch and gravity accuracy when null-pitch conditioning is retained.

An earlier scene-motion generation system, GAMMA, is relevant as a geometry-aware long-horizon precursor (Zhang et al., 2021). GAMMA represents human motion through the “SSM2 67” marker set, decomposes motion into 0.25 seconds primitives in a pelvis-centered canonical coordinate system, predicts future markers with a CVAE, regresses them back to SMPL-X, and uses policy-guided latent exploration with PPO and tree search for goal-directed rollout. The method is explicitly designed for realistic, controllable, and infinitely long motions with plausible body-scene contact (Zhang et al., 2021).

6. Specialized GeoMotion formulations in magnification, compression, and inverse problems

A further set of papers applies GeoMotion to settings where the main challenge is preserving geometric consistency under extremely weak motion signals or unusual imaging manifolds. In GeoMag, video motion magnification is recast as geometry-aware amplification with State Space Models, specifically Mamba-style selective scanning, to combine global context with linear complexity (Han et al., 28 May 2026). The model separates a motion stream from a static-detail stream, amplifies latent motion differences through

$T=15$1

and reconstructs the output frame with an SFR branch and decoder (Han et al., 28 May 2026). The accompanying Geo-200K dataset synthesizes scenes with 30% pure translation, 30% pure rotation, and 40% combined transformation, plus sensor-realistic degradations. On synthetic benchmarks, GeoMag trained on Geo-200K reports, for example, RMSE 23.61, PSNR 21.35, and LPIPS 0.16 on Synthetic-II, while runtime for a 300-frame 720p video is 13.9 s, compared with 31.6 s for STB-VMM and 50.0 s for EulerMormer (Han et al., 28 May 2026).

GeoDiffMM addresses a closely related motion-magnification problem but uses a diffusion-based Lagrangian pipeline conditioned on optical flow as a geometric cue (Liu et al., 9 Dec 2025). Its three stages are Noise-free Optical Flow Augmentation, a Diffusion Motion Magnifier controlled by a learnable magnification-factor encoding, and Flow-based Video Synthesis. On real-world evaluation it reports MANIQA 0.7129 on average; for flow generation, Motion Error is 3.087 for its DMM compared with 7.224 for MoMo and 23.439 for LFDM (Liu et al., 9 Dec 2025). The method’s central claim is that amplifying motion in flow space suppresses content-irrelevant perturbations more effectively than Eulerian intensity-based amplification.

In spherical video compression, GeoMotion takes the form of geodesic motion modeling on the sphere. “Geometry-Corrected Geodesic Motion Modeling with Per-Frame Camera Motion for 360-Degree Video Compression” improves earlier spherical translational models by representing source and destination positions on a consistent cylindrical manifold (Regensky et al., 2023). The corrected update is

$T=15$2

with global scaling $T=15$3 recommended over local scaling (Regensky et al., 2023). Integrated into VTM-17.2, the best reported setting—GED+gcg with GMA and per-frame camera motion coding—achieves -2.27% average BD-rate savings over VTM-17.2, outperforming the prior state of the art by 0.32 percentage points while reducing per-block complexity from $T=15$4 to $T=15$5.

At the inverse-problem end of the spectrum, “Motion-Enabled Tomography via Gaussian Mixture Models” treats motion itself as the source of view diversity (Burrows et al., 18 May 2026). Each Gaussian component is parameterized by geometry, projectile translation $T=15$6, and linear-in-time angular velocity $T=15$7, and because Gaussians admit closed-form ray transforms, both forward predictions and exact gradients are available (Burrows et al., 18 May 2026). The reconstruction is split into trajectory recovery via a Hausdorff-distance objective over sinogram modes and morphology/rotation recovery via a Huber loss on the full projections. The reported validation is a simulated 2D problem with 5 Gaussians and intersecting trajectories.

Across these specialized cases, the same structural thesis recurs. Motion is not merely a disturbance to be filtered away. It is a geometric carrier: of subtle deformation in magnification (Han et al., 28 May 2026, Liu et al., 9 Dec 2025), of spherical displacement in omnidirectional coding (Regensky et al., 2023), or of angular coverage in tomography (Burrows et al., 18 May 2026). A plausible implication is that GeoMotion research is most coherent when read not as a single benchmark family, but as a shared insistence that the correct motion representation is the one induced by the scene geometry, sensing geometry, or decoder geometry of the task at hand.