UniMove: Unified Motion Modeling

• UniMove is a unified motion modeling framework that leverages shared representations, dual-tower architectures, and expert routing to address diverse motion tasks.
• It standardizes feature extraction and training procedures across domains such as human mobility, autonomous driving, and IMU gesture recognition.
• Empirical evaluations show significant gains in multi-city mobility prediction and cross-domain applications, highlighting the benefits of joint training.

UniMove denotes a line of research that seeks to replace task-specific or city-specific motion pipelines with a single shared model, shared representation space, and shared training regime. In the literature provided here, the name is used explicitly for multi-city human mobility prediction and implicitly as a design motif across autonomous driving, IMU-based gesture recognition, human motion–text–vision modeling, and number-free text-to-motion generation. Across these settings, the unifying premise is that heterogeneous motion problems can be recast so that a single architecture captures common structure while specialized heads, routing mechanisms, or inference procedures handle domain-specific requirements (Han et al., 9 Aug 2025).

1. Scope and nomenclature

The term appears most directly in "UniMove: A Unified Model for Multi-city Human Mobility Prediction" (Han et al., 9 Aug 2025). Closely related papers adopt adjacent names—especially "UniMotion" and "Unified Motion Flow"—for distinct subfields, but they share the same architectural ambition: one model, one representation scheme, and one training framework spanning previously separated motion tasks or domains. A separate line, "MOVE: Unsupervised Movable Object Segmentation and Detection," uses a similar name but addresses unsupervised foreground segmentation rather than motion prediction or motion generation (Bielski et al., 2022).

System	Domain	Unification target
"UniMove: A Unified Model for Multi-city Human Mobility Prediction" (Han et al., 9 Aug 2025)	Human mobility	Multiple cities
"UniMotion: A Unified Motion Framework for Simulation, Prediction and Planning" (Song et al., 31 Jan 2026)	Autonomous driving	Simulation, prediction, planning
"UniMotion: Self-Supervised Learning for Cross-Domain IMU Motion Recognition" (Khanna et al., 12 Mar 2026)	IMU gesture recognition	Devices, populations, low-label settings
"UniMotion: A Unified Framework for Motion-Text-Vision Understanding and Generation" (Wang et al., 23 Mar 2026)	Human motion foundation modeling	Motion, text, RGB
"Unified Number-Free Text-to-Motion Generation Via Flow Matching" (Huang et al., 27 Mar 2026)	Multi-person motion synthesis	Variable agent counts
"MOVE: Unsupervised Movable Object Segmentation and Detection" (Bielski et al., 2022)	Unsupervised segmentation	Movable foreground objects

This naming pattern suggests that "UniMove" is not a single canonical architecture. A plausible implication is that it functions as an umbrella concept for unified motion modeling: shared latent spaces, shared backbones, and task- or domain-adaptive heads.

2. Core design pattern of unified motion modeling

Across the papers, unification is achieved by standardizing representation before specializing behavior. In autonomous driving, UniMotion recasts motion simulation, motion prediction, and motion planning as instances of a single motion-generative problem. It uses one decoder-only Transformer, a common tokenized representation for trajectories and maps, and task-specific interaction modes realized through attention masks and losses (Song et al., 31 Jan 2026). In human mobility prediction, UniMove uses a trajectory–location dual-tower architecture: a Location Tower creates universal spatial representations for locations across cities, while a Trajectory Tower models sequential mobility patterns and outputs an intent embedding matched against candidate locations (Han et al., 9 Aug 2025).

In IMU motion recognition, unification takes a two-stage form. UniMotion first pre-trains on abundant unlabeled human activity data with token-based, nucleus-focused masking, then fine-tunes with a text-guided classifier on scarce labeled gesture data. The same encoder is reused across devices and user populations, while semantic structure is injected through text embeddings of gesture descriptions (Khanna et al., 12 Mar 2026). In the tri-modal human motion setting, UniMotion treats motion as a first-class continuous modality aligned with text and RGB inside a shared LLM backbone. Continuous motion latents are provided by a Cross-Modal Aligned Motion VAE, and cross-modal consistency is enforced through Dual-Posterior KL Alignment and Latent Reconstruction Alignment (Wang et al., 23 Mar 2026).

The number-free text-to-motion setting uses the same logic in latent-space form. Unified Motion Flow bridges heterogeneous motion datasets through a unified latent space, then decomposes generation into motion prior generation and reaction generation. Pyramid Motion Flow handles the prior, and Semi-Noise Motion Flow handles reactions conditioned on previously generated agents, allowing a fixed architecture to scale to variable agent counts (Huang et al., 27 Mar 2026).

These systems differ in backbone choice—decoder-only Transformer, MoE Transformer, self-supervised time-series Transformer, shared LLM, or flow-based latent generator—but converge on the same principle: unification begins with a common representation and is preserved by minimizing domain-specific parameter forks.

3. UniMove as a model for multi-city human mobility prediction

In its explicit formulation, UniMove addresses next-location prediction from human mobility trajectories. A user trajectory is written as

$$S_u = \{ \text{loc}_1, t_1; \text{loc}_2, t_2; \ldots; \text{loc}_n, t_n \},$$

where $\text{loc}_i$ is a discrete location ID and $t_i$ is the start time when the user appears at that location. Given a historical sequence of length $H$, the model learns

$\hat{loc}_{H+1} = \mathcal{F}(S_{1:H}).$

The key difficulty is that location vocabularies are city-specific, whereas the goal is a single parameterization that works across all cities (Han et al., 9 Aug 2025).

UniMove resolves this through a dual-tower design. The Location Tower encodes each grid-cell location using three city-agnostic feature types: POI distribution $\mathbf{P}$, normalized geographical coordinates $\mathbf{G}$, and a discretized popularity rank $\mathbf{R}$. The raw location embedding is constructed as

$$\mathbf{E}_l = \text{concat}(\mathbf{E}_p, \mathbf{E}_g, \mathbf{E}_r),$$

with shared embedding layers across all cities. A Deep & Cross Network then refines these embeddings by combining a Cross layer and a Deep layer, and the final location representation is

$$\mathbf{L} = \text{Concat}(\mathbf{E}_{\text{cross}}, \mathbf{E}_{\text{deep}}).$$

Because these modules depend on features rather than city-local IDs, locations from different cities can occupy a common embedding space.

The Trajectory Tower is an 8-layer Transformer with masked multi-head self-attention and Mixture-of-Experts feed-forward blocks. Its input is the sum of the visited locations’ feature-based embeddings and time-slot embeddings:

$\text{loc}_i$0

The resulting intent embedding $\text{loc}_i$1 is matched against the city’s candidate-location matrix $\text{loc}_i$2 by a dot product,

$\text{loc}_i$3

followed by softmax and cross-entropy. This factorization makes the sequential model and the spatial encoder jointly trainable while remaining fully shared across cities.

Heterogeneity is handled by sparse expert routing. UniMove uses 4 experts by default, with top-$\text{loc}_i$4 gating and noisy gating for load balancing:

$\text{loc}_i$5

The intended effect is that different experts specialize in distinct mobility regimes—city-specific, subregional, or behavioral—without introducing city-specific modules. The paper’s analysis reports that first-layer expert usage differs significantly across Nanchang, Lhasa, and Shanghai, whereas last-layer usage is more similar across cities, indicating a transition from local specialization to shared higher-level regularities (Han et al., 9 Aug 2025).

4. Representative UniMove-style instantiations in other domains

In autonomous driving, UniMotion unifies motion simulation, motion prediction, and motion planning under one decoder-only Transformer. Agent trajectories and map polylines are discretized into shared token vocabularies, relative positional embeddings encode translation- and rotation-invariant geometry, and the same backbone is reconfigured through attention masks. Simulation is treated as causal next-token prediction, prediction as sequence-to-sequence forecasting with Long-range Future Regression, and planning as a hybrid of predicting surrounding agents and auto-regressively generating the ego trajectory (Song et al., 31 Jan 2026). This design emphasizes parameter sharing across tasks that had previously used separate simulators, predictors, and planners.

In IMU-based gesture recognition, UniMotion targets cross-device and cross-population generalization under severe labeled-data scarcity. Pre-training uses windows of 120 samples at 20 Hz, a nucleus detector based on energy changes, and focused masking that places approximately 80% of masks inside the nucleus while masking about 15% of all timesteps. Fine-tuning adds a text-guided classifier with BERT-derived gesture descriptions, a semantic loss, and an InfoNCE-style contrastive loss. The result is a single motion representation reused across smartwatch and earbud data and across blind and sighted users (Khanna et al., 12 Mar 2026).

In motion–text–vision foundation modeling, UniMotion introduces a shared LLM backbone with symmetric dual-path embedders for motion and RGB. Motion is encoded as continuous latent sequences rather than discrete VQ tokens through a Cross-Modal Aligned Motion VAE. Dual-Posterior KL Alignment distills a vision-fused motion encoder into a motion-only encoder, and Latent Reconstruction Alignment pre-trains the newly introduced motion pathway by reconstructing motion latents from noisy interpolants (Wang et al., 23 Mar 2026). The system thereby supports any-to-any understanding, generation, and editing among motion, text, and RGB.

In number-free multi-person text-to-motion generation, Unified Motion Flow addresses the fixed-agent limitation of prior models. It learns a unified latent space over heterogeneous single-person and dual-person datasets, then decomposes synthesis into a single-pass prior stage and multi-pass reaction stages. Pyramid Motion Flow generates a base agent in hierarchical temporal resolutions, and Semi-Noise Motion Flow generates additional agents from a context latent summarizing previously generated motions (Huang et al., 27 Mar 2026). The architecture is reused for arbitrary $\text{loc}_i$6, so the learned distribution does not encode a fixed number of agents.

These variants demonstrate that the unification agenda is not tied to one modality or output space. It appears in discrete token generators, continuous latent LLMs, sequence classifiers, and flow-matching systems alike.

5. Empirical performance

For multi-city mobility prediction, UniMove is evaluated on three real-world datasets: Shanghai, Nanchang, and Lhasa. Joint multi-city training outperforms both classical and neural baselines on all three cities. On Nanchang, UniMove (joint) reaches Acc@1 $\text{loc}_i$7, Acc@3 $\text{loc}_i$8, and Acc@5 $\text{loc}_i$9, compared with the best baseline at $t_i$0, $t_i$1, and $t_i$2. On Lhasa, it reaches $t_i$3, $t_i$4, and $t_i$5; on Shanghai, $t_i$6, $t_i$7, and $t_i$8. The paper reports average improvements over separately trained UniMove models of Acc@1 $t_i$9, Acc@3 $H$0, and Acc@5 $H$1, and the abstract summarizes the gain as over $H$2 (Han et al., 9 Aug 2025). The strongest gains appear in the data-sparse city, which the authors interpret as mutual data enhancement through joint training.

In autonomous driving, UniMotion is evaluated on the Waymo Open Motion Dataset. On the 2025 Sim Agents leaderboard it achieves a Realism Meta Metric of $H$3, with Kinematic $H$4, Interactive $H$5, Map-based $H$6, and minADE $H$7. On the WOMD prediction leaderboard, the fine-tuned model reports minADE $H$8, minFDE $H$9, MR $\hat{loc}_{H+1} = \mathcal{F}(S_{1:H}).$0, mAP $\hat{loc}_{H+1} = \mathcal{F}(S_{1:H}).$1, and Soft mAP $\hat{loc}_{H+1} = \mathcal{F}(S_{1:H}).$2. On the Waymo Planning Benchmark under DIPP’s setup, UniMotion reports collision $\hat{loc}_{H+1} = \mathcal{F}(S_{1:H}).$3, red light $\hat{loc}_{H+1} = \mathcal{F}(S_{1:H}).$4, off route $\hat{loc}_{H+1} = \mathcal{F}(S_{1:H}).$5, and planning errors $\hat{loc}_{H+1} = \mathcal{F}(S_{1:H}).$6 at 1s, $\hat{loc}_{H+1} = \mathcal{F}(S_{1:H}).$7 at 3s, and $\hat{loc}_{H+1} = \mathcal{F}(S_{1:H}).$8 at 5s (Song et al., 31 Jan 2026). The accompanying ablations show that joint NTP + LFR training improves prediction over LFR-only training and that Pred2Gen fine-tuning reduces collision and long-horizon planning error.

In cross-domain IMU recognition, UniMotion achieves an accuracy of $\hat{loc}_{H+1} = \mathcal{F}(S_{1:H}).$9, across an average of 13 gesture classes using only $\mathbf{P}$0 of labeled data for training. The paper further reports approximately $\mathbf{P}$1 accuracy on sighted watch gestures, approximately $\mathbf{P}$2 accuracy on sighted earbud gestures despite only 315 samples total, and approximately $\mathbf{P}$3 accuracy on blind watch gestures. For HAR, the average accuracy across Shoaib, MotionSense, and UCI is approximately $\mathbf{P}$4 with only $\mathbf{P}$5 labeled data (Khanna et al., 12 Mar 2026). Ablations show that removing focused masking, input encodings, text descriptions, or the text-guided contrastive classifier reduces accuracy, with especially large drops on subtle earbud gestures.

In tri-modal motion–text–vision modeling, UniMotion reports improvements over discrete and partially unified baselines across seven tasks. Relative to VQ-VAE and plain continuous baselines, CMA-VAE yields APE $\mathbf{P}$6, AVE $\mathbf{P}$7, and FID $\mathbf{P}$8 in reconstruction, and improves downstream text-to-motion and editing retrieval metrics. In the unified model table, UniMotion reports T2M R@3 $\mathbf{P}$9, M2T BertScore $\mathbf{G}$0, motion prediction ADE $\mathbf{G}$1, motion editing R@3 $\mathbf{G}$2, V2M MPJPE $\mathbf{G}$3, V2T BLEU-4 $\mathbf{G}$4, and MGIE Mot.Acc $\mathbf{G}$5 (Wang et al., 23 Mar 2026). Ablations show that both DPA and LRA materially improve performance.

In number-free text-to-motion generation, UMF improves the autoregressive FreeMotion baseline on InterHuman from FID $\mathbf{G}$6 to $\mathbf{G}$7 and from RTop3 $\mathbf{G}$8 to $\mathbf{G}$9. On InterHuman-AS, UMF reports RTop3 $\mathbf{R}$0, FID $\mathbf{R}$1, MM Dist $\mathbf{R}$2, Diversity $\mathbf{R}$3, and MModality $\mathbf{R}$4 (Huang et al., 27 Mar 2026). The paper’s efficiency analysis shows that the default 2-stage temporal pyramid achieves these results with $\mathbf{R}$5G FLOPs and AITS $\mathbf{R}$6 s, compared with $\mathbf{R}$7G FLOPs and $\mathbf{R}$8 s for FreeMotion.

6. Limitations, misconceptions, and future directions

The most immediate misconception is that UniMove refers to a single stabilized architecture. The evidence here does not support that reading. The explicit UniMove paper concerns multi-city next-location prediction, whereas related work applies the same unification logic to autonomous driving, IMU recognition, tri-modal motion foundation modeling, and number-free motion generation. This suggests a research program rather than a standardized model class.

The explicit UniMove model has several reported limits. Its universal spatial representation depends on carefully engineered features—POI distributions, normalized coordinates, and popularity ranks—and the evaluation covers only three cities. The paper also notes that rare trajectories and low-frequency locations remain difficult, that MoE training requires careful hyperparameter tuning, and that privacy, fairness, and misuse are substantive concerns for mobility prediction systems (Han et al., 9 Aug 2025). The authors position extension to mobility generation, trajectory recovery, anomaly detection, and LLM integration as future work.

Related unified motion systems expose domain-specific limits of the same paradigm. In autonomous driving, tokenization can favor diversity and generative power while slightly hurting raw distance accuracy, the system relies on accurate geometries and substantial compute, and explicit traffic-rule constraints are limited in the main model (Song et al., 31 Jan 2026). In IMU recognition, robustness in noisier environments, other user groups, and cross-modal sensing remains largely unquantified, and only IMU is used (Khanna et al., 12 Mar 2026). In tri-modal modeling, the current system is compute-intensive, visual-motion alignment is trained primarily on indoor data, and visual temporal modeling remains limited (Wang et al., 23 Mar 2026). In number-free generation, UMF is effective for medium-sized groups of approximately 10 agents but is not designed for dense crowds, and training data with three or more agents remain scarce (Huang et al., 27 Mar 2026).

The paper named MOVE underscores a different boundary. It addresses unsupervised movable object segmentation and detection through differentiable inpainting, local shifting, and adversarial training, not unified motion prediction or generation (Bielski et al., 2022). Its inclusion is conceptually useful because it shows that “move” terminology can also refer to movability as a supervisory signal rather than to unified motion representations.

Taken together, the literature indicates that UniMove is best understood as a methodological orientation: replacing siloed models with shared representations, then recovering specialization through attention masks, expert routing, semantic guidance, or latent-space conditioning. The recurrent open question is not whether unification is possible—the reported results indicate that it is—but how far the same principle can scale across tasks, modalities, agents, cities, and deployment constraints.