UniEgoMotion: Unified Egocentric Motion Diffusion

• UniEgoMotion is a unified conditional diffusion model that reconstructs, forecasts, and generates full-body motion from egocentric RGB inputs and device trajectories.
• It leverages a head-centric, floor-canonicalized representation and DINOv2-based image conditioning to align motion with the egocentric camera view.
• The approach achieves state-of-the-art performance on the EE4D-Motion dataset, enhancing physical realism and reducing artifacts compared to pelvis-centric methods.

UniEgoMotion is a unified conditional motion diffusion model for egocentric motion reconstruction, forecasting, and generation from first-person visual inputs. It was introduced to address egocentric settings in which limited field of view, frequent occlusions, and dynamic cameras hinder scene perception, and it does so without relying on explicit 3D scene representations. Its defining technical choices are a head-centric motion representation tailored for head-mounted devices, image-based scene conditioning, and a single framework that supports frame-aligned reconstruction, future forecasting, and whole-body motion generation from a single egocentric image (Patel et al., 2 Aug 2025).

1. Scope and task formulation

UniEgoMotion operates on egocentric RGB observations, an ego-device trajectory, and full-body 3D motion represented with SMPL-X. The inputs are an egocentric RGB sequence

$$I_{1:N} = (I_1,\dots,I_N),\quad I_i \in \mathbb{R}^{H\times W\times 3},$$

an ego-device trajectory

$T_{1:N} = (T_1,\dots,T_N),$

and full 3D body motion

$X_{1:N} = (X_1,\dots,X_N),$

with

$X_i = (R^i_\theta, t^i_\theta, \theta_i, \beta_i),$

where $R^i_\theta \in \mathbb{R}^3$ is the global rotation of the pelvis, $t^i_\theta \in \mathbb{R}^3$ is the global translation of the pelvis, $\theta_i \in \mathbb{R}^{21\times 3}$ are local joint angles, and $\beta_i \in \mathbb{R}^{10}$ are shape parameters that remain constant over time (Patel et al., 2 Aug 2025).

The model is defined around three tasks. Egocentric Motion Reconstruction estimates frame-aligned full-body motion from egocentric video and ego trajectory:

$p(X_{1:N}\mid I_{1:N}, T_{1:N}).$

Egocentric Motion Forecasting predicts future motion from past egocentric observations:

$p(X_{n+1:N}\mid I_{1:n}, T_{1:n}), \quad n < N.$

Egocentric Motion Generation synthesizes an entire motion sequence from a single first-person image:

$T_{1:N} = (T_1,\dots,T_N),$0

The latter two were introduced as explicit egocentric tasks in the UniEgoMotion work (Patel et al., 2 Aug 2025).

This formulation differs from standard third-person motion synthesis in several stated ways. Third-person methods commonly assume wide field of view, explicit 3D scene representations such as point clouds, meshes, voxels, SDFs, or object models, and direct visibility of the full body. UniEgoMotion instead targets first-person RGB observations with limited FoV, severe occlusion of the wearer’s body, and a dynamic head-mounted camera, while conditioning only on image observations and, when available, the ego-device trajectory (Patel et al., 2 Aug 2025).

2. Head-centric and floor-canonicalized motion representation

A central feature of UniEgoMotion is its replacement of the original pelvis-centric SMPL-X parameterization with a head-centric, floor-canonicalized representation aligned to egocentric devices. The paper motivates this change by noting that the ego trajectory $T_{1:N} = (T_1,\dots,T_N),$1 and the images $T_{1:N} = (T_1,\dots,T_N),$2 are naturally head-centric or camera-centric, whereas standard SMPL-X motion is pelvis-centric in a global frame (Patel et al., 2 Aug 2025).

Starting from SMPL-X, the method first computes per-joint $T_{1:N} = (T_1,\dots,T_N),$3 transforms in world coordinates through forward kinematics. This yields a head transform

$T_{1:N} = (T_1,\dots,T_N),$4

and transforms for the remaining joints

$T_{1:N} = (T_1,\dots,T_N),$5

which together are denoted by

$T_{1:N} = (T_1,\dots,T_N),$6

The representation therefore replaces local-angle reasoning with globally expressed joint transforms (Patel et al., 2 Aug 2025).

The model then constructs a canonical head transform $T_{1:N} = (T_1,\dots,T_N),$7 by projecting the head pose onto the floor plane. This removes pitch and roll from head orientation, retains yaw, and removes vertical height relative to the floor. The resulting $T_{1:N} = (T_1,\dots,T_N),$8 is a floor-projected head trajectory that serves as a canonical frame. Motion is then expressed relative to this frame through

$T_{1:N} = (T_1,\dots,T_N),$9

and the canonical floor trajectory itself is represented as temporal residuals

$X_{1:N} = (X_1,\dots,X_N),$0

The diffusion model predicts these residuals together with canonicalized joint poses rather than absolute world-space trajectories (Patel et al., 2 Aug 2025).

The paper attributes several benefits to this representation. It aligns the motion state with the physical placement of the egocentric device, removes redundant degrees of freedom by canonicalizing pitch, roll, and height, and improves physical plausibility by expressing joints in $X_{1:N} = (X_1,\dots,X_N),$1 relative to the canonical head frame. Ablations reported in the paper state that pelvis-centric and global alternatives underperform, with the proposed head-centric representation substantially improving Foot Slide and Foot Contact and reducing artifacts such as floating and floor penetration (Patel et al., 2 Aug 2025).

3. Unified conditional diffusion architecture

UniEgoMotion uses a conditional denoising diffusion probabilistic model over motion sequences in the head-centric representation. For a motion sequence $X_{1:N} = (X_1,\dots,X_N),$2, the forward process at diffusion step $X_{1:N} = (X_1,\dots,X_N),$3 is

$X_{1:N} = (X_1,\dots,X_N),$4

with $X_{1:N} = (X_1,\dots,X_N),$5 and a cosine noise schedule. The reverse model predicts the clean motion from a noisy motion sample and a conditioning signal $X_{1:N} = (X_1,\dots,X_N),$6:

$X_{1:N} = (X_1,\dots,X_N),$7

Training uses the simplified denoising objective

$X_{1:N} = (X_1,\dots,X_N),$8

Inference starts from Gaussian noise and iteratively denoises to a motion sample. The model is trained with classifier-free guidance (Patel et al., 2 Aug 2025).

The denoiser is a Transformer decoder with 12 layers and hidden dimension 768. Each per-frame motion vector is flattened and projected with a linear layer $X_{1:N} = (X_1,\dots,X_N),$9 to obtain

$X_i = (R^i_\theta, t^i_\theta, \theta_i, \beta_i),$0

Trajectory conditioning is encoded with a linear layer

$X_i = (R^i_\theta, t^i_\theta, \theta_i, \beta_i),$1

and is injected additively:

$X_i = (R^i_\theta, t^i_\theta, \theta_i, \beta_i),$2

Image conditioning is handled separately through cross-attention. Egocentric frames are encoded with a frozen DINOv2 Vision Transformer, and the image features act as keys and values while motion tokens act as queries. The paper states that DINOv2 outperformed CLIP and EgoVideo in this setting because it provides more fine-grained scene features (Patel et al., 2 Aug 2025).

A notable design feature is the unification of reconstruction, forecasting, and generation through conditioning masks. Missing images or trajectories are replaced with learnable mask tokens. Reconstruction uses full conditioning

$X_i = (R^i_\theta, t^i_\theta, \theta_i, \beta_i),$3

Generation from one image uses

$X_i = (R^i_\theta, t^i_\theta, \theta_i, \beta_i),$4

Forecasting is trained from partial observations $X_i = (R^i_\theta, t^i_\theta, \theta_i, \beta_i),$5, and at inference the paper also uses diffusion inpainting with RePaint-style overwriting so that reconstructed past motion is preserved while future motion is sampled. Training alternates between reconstruction mode and generation mode with probability 0.5 each (Patel et al., 2 Aug 2025).

Later work describes this task unification as one of UniEgoMotion’s key ideas. EgoPriMo explicitly identifies UniEgoMotion as “the closest published egocentric model for reconstruction, forecasting, and generation” and states that its own unified task-conditioning formulation is inspired by UniEgoMotion’s masking strategy (Ge et al., 7 Jun 2026).

4. EE4D-Motion dataset and supervision pipeline

UniEgoMotion was introduced together with EE4D-Motion, a dataset derived from EgoExo4D and designed to provide paired egocentric video, ego trajectory, and dense pseudo-ground-truth 3D motion. EgoExo4D supplies egocentric Project Aria video, synchronized exocentric views, and inertial SLAM trajectories, but its existing 3D pose annotations were described as sparse, noisy, and unsuitable for direct training of continuous multi-frame SMPL-X motion models. EE4D-Motion addresses this through a multi-stage motion fitting pipeline (Patel et al., 2 Aug 2025).

The pipeline begins with person detection and tracking in exocentric views. The camera wearer is identified using the known Aria 3D trajectory. Per-frame initial pose estimation then combines 2D keypoints from ViTPose and initial SMPL-X parameters from SMPLer-X. These per-view estimates are aggregated in a multi-view fitting stage that minimizes

$X_i = (R^i_\theta, t^i_\theta, \theta_i, \beta_i),$6

where $X_i = (R^i_\theta, t^i_\theta, \theta_i, \beta_i),$7 and $X_i = (R^i_\theta, t^i_\theta, \theta_i, \beta_i),$8 are pose and shape priors, $X_i = (R^i_\theta, t^i_\theta, \theta_i, \beta_i),$9 is the SMPL-X joint regressor, $R^i_\theta \in \mathbb{R}^3$0 is projection into view $R^i_\theta \in \mathbb{R}^3$1, $R^i_\theta \in \mathbb{R}^3$2 is the observed 2D keypoint, and $R^i_\theta \in \mathbb{R}^3$3 is a Geman–McClure loss. A subsequent sequence-level optimization fixes $R^i_\theta \in \mathbb{R}^3$4 to the sequence average, incorporates the egocentric view as an additional fitting view, and adds temporal smoothness to reduce jitter. The dataset is then filtered to remove sequences with noisy SLAM trajectories, extreme occlusions, or unreliable pose estimation (Patel et al., 2 Aug 2025).

After filtering, EE4D-Motion retains more than 110 hours of continuous SMPL-X motion from an initial 208 hours. The training setup used in the paper extracts 8-second clips at 10 fps, producing 80-frame sequences, with overlapping clips every 2 seconds to yield 143k training samples. Validation clips are extracted every 20 seconds and number approximately 4400. The activities include sports, cooking, dancing, music, bike repair, and healthcare tasks. The paper characterizes EE4D-Motion as distinctive in combining large-scale paired egocentric video, real camera trajectories, and continuous full-body motion in standard SMPL-X form (Patel et al., 2 Aug 2025).

This data regime differs from earlier egocentric datasets that target pose estimation from wearable cameras. UnrealEgo, for example, provides stereo fisheye images, depth maps, 2D projections, camera poses, and 3D joint supervision for egocentric 3D human pose estimation, but it is synthetic and benchmarked around stereo pose estimation rather than scene-aware full-body motion generation from first-person images (Akada et al., 2022).

5. Evaluation and empirical characteristics

UniEgoMotion is evaluated on reconstruction, forecasting, and generation over the EE4D-Motion validation split using joint-space, head-motion, foot-contact, semantic, and distributional metrics. These include MPJPE, MPJPE-PA, MPJPE-H, head rotation error, head translation error, Foot Sliding, Foot Contact, Semantic Similarity based on TMR motion embeddings, and FID in motion latent space (Patel et al., 2 Aug 2025).

Task	UniEgoMotion	Representative comparison
Reconstruction	MPJPE 0.100 m; MPJPE-PA 0.053; MPJPE-H 0.180; Foot Slide 3.62; Foot Contact 0.027; SS 0.918; FID 0.027	AvatarPoser MPJPE 0.116; EgoEgo 0.130; EgoAllo 0.163
Forecasting	MPJPE 0.206; Foot Slide 2.60; Foot Contact 0.026	LSTM 0.238; Two-stage 0.253
Generation	MPJPE 0.226; Foot Slide 2.89; Foot Contact 0.025	LSTM 0.216; Two-stage 0.222

In reconstruction, the paper reports that UniEgoMotion outperforms AvatarPoser, EgoEgo, and EgoAllo on every listed metric. Its MPJPE is 0.100 m compared with 0.116 m for AvatarPoser, 0.130 for EgoEgo, and 0.163 for EgoAllo. Its MPJPE-PA is 0.053, its MPJPE-H is 0.180, its Foot Slide is 3.62, and its Foot Contact is 0.027, all reported as best among the compared methods. Semantic Similarity reaches 0.918 and FID 0.027, again the best values in the table. The paper characterizes this as state-of-the-art performance for egocentric motion reconstruction (Patel et al., 2 Aug 2025).

For forecasting, evaluated over the first 2 seconds of future motion within a 2–4 second window, UniEgoMotion achieves MPJPE 0.206, outperforming LSTM forecasting at 0.238 and a two-stage method at 0.253. Its Foot Slide is 2.60, better than 7.23 for the LSTM and 3.55 for the two-stage baseline, while Foot Contact is 0.026, tied for best. For generation from a single egocentric image, MPJPE is 0.226, numerically similar to deterministic baselines, but the physical realism metrics are stronger: Foot Slide is 2.89 and Foot Contact is 0.025, both reported as best. The paper attributes some of the MPJPE behavior in generation to the stochastic nature of diffusion relative to deterministic baselines (Patel et al., 2 Aug 2025).

Ablation studies isolate the main design choices. Replacing the Transformer decoder with a Transformer encoder or a 1D-UNet degrades performance, especially in Foot Sliding and Semantic Similarity. Replacing the head-centric representation with a pelvis-centric one produces markedly worse results; for reconstruction, pelvis-centric MPJPE rises to 0.166, and forecasting MPJPE rises to 0.245. DINOv2 is reported as the strongest image encoder, outperforming CLIP and EgoVideo. Removing video conditioning harms generation and reduces Semantic Similarity, while removing trajectory conditioning sharply increases head translation error and degrades absolute positioning even when local pose plausibility remains competitive (Patel et al., 2 Aug 2025).

Qualitative examples in the paper emphasize three capabilities. In reconstruction, UniEgoMotion better aligns motion with the ground plane and visible environment than competing methods. In forecasting, it predicts plausible continuations such as repairing a bike tire, continuing salsa dance patterns, or dribbling a soccer ball around a cone. In generation, it produces full-body motion from a single egocentric frame in scenes involving soccer juggling, basketball shooting, and interaction with a lower cabinet. The paper identifies this as the first demonstration, to the authors’ knowledge, of scene-aware 3D full-body motion generation from a single egocentric frame without explicit 3D scene input (Patel et al., 2 Aug 2025).

6. Position within egocentric motion research, limitations, and outlook

Within egocentric motion modeling, UniEgoMotion occupies a specific position. It is a vision-and-trajectory model for full-body motion in SMPL-X form, unified across reconstruction, forecasting, and generation. EgoPriMo later uses UniEgoMotion as its principal egocentric baseline and describes it as a unified model for those three tasks. At the same time, EgoPriMo argues that UniEgoMotion does not provide a unified multimodal text-vision-motion prior, cannot consume text-only motion datasets such as HumanML3D, and does not use a triple-stream DiT or flow-matching formulation (Ge et al., 7 Jun 2026).

A related but distinct line is represented by EgoLM, which treats egocentric motions and natural language with a GPT-2-based motion-LLM. EgoLM unifies motion tracking, motion understanding, motion-to-text, and text-to-motion by discretizing motion with a VQ-VAE and projecting egocentric video and sparse sensors into the language-model latent space. This suggests a broader design space in which UniEgoMotion’s scene-aware diffusion formulation and EgoLM’s joint motion-language modeling address adjacent parts of the egocentric learning problem (Hong et al., 2024).

A recurring ambiguity concerns the term “ego-motion.” In UniEgoMotion, the term refers to egocentric human motion modeling from first-person observations, not to monocular visual odometry. That distinction is important because works such as UNO target camera trajectory estimation across KITTI, EuRoC-MAV, and TUM-RGBD through self-supervised monocular odometry, whereas UniEgoMotion assumes an ego-device trajectory as input and predicts full-body motion in SMPL-X form (Zhao et al., 8 Jun 2025).

The paper states several limitations. EE4D-Motion provides pseudo-ground-truth motion obtained through a multi-view fitting pipeline rather than mocap-grade supervision. Reconstruction and forecasting depend on an accurate inertial SLAM trajectory. The formulation assumes a head-mounted camera with fixed placement relative to the head. Physical realism is improved through the representation and loss design, but there is no explicit physical simulation, so interaction with unseen obstacles is not guaranteed to be realistic. The training distribution is centered on EgoExo4D-like environments, and broader domain generalization remains open (Patel et al., 2 Aug 2025).

The authors identify several future directions: more detailed modeling of egocentric scene-motion interactions, multimodal prompting with text or LLMs, stronger 3D scene reasoning without explicit scene reconstruction, and more efficient diffusion sampling for real-time deployment. A plausible implication is that UniEgoMotion functions as a foundation for later egocentric priors rather than as a closed endpoint. That interpretation is consistent with EgoPriMo’s reuse of UniEgoMotion’s unified masking concept and its extension toward language-conditioned humanoid control (Ge et al., 7 Jun 2026).