---
title: 'EgoTraj-Bench: Robust Egocentric Trajectory Prediction'
url: https://www.emergentmind.com/topics/egotraj-bench
type: topic
---

# EgoTraj-Bench: Robust Egocentric Trajectory Prediction

Searching arXiv for the benchmark paper and closely related egocentric trajectory/benchmark papers to ground the article.
In recent arXiv literature, the label **EgoTraj-Bench** is used for several egocentric trajectory benchmarking constructs rather than a single universally standardized artifact. Its most specific use is the benchmark introduced in **"EgoTraj-Bench: Towards Robust Trajectory Prediction Under Ego-view Noisy Observations"**, which pairs noisy first-person-view-derived histories with clean bird’s-eye-view future trajectories for robust trajectory prediction under realistic perceptual corruption. Closely related uses include a standardized multimodal human-navigation benchmark built on the EgoTraj dataset, a city-scale egocentric visual-inertial SLAM benchmark described as the backbone of an “EgoTraj-Bench,” and a scenario-based evaluation paradigm for egocentric pedestrian trajectory prediction [2510.00405][2605.19004][2509.26639][2310.10424].

## 1. Nomenclature and scope

The term **EgoTraj-Bench** is best understood as a research label attached to multiple benchmark designs for egocentric trajectory reasoning. In [2510.00405], it denotes a real-world benchmark for forecasting non-ego-agent trajectories from noisy ego-view observations. In [2605.19004], it denotes a standardized evaluation suite for multimodal egocentric human navigation based on RGB, gaze, head pose, scene semantics, and social cues. In [2310.10424], the paper itself does **not** name the benchmark “EgoTraj-Bench”; rather, the label refers to the scenario-based egocentric pedestrian trajectory prediction benchmarking paradigm introduced there. In [2509.26639], LaMAria is presented as a city-scale egocentric multi-sensor VI-SLAM dataset and benchmark that can serve as the backbone of an “EgoTraj-Bench.”

| Reference | What “EgoTraj-Bench” denotes | Core task |
|---|---|---|
| [2510.00405] | Noisy FPV-to-BEV trajectory benchmark | Robust trajectory prediction |
| [2605.19004] | Standardized suite on EgoTraj dataset | Multimodal human navigation prediction |
| [2509.26639] | LaMAria as backbone of an “EgoTraj-Bench” | Egocentric VI-SLAM |
| [2310.10424] | Scenario-based evaluation paradigm | Egocentric pedestrian prediction |
| [2412.04592] | Methodological input for an egocentric trajectory benchmark | Visibility-aware point tracking |

A common misconception is that the name refers to one benchmark with one protocol. The literature instead shows a cluster of benchmark formulations centered on the same underlying problem: trajectory estimation or prediction under egocentric sensing, where the dominant technical difficulties arise from partial observability, ego-motion, calibration, viewpoint change, and visibility loss.

## 2. Robust trajectory prediction under ego-view noisy observations

The benchmark in [2510.00405] addresses what the paper describes as a fundamental disconnect between model development and deployment. Most trajectory forecasters are trained and evaluated on clean, globally consistent BEV histories, whereas deployed robots perceive from an ego-centric first-person view and must handle occlusions, field-of-view truncations, identity switches, perspective distortion, and ego-motion drift. EgoTraj-Bench closes this gap by pairing **noisy FPV-derived histories**, projected into BEV/world coordinates, with **clean, human-verified BEV ground-truth futures**.

The task definition is explicit. Inputs are noisy ego-view observation histories for non-ego agents together with the ego agent’s own trajectory from robot odometry, and outputs are future trajectories in BEV/world coordinates. The formal variables are
\[
\tilde{X} \in \mathbb{R}^{A \times 2T_p}, \qquad
m \in \{0,1\}^{A \times T_p}, \qquad
Y \in \mathbb{R}^{A \times 2T_f},
\]
where $\tilde{X}$ contains noisy observed histories, $m$ is a binary visibility mask, and $Y$ is the future target. Evaluation is performed in meters in BEV space, using multi-modal ADE and FDE, reported as minADE@K and minFDE@K.

Construction is based on the TBD dataset, which provides synchronized overhead BEV videos and robot-mounted FPV videos in real indoor environments with complex layouts and dynamic crowds. FPV detection uses **YOLOv8**, per-frame visibility uses **YOLOv8-seg**, and tracking uses **BotSort**. For each tracked agent, the bottom center of the FPV bounding box is treated as the contact point with the ground, and back-projection to BEV/world coordinates uses calibrated intrinsics and synchronized ego-motion. Noisy FPV-derived tracks are then associated with clean BEV tracks via **Hungarian matching**, using a weighted mean squared error over position, velocity, and acceleration [2510.00405].

Sample construction uses sliding windows of **8 seconds** per sample, comprising **8 frames (3.2 s) observation** and **12 frames (4.8 s) prediction**, with a modeling interval of approximately **0.4 s** per step and raw video recorded at **30 Hz**. Samples require at least **three valid observed frames**, where validity is defined by visibility threshold **\(>100\) pixels in segmentation mask** and plausibility **frame speed \(<2\) m/s**. The benchmark’s core dataset, **EgoTraj-TBD**, contains **210 minutes of recordings at 30 Hz**, **36,947 aligned trajectory pairs**, and a **70% train, 10% validation, 20% test** chronological split.

The benchmark quantifies the severity of real ego-view corruption. EgoTraj-TBD reports an **FPV Noisy Rate** of **0.37**, defined as the average probability of a history point being invisible, and a **historical alignment error** of **0.66 m**, defined as the distance between FPV-derived BEV history and clean BEV history. These statistics make the benchmark distinct from synthetic FPV renderings such as T2FPV-ETH, where realistic visual nuances and dense interactions are more limited [2510.00405].

## 3. BiFlow and the modeling program associated with EgoTraj-Bench

Built on top of the benchmark in [2510.00405] is **BiFlow**, a dual-stream flow matching model that jointly denoises historical observations and forecasts future motion using a shared encoder. The central idea is that denoised historical semantics improve future prediction robustness, while parameter sharing promotes efficiency. An auxiliary mechanism, **EgoAnchor**, distills intent-aware priors from history features and modulates the future decoder through feature-wise affine transformations.

BiFlow adopts a rectified-flow-like formulation with linear paths. For history and future, Gaussian noise samples $X_0$ and $Y_0$ are interpolated with clean targets $X_1$ and $Y_1$:
\[
X_t = (1 - t)\, X_0 + t\, X_1, \qquad
Y_{t'} = (1 - t')\, Y_0 + t'\, Y_1,
\]
with $t,t' \in [0,1]$. The architecture contains a shared contextual encoder implemented with Transformer MHSA, a history decoder for reconstruction, and a future decoder for multi-candidate trajectory prediction. The future branch is conditioned by EgoAnchor, whereas the history branch mirrors the decoder structure without EgoAnchor modulation.

Training uses MoFlow-style multi-candidate objectives. The history branch reconstructs clean histories in absolute coordinates, while the future branch predicts displacement-based targets for temporal coherence. Reported training settings are **AdamW**, learning rate **0.001**, **cosine annealing with warmup**, weight decay **0.01**, batch size **64**, latent dimension **128**, **150 epochs**, and **10 denoising steps with a logit-normal time scheduler**. At inference time, only the noisy input $\tilde{X}$ and visibility mask $m$ are processed by the shared encoder and future decoder; the reconstruction branch is disabled [2510.00405].

Quantitatively, the paper reports state-of-the-art results. On **T2FPV-ETH**, average performance across folds is **0.60 / 0.74** for BiFlow versus **0.67 / 0.88** for the strongest baseline, **MoFlow**, corresponding to an approximately **11% reduction in minADE** and **16% reduction in minFDE**. On **EgoTraj-TBD**, BiFlow obtains **0.19 / 0.27** versus **0.21 / 0.29** for MoFlow and **0.26 / 0.36** for CoFE. The paper also states that all BEV-trained SOTA models degrade substantially under ego-view noise, with minADE@20 on ETH-UCY rising to approximately **0.67 m** from approximately **0.20 m** under clean histories reported elsewhere [2510.00405].

Ablations support the dual-stream interpretation. On EgoTraj-TBD, at **K = 10**, baseline MoFlow yields **0.31 / 0.48**, BiFlow with Social Interaction only yields **0.29 / 0.47**, BiFlow with Social Interaction plus EgoAnchor yields **0.28 / 0.45**, and full BiFlow with Social Interaction, EgoAnchor, and Shared Encoder yields **0.26 / 0.41**. This suggests that shared latent denoising and intent-conditioned decoding are not merely auxiliary additions but structurally important components of robustness under occlusion, ID switches, and drift.

## 4. Multimodal EgoTraj-Bench for urban human navigation

In [2605.19004], **EgoTraj-Bench** is a standardized evaluation suite built on the **EgoTraj** dataset for multimodal egocentric prediction in real urban environments. Its target is long-horizon, self-directed walking by a **Meta Quest Pro** wearer across sidewalks, crosswalks, and busy streets. The benchmark covers **egocentric human trajectory prediction**, **6-DoF head pose forecasting**, and optional **gaze prediction**. Supported modalities include egocentric RGB, 6-DoF head pose, 3D gaze and image-plane gaze coordinates, scene semantics from segmentation and relative depth, and social cues from nearby people.

The dataset scale is substantial: **75 sessions**, one per participant, with **75 volunteers balanced across gender**, aged **18–38**, representing **14 nationalities**. It contains **10.7 hours**, **1.15 million RGB frames at 30 fps**, and **46.73 km** of walking. Typical sequence lengths are **5–15 minutes per session** with mean approximately **8 minutes**. The standard protocol uses **\(T_{\text{obs}} = 1.5\) s** and **\(T_{\text{pred}} = 3.5\) s**, synchronized at **30 Hz**. Pose and gaze logs are captured at **50 Hz** and aligned to the video timeline during preprocessing, with synchronization bounded by a single-frame interval of approximately **33.3 ms** [2605.19004].

The benchmark formalizes the egocentric state as an **SE(3)** pose, with translation evaluated in meters in the headset world frame and rotation evaluated on **SO(3)**. Translation metrics are standard:
\[
\mathrm{ADE} = \frac{1}{T} \sum_{t=1}^{T} \left\lVert \hat{\mathbf{p}}_{t} - \mathbf{p}_{t} \right\rVert_{2},
\qquad
\mathrm{FDE} = \left\lVert \hat{\mathbf{p}}_{T} - \mathbf{p}_{T} \right\rVert_{2}.
\]
Head orientation is reported with an **L1 rotation loss over SO(3)** based on the elementwise L1 norm of $(\hat{R}R^\top - I)$ averaged across the prediction horizon. The paper also lists geodesic-angle and log-map distances as canonical alternatives.

Scene annotation is an unusually rich component. The dataset provides per-frame structured egocentric descriptions at **1 fps** for **38,606 frames** using **Qwen2.5-VL-7B**, with tags emphasizing navigation context, dynamic agents, traffic activity, gaze fixation targets, and short-term intent. Reported annotation quality is **96% structural compliance** and approximately **93% inter-annotator agreement**.

Benchmark baselines span simple kinematic methods and multimodal Transformers. On the main **80/10/10** split, **Const_Vel** reports **ADE 0.24, FDE 0.35, L1_head 0.82**; **Lin_Ext** reports **0.26, 0.39, 1.39**; **M_Transformer** reports **0.20, 0.32, 0.74**; **CXA-Transformer** reports **0.19, 0.29, 0.69**; and **EgoCast** reports **0.16, 0.28, 0.78**. EgoCast is the best trajectory forecaster by ADE/FDE, whereas CXA-Transformer is best on head rotation. Ablations with CXA-Transformer show the strongest combination to be **\(\mathcal{Y}+\mathcal{P}+\mathcal{S}+\mathcal{G}\)**, which reaches **ADE 0.12, FDE 0.23, L1_head 0.58**. The paper interprets these results as evidence that gaze and scene semantics are highly informative beyond ego-motion alone, and that fine-grained social cues outperform coarse ones [2605.19004].

## 5. Egocentric SLAM as a trajectory benchmark backbone

The paper **"Benchmarking Egocentric Visual-Inertial SLAM at City Scale"** does not present a trajectory-prediction benchmark in the same sense as [2510.00405] or [2605.19004], but it explicitly frames **LaMAria** as the backbone of an **“EgoTraj-Bench”** for egocentric SLAM [2509.26639]. The emphasis here is on long-duration, city-scale trajectory estimation from onboard sensors, with centimeter-accurate surveying control points used as indirect pose annotations.

LaMAria records **more than 22 hours and 70 km of trajectories** over **63 sequences**, covering roughly **1.5 square kilometers** in downtown Zurich with more than **50 m elevation change**. Captures are made with **Project Aria** glasses-like devices carrying two grayscale global-shutter SLAM cameras at **640×480, 20 FPS**, one rolling-shutter RGB camera at **1408×1408, 10 FPS**, two IMUs at **1 kHz** and **800 Hz**, a magnetometer, barometer, thermometer, GNSS receiver, and WiFi/Bluetooth transceivers. The benchmark stresses passive head-worn motion, dynamic crowds and traffic, exposure transitions, low light, moving platforms, and time-varying calibration.

Ground truth uses **483 control points**, of which **134** have full 3D coordinates and **349** have 2D horizontal positions, with average spacing approximately **35 m**. Public control points have approximately **1 cm horizontal** accuracy; the authors’ own measurements have approximately **1.5 cm horizontal** and **3 cm vertical** uncertainties. Sparse evaluation uses control-point alignment errors, **CP@1m recall**, and **R@5m recall** versus pseudo-ground-truth, with three independent runs per sequence. The main benchmark is evaluated in **2D**, because **72.3%** of the control points are 2D.

The benchmark design includes **Level I–IV** controlled tracks, **Short**, **Medium**, and **Long** splits, and challenge subsets for **low-light** and **moving-platform** sequences. Evaluated systems include DSO, ORB-SLAM3, OpenVINS, Kimera VIO, DM-VIO, OKVIS2, and Aria’s SLAM as a closed-source reference. The paper’s high-level conclusion is that **all academic systems suffer on low-light and moving-platform subsets**, while **multi-camera+IMU markedly improves robustness**, and **Aria’s SLAM substantially outperforms academic baselines in all splits** [2509.26639].

Within the broader EgoTraj-Bench landscape, LaMAria shifts the focus from forecasting to egocentric trajectory estimation under long-horizon calibration drift, weak texture, dynamic content, and non-controlled motion. A plausible implication is that it provides the localization and pose-estimation counterpart to the forecasting benchmarks, extending the meaning of “trajectory benchmark” from future prediction to full egocentric trajectory recovery.

## 6. Scenario-based EgoTraj-Bench for egocentric pedestrian prediction

The benchmarking paradigm of [2310.10424] is another distinct use of the EgoTraj-Bench label. The paper introduces a **scenario-based egocentric pedestrian trajectory prediction benchmarking paradigm** on the **PIE** dataset, motivated by the claim that aggregate evaluation hides failures caused by pedestrian scale, ego-motion, and state changes. The proposal therefore segments the test set into scenario slices and ranks models with a scale-aware error.

Scenario extraction uses single-factor, two-factor, and “challenging” settings. Single-factor scenarios include pedestrian scale bins **0–50, 50–80, 80–100, 100–150, 150–200, 200–300, 300+** pixels, pedestrian state **Walking** and **Standing**, and ego-speed bins **0, 0–5, 5–10, 10–20, 20–30, 30+ km/h**. Challenging scenarios include **\(W_o \rightarrow S_p\)**, **\(S_o \rightarrow W_p\)**, **Constant vs. Change** ego-speed, and **Straight vs. Turn**, where turning is defined by yaw change greater than **5°** and changing speed by acceleration magnitude exceeding **0.3 m/s²**.

The paper’s central metric is **scaled bounding-box MSE**, denoted **sB_MSE**, which normalizes per-sequence prediction error by adjusted pedestrian box area to remove scale bias. Occlusion and truncation are handled through a dataset-wide width/height ratio **\(r = 0.34\)**. The motivation is that absolute pixel-space metrics such as B_MSE, C_MSE, and CF_MSE can be misleading in egocentric views because the same pixel error corresponds to different metric distances at different pedestrian scales.

The associated model, **ENCORE**, is fully attention-based and combines step-wise hierarchical fusion, explicit ego-motion modeling, a scaled future trajectory auxiliary task, and an observation reconstruction auxiliary task. The paper reports that the approach achieves **up to 40%** improvement in challenging scenarios. Representative comparisons include **scale 0–50 px**, where **BiTraP 134/0.220**, **PedFormer 76/0.125**, and **ENCORE-D 55/0.091** are reported for **B_MSE/sB_MSE**; **speed 30+ km/h**, where **BiTraP 386/0.190**, **PedFormer 225/0.111**, and **ENCORE-D 145/0.071** are reported; and **Turning**, where **PedFormer** has approximately **0.147** and **ENCORE-D** approximately **0.109** in sB_MSE. On the full PIE benchmark, **ENCORE-D** reports **B_MSE 251, C_MSE 222, CF_MSE 805, sB_MSE 0.023**, outperforming PedFormer’s **295/265/943/0.027** [2310.10424].

This benchmark contributes a different notion of robustness from [2510.00405]. Instead of modeling noisy observation histories directly, it reveals performance heterogeneity by conditioning evaluation on interpretable scenario variables such as scale, ego-speed, and turning.

## 7. Visibility-aware tracking, open issues, and the broader significance of EgoTraj-Bench

The paper **"EgoPoints: Advancing Point Tracking for Egocentric Videos"** is not itself named EgoTraj-Bench, but it provides methodological components that the authors explicitly propose for an egocentric trajectory benchmark [2412.04592]. EgoPoints targets dense point tracking in unconstrained egocentric video and introduces a visibility taxonomy that separates **in-view**, **out-of-view**, and **occluded** states, together with a dedicated **ReID** condition for points that leave the field of view and later return.

Its annotated benchmark contains **517 sequences** and **4,703 tracks**, including **875 out-of-view tracks** and **593 ReID tracks**. Compared with TAP-Vid-DAVIS, it contains **9× more** out-of-view points and **59× more** points requiring ReID, with sequences nearly **8× longer on average**. Metrics include **\(\delta_{\text{avg}}\)** over thresholds **\(\{1,2,4,8,16\}\)**, a relaxed **\(\delta_{\text{avg}}^{*}\)** over **\(\{8,16,24\}\)**, **ReID\(\delta_{\text{avg}}\)**, **IVA**, **OOVA**, **OA**, and **MTE**. The authors also introduce the semi-real **K-EPIC** training pipeline with **11K sequences** and approximately **22.1M tracks**, and show that fine-tuning improves CoTracker and PIPs++ on EgoPoints [2412.04592].

Across the different EgoTraj-Bench formulations, several limitations recur. Platform-specific sensing matters: [2510.00405] notes that camera height, field of view, and odometry quality can shift the noise distribution; [2509.26639] shows that time-varying intrinsics and extrinsics are major failure sources in wearable SLAM; [2605.19004] identifies abrupt near-instantaneous turns, sensor drift, viewpoint instability, and participant variability as persistent difficulties; and [2310.10424] emphasizes long-tail scenario sparsity and dependence on reliable ego-motion and state labels. Availability is also uneven: [2510.00405] states that file formats, schema details, licensing, and a direct URL for EgoTraj-Bench distribution are not specified in the text, whereas [2605.19004] and [2509.26639] provide public repositories or websites.

Taken together, the literature shows that **EgoTraj-Bench** has evolved into a broader designation for benchmarks that evaluate trajectory reasoning under egocentric sensing. In one line of work it means robust FPV-to-BEV forecasting under perceptual noise; in another, multimodal urban navigation prediction with gaze and head pose; in another, city-scale VI-SLAM with survey-grade control points; and in another, scenario-stratified egocentric pedestrian prediction. The unifying technical theme is that egocentric trajectory problems are dominated not by idealized clean histories but by realistic visibility loss, ego-motion, calibration drift, and multimodal intent cues.

Source: https://www.emergentmind.com/topics/egotraj-bench