EgoTraj-Bench: Robust Egocentric Trajectory Prediction

• EgoTraj-Bench is a research label covering diverse benchmark designs for egocentric trajectory estimation, spanning noisy FPV-to-BEV prediction, multimodal navigation, and SLAM.
• The benchmarks address challenges like occlusions, calibration drift, and ego-motion errors by pairing noisy first-person view inputs with clean, verified ground-truth trajectories.
• Recent implementations such as BiFlow and ENCORE show significant performance gains, demonstrating the practical benefits of dual-stream architectures and scenario-specific evaluations.

Searching arXiv for the benchmark paper and closely related egocentric trajectory/benchmark papers to ground the article. In recent arXiv literature, the label EgoTraj-Bench is used for several egocentric trajectory benchmarking constructs rather than a single universally standardized artifact. Its most specific use is the benchmark introduced in "EgoTraj-Bench: Towards Robust Trajectory Prediction Under Ego-view Noisy Observations", which pairs noisy first-person-view-derived histories with clean bird’s-eye-view future trajectories for robust trajectory prediction under realistic perceptual corruption. Closely related uses include a standardized multimodal human-navigation benchmark built on the EgoTraj dataset, a city-scale egocentric visual-inertial SLAM benchmark described as the backbone of an “EgoTraj-Bench,” and a scenario-based evaluation paradigm for egocentric pedestrian trajectory prediction (Liu et al., 1 Oct 2025, Yehia et al., 18 May 2026, Krishnan et al., 30 Sep 2025, Rasouli, 2023).

1. Nomenclature and scope

The term EgoTraj-Bench is best understood as a research label attached to multiple benchmark designs for egocentric trajectory reasoning. In (Liu et al., 1 Oct 2025), it denotes a real-world benchmark for forecasting non-ego-agent trajectories from noisy ego-view observations. In (Yehia et al., 18 May 2026), it denotes a standardized evaluation suite for multimodal egocentric human navigation based on RGB, gaze, head pose, scene semantics, and social cues. In (Rasouli, 2023), the paper itself does not name the benchmark “EgoTraj-Bench”; rather, the label refers to the scenario-based egocentric pedestrian trajectory prediction benchmarking paradigm introduced there. In (Krishnan et al., 30 Sep 2025), LaMAria is presented as a city-scale egocentric multi-sensor VI-SLAM dataset and benchmark that can serve as the backbone of an “EgoTraj-Bench.”

Reference	What “EgoTraj-Bench” denotes	Core task
(Liu et al., 1 Oct 2025)	Noisy FPV-to-BEV trajectory benchmark	Robust trajectory prediction
(Yehia et al., 18 May 2026)	Standardized suite on EgoTraj dataset	Multimodal human navigation prediction
(Krishnan et al., 30 Sep 2025)	LaMAria as backbone of an “EgoTraj-Bench”	Egocentric VI-SLAM
(Rasouli, 2023)	Scenario-based evaluation paradigm	Egocentric pedestrian prediction
(Darkhalil et al., 2024)	Methodological input for an egocentric trajectory benchmark	Visibility-aware point tracking

A common misconception is that the name refers to one benchmark with one protocol. The literature instead shows a cluster of benchmark formulations centered on the same underlying problem: trajectory estimation or prediction under egocentric sensing, where the dominant technical difficulties arise from partial observability, ego-motion, calibration, viewpoint change, and visibility loss.

2. Robust trajectory prediction under ego-view noisy observations

The benchmark in (Liu et al., 1 Oct 2025) addresses what the paper describes as a fundamental disconnect between model development and deployment. Most trajectory forecasters are trained and evaluated on clean, globally consistent BEV histories, whereas deployed robots perceive from an ego-centric first-person view and must handle occlusions, field-of-view truncations, identity switches, perspective distortion, and ego-motion drift. EgoTraj-Bench closes this gap by pairing noisy FPV-derived histories, projected into BEV/world coordinates, with clean, human-verified BEV ground-truth futures.

The task definition is explicit. Inputs are noisy ego-view observation histories for non-ego agents together with the ego agent’s own trajectory from robot odometry, and outputs are future trajectories in BEV/world coordinates. The formal variables are

$$\tilde{X} \in \mathbb{R}^{A \times 2T_p}, \qquad m \in \{0,1\}^{A \times T_p}, \qquad Y \in \mathbb{R}^{A \times 2T_f},$$

where $\tilde{X}$ contains noisy observed histories, $m$ is a binary visibility mask, and $Y$ is the future target. Evaluation is performed in meters in BEV space, using multi-modal ADE and FDE, reported as minADE@K and minFDE@K.

Construction is based on the TBD dataset, which provides synchronized overhead BEV videos and robot-mounted FPV videos in real indoor environments with complex layouts and dynamic crowds. FPV detection uses YOLOv8, per-frame visibility uses YOLOv8-seg, and tracking uses BotSort. For each tracked agent, the bottom center of the FPV bounding box is treated as the contact point with the ground, and back-projection to BEV/world coordinates uses calibrated intrinsics and synchronized ego-motion. Noisy FPV-derived tracks are then associated with clean BEV tracks via Hungarian matching, using a weighted mean squared error over position, velocity, and acceleration (Liu et al., 1 Oct 2025).

Sample construction uses sliding windows of 8 seconds per sample, comprising 8 frames (3.2 s) observation and 12 frames (4.8 s) prediction, with a modeling interval of approximately 0.4 s per step and raw video recorded at 30 Hz. Samples require at least three valid observed frames, where validity is defined by visibility threshold $>100$ pixels in segmentation mask and plausibility frame speed $<2$ m/s. The benchmark’s core dataset, EgoTraj-TBD, contains 210 minutes of recordings at 30 Hz, 36,947 aligned trajectory pairs, and a 70% train, 10% validation, 20% test chronological split.

The benchmark quantifies the severity of real ego-view corruption. EgoTraj-TBD reports an FPV Noisy Rate of 0.37, defined as the average probability of a history point being invisible, and a historical alignment error of 0.66 m, defined as the distance between FPV-derived BEV history and clean BEV history. These statistics make the benchmark distinct from synthetic FPV renderings such as T2FPV-ETH, where realistic visual nuances and dense interactions are more limited (Liu et al., 1 Oct 2025).

3. BiFlow and the modeling program associated with EgoTraj-Bench

Built on top of the benchmark in (Liu et al., 1 Oct 2025) is BiFlow, a dual-stream flow matching model that jointly denoises historical observations and forecasts future motion using a shared encoder. The central idea is that denoised historical semantics improve future prediction robustness, while parameter sharing promotes efficiency. An auxiliary mechanism, EgoAnchor, distills intent-aware priors from history features and modulates the future decoder through feature-wise affine transformations.

BiFlow adopts a rectified-flow-like formulation with linear paths. For history and future, Gaussian noise samples $X_0$ and $Y_0$ are interpolated with clean targets $X_1$ and $Y_1$: $\tilde{X}$0 with $\tilde{X}$1. The architecture contains a shared contextual encoder implemented with Transformer MHSA, a history decoder for reconstruction, and a future decoder for multi-candidate trajectory prediction. The future branch is conditioned by EgoAnchor, whereas the history branch mirrors the decoder structure without EgoAnchor modulation.

Training uses MoFlow-style multi-candidate objectives. The history branch reconstructs clean histories in absolute coordinates, while the future branch predicts displacement-based targets for temporal coherence. Reported training settings are AdamW, learning rate 0.001, cosine annealing with warmup, weight decay 0.01, batch size 64, latent dimension 128, 150 epochs, and 10 denoising steps with a logit-normal time scheduler. At inference time, only the noisy input $\tilde{X}$2 and visibility mask $\tilde{X}$3 are processed by the shared encoder and future decoder; the reconstruction branch is disabled (Liu et al., 1 Oct 2025).

Quantitatively, the paper reports state-of-the-art results. On T2FPV-ETH, average performance across folds is 0.60 / 0.74 for BiFlow versus 0.67 / 0.88 for the strongest baseline, MoFlow, corresponding to an approximately 11% reduction in minADE and 16% reduction in minFDE. On EgoTraj-TBD, BiFlow obtains 0.19 / 0.27 versus 0.21 / 0.29 for MoFlow and 0.26 / 0.36 for CoFE. The paper also states that all BEV-trained SOTA models degrade substantially under ego-view noise, with minADE@20 on ETH-UCY rising to approximately 0.67 m from approximately 0.20 m under clean histories reported elsewhere (Liu et al., 1 Oct 2025).

Ablations support the dual-stream interpretation. On EgoTraj-TBD, at K = 10, baseline MoFlow yields 0.31 / 0.48, BiFlow with Social Interaction only yields 0.29 / 0.47, BiFlow with Social Interaction plus EgoAnchor yields 0.28 / 0.45, and full BiFlow with Social Interaction, EgoAnchor, and Shared Encoder yields 0.26 / 0.41. This suggests that shared latent denoising and intent-conditioned decoding are not merely auxiliary additions but structurally important components of robustness under occlusion, ID switches, and drift.

4. Multimodal EgoTraj-Bench for urban human navigation

In (Yehia et al., 18 May 2026), EgoTraj-Bench is a standardized evaluation suite built on the EgoTraj dataset for multimodal egocentric prediction in real urban environments. Its target is long-horizon, self-directed walking by a Meta Quest Pro wearer across sidewalks, crosswalks, and busy streets. The benchmark covers egocentric human trajectory prediction, 6-DoF head pose forecasting, and optional gaze prediction. Supported modalities include egocentric RGB, 6-DoF head pose, 3D gaze and image-plane gaze coordinates, scene semantics from segmentation and relative depth, and social cues from nearby people.

The dataset scale is substantial: 75 sessions, one per participant, with 75 volunteers balanced across gender, aged 18–38, representing 14 nationalities. It contains 10.7 hours, 1.15 million RGB frames at 30 fps, and 46.73 km of walking. Typical sequence lengths are 5–15 minutes per session with mean approximately 8 minutes. The standard protocol uses $\tilde{X}$4 s and $\tilde{X}$5 s, synchronized at 30 Hz. Pose and gaze logs are captured at 50 Hz and aligned to the video timeline during preprocessing, with synchronization bounded by a single-frame interval of approximately 33.3 ms (Yehia et al., 18 May 2026).

The benchmark formalizes the egocentric state as an SE(3) pose, with translation evaluated in meters in the headset world frame and rotation evaluated on SO(3). Translation metrics are standard: $\tilde{X}$6 Head orientation is reported with an L1 rotation loss over SO(3) based on the elementwise L1 norm of $\tilde{X}$7 averaged across the prediction horizon. The paper also lists geodesic-angle and log-map distances as canonical alternatives.

Scene annotation is an unusually rich component. The dataset provides per-frame structured egocentric descriptions at 1 fps for 38,606 frames using Qwen2.5-VL-7B, with tags emphasizing navigation context, dynamic agents, traffic activity, gaze fixation targets, and short-term intent. Reported annotation quality is 96% structural compliance and approximately 93% inter-annotator agreement.

Benchmark baselines span simple kinematic methods and multimodal Transformers. On the main 80/10/10 split, Const_Vel reports ADE 0.24, FDE 0.35, L1_head 0.82; Lin_Ext reports 0.26, 0.39, 1.39; M_Transformer reports 0.20, 0.32, 0.74; CXA-Transformer reports 0.19, 0.29, 0.69; and EgoCast reports 0.16, 0.28, 0.78. EgoCast is the best trajectory forecaster by ADE/FDE, whereas CXA-Transformer is best on head rotation. Ablations with CXA-Transformer show the strongest combination to be $\tilde{X}$8, which reaches ADE 0.12, FDE 0.23, L1_head 0.58. The paper interprets these results as evidence that gaze and scene semantics are highly informative beyond ego-motion alone, and that fine-grained social cues outperform coarse ones (Yehia et al., 18 May 2026).

5. Egocentric SLAM as a trajectory benchmark backbone

The paper "Benchmarking Egocentric Visual-Inertial SLAM at City Scale" does not present a trajectory-prediction benchmark in the same sense as (Liu et al., 1 Oct 2025) or (Yehia et al., 18 May 2026), but it explicitly frames LaMAria as the backbone of an “EgoTraj-Bench” for egocentric SLAM (Krishnan et al., 30 Sep 2025). The emphasis here is on long-duration, city-scale trajectory estimation from onboard sensors, with centimeter-accurate surveying control points used as indirect pose annotations.

LaMAria records more than 22 hours and 70 km of trajectories over 63 sequences, covering roughly 1.5 square kilometers in downtown Zurich with more than 50 m elevation change. Captures are made with Project Aria glasses-like devices carrying two grayscale global-shutter SLAM cameras at 640×480, 20 FPS, one rolling-shutter RGB camera at 1408×1408, 10 FPS, two IMUs at 1 kHz and 800 Hz, a magnetometer, barometer, thermometer, GNSS receiver, and WiFi/Bluetooth transceivers. The benchmark stresses passive head-worn motion, dynamic crowds and traffic, exposure transitions, low light, moving platforms, and time-varying calibration.

Ground truth uses 483 control points, of which 134 have full 3D coordinates and 349 have 2D horizontal positions, with average spacing approximately 35 m. Public control points have approximately 1 cm horizontal accuracy; the authors’ own measurements have approximately 1.5 cm horizontal and 3 cm vertical uncertainties. Sparse evaluation uses control-point alignment errors, CP@1m recall, and R@5m recall versus pseudo-ground-truth, with three independent runs per sequence. The main benchmark is evaluated in 2D, because 72.3% of the control points are 2D.

The benchmark design includes Level I–IV controlled tracks, Short, Medium, and Long splits, and challenge subsets for low-light and moving-platform sequences. Evaluated systems include DSO, ORB-SLAM3, OpenVINS, Kimera VIO, DM-VIO, OKVIS2, and Aria’s SLAM as a closed-source reference. The paper’s high-level conclusion is that all academic systems suffer on low-light and moving-platform subsets, while multi-camera+IMU markedly improves robustness, and Aria’s SLAM substantially outperforms academic baselines in all splits (Krishnan et al., 30 Sep 2025).

Within the broader EgoTraj-Bench landscape, LaMAria shifts the focus from forecasting to egocentric trajectory estimation under long-horizon calibration drift, weak texture, dynamic content, and non-controlled motion. A plausible implication is that it provides the localization and pose-estimation counterpart to the forecasting benchmarks, extending the meaning of “trajectory benchmark” from future prediction to full egocentric trajectory recovery.

6. Scenario-based EgoTraj-Bench for egocentric pedestrian prediction

The benchmarking paradigm of (Rasouli, 2023) is another distinct use of the EgoTraj-Bench label. The paper introduces a scenario-based egocentric pedestrian trajectory prediction benchmarking paradigm on the PIE dataset, motivated by the claim that aggregate evaluation hides failures caused by pedestrian scale, ego-motion, and state changes. The proposal therefore segments the test set into scenario slices and ranks models with a scale-aware error.

Scenario extraction uses single-factor, two-factor, and “challenging” settings. Single-factor scenarios include pedestrian scale bins 0–50, 50–80, 80–100, 100–150, 150–200, 200–300, 300+ pixels, pedestrian state Walking and Standing, and ego-speed bins 0, 0–5, 5–10, 10–20, 20–30, 30+ km/h. Challenging scenarios include $\tilde{X}$9, $m$0, Constant vs. Change ego-speed, and Straight vs. Turn, where turning is defined by yaw change greater than 5° and changing speed by acceleration magnitude exceeding 0.3 m/s².

The paper’s central metric is scaled bounding-box MSE, denoted sB_MSE, which normalizes per-sequence prediction error by adjusted pedestrian box area to remove scale bias. Occlusion and truncation are handled through a dataset-wide width/height ratio $m$1. The motivation is that absolute pixel-space metrics such as B_MSE, C_MSE, and CF_MSE can be misleading in egocentric views because the same pixel error corresponds to different metric distances at different pedestrian scales.

The associated model, ENCORE, is fully attention-based and combines step-wise hierarchical fusion, explicit ego-motion modeling, a scaled future trajectory auxiliary task, and an observation reconstruction auxiliary task. The paper reports that the approach achieves up to 40% improvement in challenging scenarios. Representative comparisons include scale 0–50 px, where BiTraP 134/0.220, PedFormer 76/0.125, and ENCORE-D 55/0.091 are reported for B_MSE/sB_MSE; speed 30+ km/h, where BiTraP 386/0.190, PedFormer 225/0.111, and ENCORE-D 145/0.071 are reported; and Turning, where PedFormer has approximately 0.147 and ENCORE-D approximately 0.109 in sB_MSE. On the full PIE benchmark, ENCORE-D reports B_MSE 251, C_MSE 222, CF_MSE 805, sB_MSE 0.023, outperforming PedFormer’s 295/265/943/0.027 (Rasouli, 2023).

This benchmark contributes a different notion of robustness from (Liu et al., 1 Oct 2025). Instead of modeling noisy observation histories directly, it reveals performance heterogeneity by conditioning evaluation on interpretable scenario variables such as scale, ego-speed, and turning.

7. Visibility-aware tracking, open issues, and the broader significance of EgoTraj-Bench

The paper "EgoPoints: Advancing Point Tracking for Egocentric Videos" is not itself named EgoTraj-Bench, but it provides methodological components that the authors explicitly propose for an egocentric trajectory benchmark (Darkhalil et al., 2024). EgoPoints targets dense point tracking in unconstrained egocentric video and introduces a visibility taxonomy that separates in-view, out-of-view, and occluded states, together with a dedicated ReID condition for points that leave the field of view and later return.

Its annotated benchmark contains 517 sequences and 4,703 tracks, including 875 out-of-view tracks and 593 ReID tracks. Compared with TAP-Vid-DAVIS, it contains 9× more out-of-view points and 59× more points requiring ReID, with sequences nearly 8× longer on average. Metrics include $m$2 over thresholds $m$3, a relaxed $m$4 over $m$5, ReID$m$6, IVA, OOVA, OA, and MTE. The authors also introduce the semi-real K-EPIC training pipeline with 11K sequences and approximately 22.1M tracks, and show that fine-tuning improves CoTracker and PIPs++ on EgoPoints (Darkhalil et al., 2024).

Across the different EgoTraj-Bench formulations, several limitations recur. Platform-specific sensing matters: (Liu et al., 1 Oct 2025) notes that camera height, field of view, and odometry quality can shift the noise distribution; (Krishnan et al., 30 Sep 2025) shows that time-varying intrinsics and extrinsics are major failure sources in wearable SLAM; (Yehia et al., 18 May 2026) identifies abrupt near-instantaneous turns, sensor drift, viewpoint instability, and participant variability as persistent difficulties; and (Rasouli, 2023) emphasizes long-tail scenario sparsity and dependence on reliable ego-motion and state labels. Availability is also uneven: (Liu et al., 1 Oct 2025) states that file formats, schema details, licensing, and a direct URL for EgoTraj-Bench distribution are not specified in the text, whereas (Yehia et al., 18 May 2026) and (Krishnan et al., 30 Sep 2025) provide public repositories or websites.

Taken together, the literature shows that EgoTraj-Bench has evolved into a broader designation for benchmarks that evaluate trajectory reasoning under egocentric sensing. In one line of work it means robust FPV-to-BEV forecasting under perceptual noise; in another, multimodal urban navigation prediction with gaze and head pose; in another, city-scale VI-SLAM with survey-grade control points; and in another, scenario-stratified egocentric pedestrian prediction. The unifying technical theme is that egocentric trajectory problems are dominated not by idealized clean histories but by realistic visibility loss, ego-motion, calibration drift, and multimodal intent cues.