EgoX: Cross-Domain Egocentric Video QA

• EgoX is a benchmark for cross-domain egocentric video QA that challenges models with severe domain shifts by focusing on temporal reasoning, domain-specific terminology, and dynamic interactions.
• The benchmark curates 798 video clips and 957 QA pairs from 5 datasets across 4 specialized domains, including surgery, industry, extreme sports, and animal perspectives.
• EgoX employs dual QA formats (OpenQA and CloseQA) with zero-shot evaluation to reveal significant performance degradation when models trained on daily-life scenarios are faced with specialized content.

Searching arXiv for the benchmark and closely related egocentric video QA work to ground the article with current citations. EgoX is the informal name used for EgoCross, a benchmark for cross-domain egocentric video question answering that evaluates whether multimodal LLMs (MLLMs) can generalize beyond everyday first-person activities such as cooking and cleaning to domains that differ substantially in both visual style and semantic content. EgoX covers surgery, industry, extreme sports, and animal-perspective video, and it pairs each question with both OpenQA and CloseQA formulations. Its central finding is that models that perform well on in-domain egocentric benchmarks degrade markedly under domain shift, including models specifically tailored to egocentric video (Li et al., 14 Aug 2025).

1. Definition, scope, and naming

In the benchmark literature, EgoX denotes the EgoCross benchmark rather than a separate framework. EgoCross is explicitly designed to measure the cross-domain generalization gap in EgocentricQA: the target setting is first-person video where both appearance and semantics depart from daily-life data, requiring robust identification, localization, prediction, and counting under domain shift (Li et al., 14 Aug 2025).

The benchmark is motivated by a mismatch between prior EgocentricQA resources and deployment settings. Existing datasets and studies are described as being concentrated on common daily activities such as cooking, eating, and gardening, whereas real applications involve specialized knowledge structures, domain-specific terminology, unusual camera motion, and interaction patterns. EgoCross therefore frames cross-domain EgocentricQA as a robustness problem rather than a simple extension of daily-life egocentric understanding.

A common source of confusion is the name itself. In adjacent literature, “X-Ego” refers to a different project on synchronized multi-agent esports video and “EgoX” can also denote a separate exocentric-to-egocentric video generation framework. Those works address cross-egocentric team modeling and video synthesis, respectively, rather than cross-domain EgocentricQA (Wang et al., 22 Oct 2025, Kang et al., 9 Dec 2025).

2. Dataset composition and curation

EgoX comprises 798 egocentric video clips, 957 QA pairs, and 5 source datasets spanning 4 domains, with an average clip duration of 22.5 seconds (Li et al., 14 Aug 2025).

Domain	Sources	Statistics
Surgery	CholecTrack20; EgoSurgery	212 clips, 283 QA
Industry	ENIGMA-51	176 clips, 245 QA
Extreme Sports	ExtremeSportFPV	242 clips, 246 QA
Animal Perspective	EgoPet	168 clips, 183 QA

The domain breakdown is more granular. CholecTrack20 contributes 112 clips, 183 QA, average 29.7s; EgoSurgery contributes 100 clips, 100 QA, average 20.4s; ENIGMA-51 contributes 176 clips, 245 QA, average 16.5s; ExtremeSportFPV contributes 242 clips, 246 QA, average 13.7s; and EgoPet contributes 168 clips, 183 QA, average 31.5s. The selected domains are described as application-oriented and high-impact, spanning healthcare and industrial operations, rare environments, high-speed camera motion, and non-human motion and interaction patterns that challenge anthropocentric bias and standard temporal-spatial reasoning.

The curation pipeline has three explicit stages. First, meta annotation refinement unifies and manually reviews original dataset annotations such as bounding boxes and temporal segments. Second, QA template design begins with 8 human-authored templates—two per core task category—and expands them to 15 domain-specific subtasks using Gemini 2.5 Pro; every LLM-generated question is then verified by human annotators for clarity, logic, and answerability. Third, batch instantiation and quality control samples clips per template with predefined durations and derives ground truth via programmatic reasoning over the crop. Each QA pair is released in both CloseQA and OpenQA form.

Temporal handling is intentionally permissive. Inference uses frames extracted at 0.5 fps—or dataset-provided sampling for some sources—and the benchmark imposes no maximum frame limit, allowing models to process full temporal context. This design is consequential because EgoX emphasizes temporal localization and prediction rather than static-frame recognition alone.

3. Task design and evaluation protocol

EgoX organizes question answering into four core task categories, further decomposed into 15 subtasks: Identification, Localization, Prediction, and Counting. The abstract summarizes these as prediction, recognition, localization, and counting (Li et al., 14 Aug 2025).

Identification targets recognition of domain-specific objects, actions, and events, such as surgical instruments, repair steps, or sports maneuvers. Localization requires precise spatial or temporal localization of objects, interactions, or events. Prediction asks the model to forecast next actions or outcomes from current content, such as the next surgical step or an anticipated sports move. Counting requires tracking instances or occurrences over time.

The task interface is uniform. Input consists of a video clip, represented by sampled frames, and a question; CloseQA additionally supplies candidate options. Output differs by format. In CloseQA, the model selects one option from multiple choices; prompts request a structured JSON containing “prediction” and reasoning, and the final answer is the option letter. In OpenQA, the model returns a free-form textual “prediction” answer.

The main evaluation protocol is zero-shot, single-round inference over the entire benchmark, with no training on EgoCross for main results. Decoding is deterministic, using do_sample=False (greedy). Video frames are sampled at 0.5–1 fps, and there is no frame cap. A 70/30 split (train/test) exists only for pilot studies in supervised and RL ablations and is explicitly not part of the main zero-shot benchmark.

For CloseQA, the metric is accuracy, defined as

$$\mathrm{Accuracy} = \frac{1}{N}\sum_{i=1}^{N} \mathbf{1}(\hat{y}_i = y_i).$$

The benchmark does not report top-k accuracy, F1, mAP, MAE, MSE, or MAPE. Counting is also evaluated with accuracy, because counts are treated as a classification target rather than a regression target. For OpenQA, evaluation is two-stage: exact match, followed—if no match exists—by LLM-as-judge semantic checking with Qwen-Max using a standardized judging prompt (Li et al., 14 Aug 2025).

4. Empirical performance and the cross-domain generalization gap

The benchmark evaluates proprietary MLLMs, open-source general-purpose MLLMs, and egocentric-specialized models. All receive video frames only; no audio is reported. The overall results establish that most systems remain far from robust under domain shift (Li et al., 14 Aug 2025).

Model	CloseQA	OpenQA
Gemini 2.5 Pro	52.95	34.40
GPT-4.1	52.63	26.65
Qwen2.5-VL-7B	44.82	20.41
InternVL3-8B	42.58	17.87
VideoLLaMA3-7B	42.03	18.76
Qwen2.5-VL-3B	37.54	14.81
EgoGPT	30.66	15.97
EgoVLPv2	27.10	not evaluated

The domain-specific breakdown is uneven. Surgery is relatively strong for the best proprietary models: Gemini 2.5 Pro reaches 61.48 CloseQA and 42.40 OpenQA, while GPT-4.1 reaches 57.24 CloseQA and 39.58 OpenQA. Industry is identified as the most challenging domain: Gemini 2.5 Pro records 37.55 CloseQA and 24.49 OpenQA, while GPT-4.1 records 45.71 CloseQA and 12.24 OpenQA; many other models are below 41% CloseQA and below 22% OpenQA. In Extreme Sports, both GPT-4.1 and Gemini 2.5 Pro remain near the low forties in CloseQA and low twenties in OpenQA, while open-source models often fall to single-digit or low-teen OpenQA. Animal Perspective is comparatively easier, with Gemini 2.5 Pro at 68.85 CloseQA and 49.18 OpenQA, and GPT-4.1 at 64.48 CloseQA and 34.43 OpenQA.

Several capability-level findings are diagnostically important. Prediction (OpenQA) is particularly difficult; one reported case shows Gemini 2.5 Pro reaching 62.50% in surgery while many other models remain in single digits. Localization exhibits strong brittleness: in Animal Perspective localization OpenQA, Gemini 2.5 Pro scores 42.42%, whereas GPT-4.1 scores 0% in that setting because it follows the instruction incorrectly, referring to frame indices rather than the required timestamps. Counting can look less challenging in CloseQA for some open-source systems, but the same models often deteriorate in OpenQA.

Because random chance for CloseQA is approximately 25%, a substantial fraction of scores on harder domains are only modestly above chance. The benchmark therefore diagnoses not merely imperfect calibration, but weak transfer of temporal reasoning and domain semantics.

The comparison with EgoSchema makes the transfer failure explicit. On aligned task types, Qwen2.5-VL drops from 73.58% on in-domain EgoSchema to 43.14% on cross-domain EgoCross. The decline is sharper on temporally sensitive tasks: temporal localization falls from 92.31% to 34.13%, and next action prediction falls from 85.71% to 37.50%. At the overall dataset level, the comparison reported is 69.60% (EgoSchema) versus 44.31% (EgoCross). Complementing these metrics, t-SNE of CLIP features shows clear distribution shifts in both visual and textual embeddings between EgoSchema and EgoCross domains (Li et al., 14 Aug 2025).

5. Adaptation studies and improvement attempts

EgoX includes three pilot improvement studies built on Qwen2.5-VL-7B and a 70/30 train/test split created from initial test QA pairs. These studies are explicitly labeled as pilots rather than benchmark-defining results (Li et al., 14 Aug 2025).

The first pilot is prompt learning. The method uses a two-part prompt: (1) domain-specific context/examples, (2) the video question. Under this setup, average CloseQA accuracy improves from 37.80 to 43.76, with a particularly large gain in Extreme Sports (52.78%). This suggests that part of the benchmark difficulty lies in missing domain priors rather than only missing visual perception.

The second pilot is supervised fine-tuning (SFT). The study performs full-parameter SFT on Qwen2.5-VL-7B using 4× H100 GPUs, per-device batch size 1, gradient accumulation 2, effective batch size 8, 12 epochs, and LR 1e-6. Optimization uses DeepSpeed ZeRO-2, BF16, gradient checkpointing, and Flash Attention 2. Video frames are sampled 4–16, longer sequences are truncated to 16, and inference settings are otherwise kept consistent. The reported gain is especially strong in Industry, where CloseQA rises from 35.71% to 52.86%; average CloseQA becomes 43.47.

The third pilot is reinforcement learning, using Generative Reward-based Policy Optimization (GRPO) without an SFT warm-up. Training uses 8× H100 GPUs, 16 epochs, LR 1e-6 with cosine schedule, DeepSpeed ZeRO-3, BF16, gradient checkpointing, and Flash Attention 2. RL-specific settings include reward-shaping beta 0.04 and 8 responses per prompt during training. This is the strongest reported intervention: average CloseQA rises from 37.80 to 60.12, with per-domain results of 49.40% for Surgery, 61.43% for Industry, 54.17% for Extreme Sports, and 75.47% for Animal Perspective.

The paper does not provide explicit SFT or RL objective formulas such as PPO or GRPO losses, KL penalties, or reward models. Only training configurations and hyperparameters are reported. Even so, the ablations collectively indicate that domain-aware prompting, supervision, and reward-driven adaptation can substantially reduce the cross-domain gap, with RL showing the largest effect under the reported setup.

6. Significance, limitations, and adjacent research directions

EgoX is presented as the first benchmark explicitly focused on cross-domain EgocentricQA. Its distinguishing properties relative to prior resources such as EgoVQA, EgoTaskQA, EgoSchema, EgoThink, and EgoTempo are its cross-domain coverage, emphasis on temporal tasks, and provision of dual QA formats for every instance (Li et al., 14 Aug 2025).

Its limitations are also explicit. Coverage is restricted to four application-oriented domains and roughly 1,000 QA pairs; broader coverage, larger scale, and multi-sensor inputs remain open directions. Although original annotations were refined and LLM-generated questions were human-verified, the benchmark still depends on available metadata and on the constraints of programmatic grounding. OpenQA evaluation relies on LLM-as-judge, which standardizes semantic checking but does not eliminate the need for more robust automatic generative evaluation.

One of the benchmark’s most consequential findings is that egocentric-specialized models underperform despite prior egocentric training. That result argues against a simple equation of “egocentric pretraining” with “domain robustness.” A plausible implication is that future progress will require domain-aware pretraining and adaptation strategies, not merely more exposure to first-person video.

Adjacent work broadens the surrounding landscape but does not solve the same problem. EgoExoMem studies synchronized ego–exo memory reasoning over paired views rather than cross-domain QA under severe semantic shift (Liu et al., 18 May 2026). EgoVITA studies plan-then-verify reasoning for egocentric video, coupling an egocentric plan with exocentric verification, but it is a reasoning framework rather than a cross-domain benchmark (Kulkarni et al., 23 Nov 2025). The later generative framework titled “EgoX: Egocentric Video Generation from a Single Exocentric Video” addresses exocentric-to-egocentric synthesis, not question answering (Kang et al., 9 Dec 2025).

Within egocentric video understanding, EgoX therefore occupies a specific position: it is a diagnostic benchmark for domain-robust, temporally grounded, instruction-following MLLMs. Its principal contribution is not merely to add new videos or harder questions, but to operationalize a deployment-relevant failure mode—generalization beyond daily life—and to quantify that failure with standardized zero-shot evaluation, dual QA formats, and targeted adaptation studies (Li et al., 14 Aug 2025).