EgoCross: Cross-Domain Egocentric Video Benchmark

• EgoCross is a research area and benchmark focused on transferring and aligning knowledge across egocentric visual domains such as surgery, extreme sports, industry, and animal perspectives.
• It evaluates multimodal models on QA tasks—including prediction, localization, and counting—highlighting challenges from significant view and semantic domain shifts.
• Emerging methods like reflective dialogue and view-invariant learning are used to boost cross-view generalization in specialized egocentric video applications.

EgoCross is a term used in contemporary egocentric-vision research for closely related problems of transferring, aligning, or evaluating knowledge across visual domains centered on first-person video. In the most specific sense, it denotes a benchmark for cross-domain egocentric video question answering (EgocentricQA) covering surgery, industry, extreme sports, and animal perspective (Li et al., 14 Aug 2025). In a broader sense, the term appears in work on exocentric-to-egocentric dense video captioning, exocentric-to-egocentric video synthesis, and asynchronous ego/exo procedural understanding, all of which study how representations or predictions learned from one viewpoint can be carried into another (Ohkawa et al., 2023, Luo et al., 2024, Huang et al., 2024). A plausible implication is that EgoCross names an emerging research area rather than a single task family, unified by cross-view or cross-domain transfer under substantial visual and semantic shift.

1. Terminological scope and research setting

The most explicit use of the term appears in "EgoCross: Benchmarking Multimodal LLMs for Cross-Domain Egocentric Video Question Answering" (Li et al., 14 Aug 2025). There, EgoCross is a benchmark designed to evaluate the cross-domain generalization of Multimodal LLMs (MLLMs) in EgocentricQA, with four target domains: surgery, industry, extreme sports, and animal perspective. The benchmark comprises approximately 1,000 QA pairs across 798 video clips and spans four key QA tasks: prediction, recognition, localization, and counting (Li et al., 14 Aug 2025).

The same label is also used more broadly in other sources. "Exo2EgoDVC: Dense Video Captioning of Egocentric Procedural Activities Using Web Instructional Videos" describes EgoCross as cross-view dense video captioning from exocentric to egocentric video, emphasizing knowledge transfer from YouCook2 to EgoYC2 (Ohkawa et al., 2023). "Put Myself in Your Shoes: Lifting the Egocentric Perspective from Exocentric Videos" states, “We call this exo→ego mapping ‘EgoCross’,” to emphasize crossing from a third-person view to a first-person view without relying on calibrated cameras or full 3D reconstruction (Luo et al., 2024). "EgoExoLearn" similarly frames EgoCross as cross-view mapping between egocentric and exocentric perspectives for association, planning, and skill assessment (Huang et al., 2024).

This distribution of usages suggests that EgoCross is not restricted to a single benchmark or architecture. Rather, it indexes a class of problems in which the central difficulty is a large mismatch between source and target viewpoints or domains. In the data provided here, that mismatch is instantiated as exo-to-ego translation, ego/exo retrieval, cross-view caption transfer, and cross-domain video QA.

2. EgoCross as a benchmark for cross-domain egocentric video question answering

In its benchmark form, EgoCross addresses real-world egocentric video applications that “inevitably encounter domain shifts far beyond ‘daily-life’ footage” (Li et al., 14 Aug 2025). The benchmark sources are domain-specific: EgoSurgery and CholecTrack20 for surgery, ENIGMA-51 for industry, ExtremeSportFPV for XSports, and EgoPet for animal perspective (Li et al., 14 Aug 2025). The reported challenges are likewise domain-specific: fine-grained tool differentiation and structured procedural logic in surgery; tiny components and precise spatial reasoning in industry; rapid camera motion and motion blur in extreme sports; and non-human viewpoints with erratic motion in animal-borne video (Li et al., 14 Aug 2025).

The benchmark design formalizes four task categories and 15 total subtasks. Identification includes object identification and action/event identification; localization includes spatial and temporal localization; prediction targets next-action or outcome prediction; and counting requires tracking and counting occurrences over time (Li et al., 14 Aug 2025). Each QA pair is provided in both CloseQA and OpenQA formats, supporting both multiple-choice and free-form evaluation (Li et al., 14 Aug 2025).

The dataset composition is precisely specified as 798 video clips with average duration 22.5 s and 957 QA pairs. The per-domain breakdown is Surgery: 212 clips and 283 QA; Industry: 176 clips and 245 QA; XSports: 242 clips and 246 QA; Animal Perspective: 168 clips and 183 QA (Li et al., 14 Aug 2025). This same 798-clip, 957-question substrate is also the basis of the EgoCross challenge setting used by the reflective-dialogue technical report, which describes the dataset as spanning 15 subtasks in four categories: identification, localization, prediction, and counting (Murakawa et al., 27 May 2026).

A central feature of the benchmark is its emphasis on domain shift rather than merely egocentric perception. The benchmark description states that existing EgocentricQA datasets focus on common indoor/outdoor daily activities and do not test models on visually and semantically distinct, high-impact domains (Li et al., 14 Aug 2025). This establishes EgoCross as a diagnostic resource for evaluating whether MLLMs trained or tuned on common activities can generalize to specialized first-person environments.

3. Evaluation protocols, baselines, and observed generalization gaps

EgoCross evaluates models in a zero-shot, single-round, fully greedy decoding setting, with all frames sampled at 0.5 fps and no length limit (Li et al., 14 Aug 2025). For CloseQA, the metric is Top-1 accuracy. For OpenQA, evaluation uses exact match and, if there is no exact match, LLM-as-Judge for semantic equivalence (Li et al., 14 Aug 2025). The benchmark also specifies task-specific metrics: counting accuracy,

$$\mathrm{Accuracy}_{\mathrm{count}} = 1 - \frac{|\hat C - C|}{C},$$

localization IoU,

$\mathrm{IoU} = \frac{|P \cap G|}{|P \cup G|},$

and $F_1$ for extraction tasks (Li et al., 14 Aug 2025).

The evaluated models include proprietary MLLMs, open-source general MLLMs, and egocentric-specialized MLLMs. The listed systems are GPT-4.1, Gemini 2.5 Pro, Qwen2.5-VL (3B, 7B), VideoLLaMA3-7B, InternVL3-8B, EgoVLPv2, and EgoGPT (Li et al., 14 Aug 2025). Quantitative results show broad degradation under cross-domain conditions.

Model	Overall CloseQA / OpenQA
GPT-4.1	52.6 / 26.7
Gemini 2.5 Pro	53.0 / 34.4
Qwen2.5-VL-7B	44.8 / 20.4
VideoLLaMA3-7B	42.0 / 18.8
InternVL3-8B	42.6 / 17.9
EgoVLPv2	27.1 / —
EgoGPT	30.7 / 16.0

The reported observations are that overall CloseQA is approximately $50\%$ for proprietary models, approximately $40\%$ for open-source models, and approximately $30\%$ for egocentric-specialized models, with random guess at $25\%$ for CloseQA (Li et al., 14 Aug 2025). OpenQA performance is weaker, with the best result approximately $35\%$ and many open-source models below $20\%$ (Li et al., 14 Aug 2025). The benchmark authors further state that domain shift severely degrades performance compared to EgoSchema, with an average drop of approximately $1.6\times$, and that egocentric-specialized models underperform general-purpose MLLMs (Li et al., 14 Aug 2025).

These findings define a core empirical meaning of EgoCross: it is a stress test for robustness under simultaneous shifts in visual appearance, semantic content, and domain terminology. The benchmark therefore functions less as a saturation benchmark and more as a diagnostic benchmark for failure under specialization.

4. Inference-time adaptation on EgoCross: Reflective Dialogue

"Reflective Dialogue between Teacher and Solver Agents for Video Question Answering" introduces an inference-time adaptation method evaluated on the EgoCross benchmark (Murakawa et al., 27 May 2026). The technical report highlights three constraints: drastic visual appearance shifts across domains, a small support set per domain, and the practical limitation that proprietary models often cannot be fine-tuned, so inference-time adaptation is required (Murakawa et al., 27 May 2026).

The method constructs a Reflective Dialogue (RD), defined as a multi-turn conversation between two agents. The Teacher presents support questions, provides correctness feedback, and prompts reflection. The Solver answers and then produces visual grounding explanations for both correct and incorrect answers (Murakawa et al., 27 May 2026). For each domain $\mathrm{IoU} = \frac{|P \cap G|}{|P \cup G|},$0 and question type $\mathrm{IoU} = \frac{|P \cap G|}{|P \cup G|},$1, support examples

$\mathrm{IoU} = \frac{|P \cap G|}{|P \cup G|},$2

are processed in a four-turn interaction: question presentation, answer selection, correctness feedback, and a 2–3 sentence reflection describing critical frames and, when incorrect, why the distractor seemed plausible (Murakawa et al., 27 May 2026).

The resulting dialogue history is concatenated into a domain- and type-specific warm-up history

$\mathrm{IoU} = \frac{|P \cap G|}{|P \cup G|},$3

At inference time, a test example $\mathrm{IoU} = \frac{|P \cap G|}{|P \cup G|},$4 forms the context

$\mathrm{IoU} = \frac{|P \cap G|}{|P \cup G|},$5

and the model computes

$\mathrm{IoU} = \frac{|P \cap G|}{|P \cup G|},$6

for each $\mathrm{IoU} = \frac{|P \cap G|}{|P \cup G|},$7 (Murakawa et al., 27 May 2026).

The report compares three settings: Zero-Shot, standard in-context learning (ICL), and RD. Zero-Shot uses only a domain-specific system prompt plus $\mathrm{IoU} = \frac{|P \cap G|}{|P \cup G|},$8; ICL prepends the raw support set as a flat list of $\mathrm{IoU} = \frac{|P \cap G|}{|P \cup G|},$9 triples; RD prepends the richer multi-turn dialogue history, while remaining a single-pass inference method without per-test-question trial loops (Murakawa et al., 27 May 2026).

For open-weight Qwen3-VL-4B-Instruct, the reported CloseQA accuracies are 0.424 overall for Zero-Shot, 0.468 for ICL, and 0.489 for RD, corresponding to +6.5 points for RD over Zero-Shot and +2.1 over ICL (Murakawa et al., 27 May 2026). Domain-wise, RD achieves 0.590 on Animal, 0.419 on XSports, 0.457 on Industry, and 0.512 on Surgery, with strongest gains in Surgery (Murakawa et al., 27 May 2026). For Gemini 3.1 Pro Preview, RD improves overall performance from 0.564 to 0.625, and RD + Timestamps reaches 0.659, a +9.5-point gain over zero-shot; context caching in Gemini Pro reduces token costs by approximately 49% (Murakawa et al., 27 May 2026).

The ablations are also revealing. RD outperforms ICL in XSports, Industry, and Surgery, whereas in Animal pure ICL is slightly better, suggesting that direct visual cues may suffice for straightforward recognition tasks (Murakawa et al., 27 May 2026). A 6-epoch LoRA fine-tuning baseline yields 51.4% overall, and FT + RD reaches 52.8%, while RD alone reaches 48.9%, approaching fine-tuning performance (Murakawa et al., 27 May 2026). The cited limitations are token length and over-reflection noise (Murakawa et al., 27 May 2026). In the EgoCross context, RD therefore operationalizes a specific adaptation regime: inference-time context injection using correctness-conditioned visual reflections.

5. Cross-view formulations associated with EgoCross

Outside the QA benchmark, EgoCross is associated with cross-view transfer between exocentric and egocentric observations. "Exo2EgoDVC" studies cross-view dense video captioning in which a model must detect temporal segments in a long video and generate a natural-language caption for each segment, while transferring knowledge from exocentric web instructional videos to egocentric recordings (Ohkawa et al., 2023). To isolate view-related domain shifts, EgoYC2 re-records a subset of YouCook2 recipes in real-life home kitchens with a head-mounted camera and shares exactly the same step descriptions and vocabulary as YouCook2 (Ohkawa et al., 2023). The method builds on PDVC and introduces a view-invariant adversarial learning scheme with pre-training on YouCook2 and fine-tuning on EgoYC2, using a view classifier trained through a Gradient Reversal Layer (Ohkawa et al., 2023).

The quantitative results show strong gains from view-invariant training. On the YouCook2 $F_1$0 EgoYC2 transfer setting, PT+FT with VC+HO obtains CIDEr 52.5, VI-PT + FT reaches CIDEr 55.2, and VI-PT + VI-FT reaches CIDEr 59.0, while SODA-CIDEr rises from 37.3 to 45.2 with full view-invariant learning (Ohkawa et al., 2023). The paper reports reduced irrelevant ingredients, fewer duplicate captions, and better segment boundaries as qualitative effects of the method (Ohkawa et al., 2023). In that work, EgoCross denotes a cross-view transfer problem grounded in matched textual supervision rather than question answering.

A more generative formulation appears in "Put Myself in Your Shoes: Lifting the Egocentric Perspective from Exocentric Videos" (Luo et al., 2024). There, the exo→ego mapping is explicitly called EgoCross. The task is to generate a first-person egocentric view of an actor from an exocentric video recording, approximating

$F_1$1

The proposed Exo2Ego framework decouples the problem into a high-level structure transformation stage and a diffusion-based pixel hallucination stage, with a hand layout prior injected at multiple U-Net scales (Luo et al., 2024). The benchmark consists of synchronized ego-exo tabletop activity pairs drawn from H2O, Aria Pilot, and Assembly101 (Luo et al., 2024). On the H2O new-action split, Exo2Ego achieves SSIM = 0.433, PSNR = 30.56 dB, FID = 38.0, LPIPS-VGG = 0.295, and Feasibility = 0.976, outperforming several baselines (Luo et al., 2024).

A third cross-view formulation appears in EgoExoLearn, which introduces a large-scale dataset for bridging asynchronous ego- and exo-centric procedural activities (Huang et al., 2024). The dataset contains 747 videos and approximately 120 hours of footage across eight procedural tasks, with multimodal annotations including actions, free-form descriptions, gaze, and skill rankings (Huang et al., 2024). The benchmarks include cross-view association, cross-view action planning, and cross-view referenced skill assessment. Cross-view association is cast as learning an embedding $F_1$2 with cosine similarity maximized for true ego/exo pairs under a symmetric contrastive InfoNCE objective; planning predicts future steps across views; skill assessment compares egocentric performances relative to an exocentric reference (Huang et al., 2024). The reported Top-1 association results improve from 49.0 / 45.3 without gaze to 55.3 / 51.1 with gaze for fine-tuned hybrid ego+exo training (Huang et al., 2024).

Taken together, these works show that EgoCross has been instantiated as caption transfer, video synthesis, association, planning, and referenced assessment. The commonality is not a fixed output format, but the requirement that a model bridge viewpoint-induced representation gaps.

6. Historical antecedents, methodological themes, and open problems

An earlier precursor to the broader EgoCross agenda is "EgoTransfer: Transferring Motion Across Egocentric and Exocentric Domains using Deep Neural Networks" (Ardeshir et al., 2016). That work constructed a dataset of 420 synchronized egocentric–exocentric video pairs across seven actions and studied retrieval of egocentric or exocentric motion features from the opposite view (Ardeshir et al., 2016). Motion was represented using HOOF and PCA-reduced C3D descriptors, and mappings were learned via linear regression, a non-linear reconstruction network, and a two-stream matching network (Ardeshir et al., 2016). Evaluation used area under the cumulative matching curve across ego→side, side→ego, ego→top, and top→ego retrieval tasks. The reported results already established that motion information can be transferred across views and that side-view mappings are generally easier than top-view mappings (Ardeshir et al., 2016).

Across the later literature, several recurring methodological themes can be identified directly from the data. One is view-invariant representation learning, realized through adversarial confusion in Exo2EgoDVC (Ohkawa et al., 2023), contrastive alignment in EgoExoLearn (Huang et al., 2024), and contrastive matching or reconstruction in EgoTransfer (Ardeshir et al., 2016). A second is explicit structural priors, such as the hand layout prior in Exo2Ego (Luo et al., 2024) and gaze cropping in EgoExoLearn (Huang et al., 2024). A third is inference-time adaptation under low-support conditions, exemplified by Reflective Dialogue on EgoCross (Murakawa et al., 27 May 2026). A fourth is evaluation under severe shift, which is central to the EgoCross benchmark itself (Li et al., 14 Aug 2025).

The limitations are likewise consistent. EgoCross benchmark results show that even top MLLMs remain below 55% CloseQA and below 35% OpenQA (Li et al., 14 Aug 2025). Reflective Dialogue reports token-length cost and over-reflection noise (Murakawa et al., 27 May 2026). EgoExoLearn notes that skill models benefit only marginally from exo reference and that bridging asynchronous views remains challenging (Huang et al., 2024). EgoTransfer acknowledges limits of scale and view diversity (Ardeshir et al., 2016). These observations suggest that the central difficulty is not only visual recognition but coordinated reasoning over viewpoint, temporal structure, and domain semantics.

Future directions stated in the sources include domain-adaptive pretraining with synthetic or real specialized data, retrieval-augmented MLLMs using external knowledge bases, unified instruction-tuning across domains, modular architectures that route features based on domain cues, contrastive learning across domains, style transfer, context compression for reflective prompts, automated grouping of question types, and hybrid schemes combining light domain-tuning with reflective prompts (Li et al., 14 Aug 2025, Murakawa et al., 27 May 2026). In the cross-view setting, the literature also points to better cross-view transformers, temporal alignment modules, and generative planning networks that convert exo demonstration semantics into ego-adapted action sequences (Huang et al., 2024).

Within the literature represented here, EgoCross therefore denotes a technically coherent but heterogeneous field: the study of how models can move between first-person and third-person views, or between familiar and specialized first-person domains, while preserving actionable semantic structure.