EgoMonth: A Month-Level Egocentric Video Benchmark for Long-Term Spatiotemporal Memory

Abstract: Recent advances in Multimodal LLMs (MLLMs) have led to substantial progress in video understanding, accompanied by a growing number of long video benchmarks. However, existing benchmarks rely predominantly on web-sourced videos that lack inter-clip spatiotemporal continuity, making it difficult to assess whether models can maintain consistent memory across days or weeks of real-world experience. We introduce EgoMonth, the first month-level egocentric video understanding benchmark. EgoMonth comprises over 300 hours of first-person daily-life recordings from 20 participants spanning 20 to 120 days, paired with 1,443 human-crafted multiple-choice question-answer pairs. We design a cognitively grounded 14-task evaluation framework organized into three hierarchical cognitive levels: Schema Consolidation, Episodic Indexing, and Cascading Reasoning. Evaluation of state-of-the-art open-source and closed-source MLLMs reveals that even the best-performing model, Gemini 2.5 Pro, achieves only 71.8% macro-average accuracy, remaining 22.4 percentage points below the corrected human baseline of 94.2%. Several models perform near or below the 25% chance level on tasks such as Route Reasoning, Cross-view Spatial Reasoning, and Direction Judgement, while even the strongest closed-source model remains substantially below human performance. These results indicate that current MLLMs function as lossy summarizers rather than faithful memorizers, highlighting the need for architectures with genuine long-term spatiotemporal memory.

• The paper introduces EgoMonth, the first month-level egocentric video benchmark, containing 738 clips, 301 hours of footage, 20 participants, and 1,443 human-verified multiple-choice questions spanning 20–120 days.
• The evaluation shows that Gemini 2.5 Pro leads with 71.8% average accuracy but remains 22.4 percentage points below humans, while model performance declines from schema consolidation to episodic indexing and cascading reasoning.
• The paper finds that higher frame density and larger model size do not reliably solve long-term memory, motivating selective retrieval, event-level temporal indexing, persistent object tracking, map-like spatial representations, and intermediate-state verification.

Motivation and positioning

Existing long-video benchmarks—Video-MME, LongVideoBench, MLVU, LVBench, HLV-1K, HourVideo—are built from web videos, movies, or isolated recording sessions in which clips do not share persistent environments, recurring routines, or cross-day dependencies. Egocentric corpora such as Ego4D and Ego-Exo4D capture rich first-person interaction but lack per-subject temporal depth, and EgoLife extends only to one week of cohabitation. As a result, the question of whether current MLLMs can maintain consistent memory across days or weeks of real-world experience has remained untested. EgoMonth addresses this gap as the first month-level egocentric video understanding benchmark, emphasizing per-participant continuity and cross-video reasoning over 20–120 day spans.

Dataset construction

The benchmark comprises 738 video clips totaling approximately 301 hours (18,072 minutes) from 20 participants, with per-participant spans of 20 to 120 days and an average clip duration of about 24.5 minutes. From an initial pool of 30 volunteers and over 400 hours of raw footage, quality screening on visual quality, temporal continuity, viewpoint stability, and activity diversity removed low-quality segments; all retained video is at least 1K resolution at 25 fps, captured across heterogeneous devices (smartphones, GoPro, Insta360, DJI). Privacy protection uses a pipeline combining Grounding DINO 1.5 for sensitive-region detection and SAM 2 for instance segmentation, followed by manual inspection—a necessary step given that month-long first-person footage inevitably captures bystanders and private content.

Annotation yields 1,443 four-option multiple-choice QA pairs, all human-crafted without LLM generation, verified through three-annotator cross-review under an unambiguous-answer principle. Distractors follow four systematic categories: temporal confusion, spatial confusion, entity substitution, and quantity perturbation. Notably, a subset of questions is cross-video, requiring evidence integration across recordings separated by days or weeks—an evaluation dimension absent from all prior long-video benchmarks.

Task taxonomy

The 14 tasks are organized into three cognitively grounded levels:

• Level 1 — Schema Consolidation: Habit Inference and Personality Inference, probing whether models can extract stable behavioral patterns tolerant to local feature loss.
• Level 2 — Episodic Indexing: Detail Retrieval, Spatial Relation, Self-localization, Temporal Ordering, Event Time, and Object Location, requiring precise retrieval of sparse, non-redundant evidence where retrieval failure produces confident but wrong answers.
• Level 3 — Cascading Reasoning: Procedure Planning, Event Counting, Object Counting, Route Reasoning, Cross-view Spatial Reasoning, and Direction Judgement, involving dependency-sensitive composition of multiple memory units where single-cue errors cascade.

The QA distribution is uneven: Detail Retrieval dominates at 19.9% of pairs, while Cross-view Spatial Reasoning (2.1%) and Direction Judgement (2.5%) are comparatively scarce—a point relevant when interpreting macro-average scores.

Evaluation results

Twelve MLLMs were evaluated under a unified multiple-choice protocol without LLM-as-judge, against a human baseline of 94.2% macro-average accuracy (Fleiss' $\kappa = 0.78$ among three independent annotators).

Model	Avg (%)	Acc (%)
Gemini 2.5 Pro	71.8	72.6
Qwen2.5-VL (32B)	58.0	60.8
MiniCPM-V 4.5	56.0	60.6
Qwen3-VL-30B-A3B	53.0	56.7
Qwen3-VL (8B)	51.4	53.7
VITA-1.5	51.3	53.6
VideoLLaMA3	50.3	53.1
Chat-UniVi-V1.5	39.5	39.9
ST-LLM	38.8	40.8
ShareGPT4Video	37.1	36.9
Human	94.2	95.1

Three findings stand out. First, performance degrades monotonically with cognitive level: models are strongest on Schema Consolidation and weakest on Cascading Reasoning, confirming that multi-evidence spatiotemporal composition is the primary bottleneck. Second, several open-source models fall at or below the 25% chance level on specific tasks—ST-LLM scores 8.0% on Event Counting and 20.0% on Cross-view Spatial Reasoning, and Chat-UniVi-V1.5 scores 8.8% on Event Counting—indicating systematic failure rather than random guessing. Third, even Gemini 2.5 Pro remains 22.4 percentage points below humans overall, with particularly large deficits on Direction Judgement (58.3% vs. 86.1%) and Cross-view Spatial Reasoning (60.0% vs. 93.3%).

Diagnostic analysis

The paper's analysis yields three claims with architectural implications.

Frame density does not determine memory fidelity. VITA-1.5, using only 16 frames, achieves 41.6% on Event Counting, exceeding Qwen2-VL's 36.8% despite the latter consuming 256 frames. The authors attribute this to temporal attention dilution: without effective evidence selection, redundant visual tokens obscure decisive moments, and dense sampling increases the burden of cross-frame correspondence modeling, causing repeated views of one event to be miscounted as independent occurrences.

Scale helps integration but not indexing. Larger models such as Qwen2.5-VL (32B) perform relatively well on Level 3 tasks (78.6% Procedure Planning), yet Level 2 tasks depend on precise temporal localization rather than reasoning capacity; retrieving a visually similar but incorrect episode yields confident errors regardless of parameter count.

Structured spatiotemporal representations are missing. The central failure mode is the absence of persistent temporal indices and stable spatial structures. Models retrieve semantically related events assigned to wrong days, orders, or occurrences, and egocentric viewpoint-dependence compounds this for Self-localization, Route Reasoning, and Direction Judgement. The authors argue for event-level temporal indexing, persistent object-state tracking, map-like spatial representations, and intermediate-state verification—concrete design targets rather than generic scaling prescriptions.

Limitations

The paper concedes two principal limitations. The participant pool of 20 constrains demographic coverage and precludes fine-grained subgroup analysis. Additionally, the 20–120 day span leaves seasonal and year-scale behavioral change untested; whether the observed failure patterns intensify or stabilize at longer horizons remains an open empirical question. A further caveat implicit in the results is the small QA counts for several Level 3 tasks (e.g., 30 questions for Cross-view Spatial Reasoning), which makes per-task comparisons noisy even though they motivate some of the strongest claims.

Conclusion

EgoMonth establishes a month-level egocentric benchmark demonstrating that current MLLMs operate as lossy summarizers rather than faithful memorizers of extended real-world experience. With a 22.4-point gap between the best model and humans, near-chance performance on spatial and counting tasks, and evidence that neither frame density nor parameter scale resolves the deficit, the benchmark provides both a diagnostic framework and a concrete set of architectural requirements—selective retrieval, accurate temporal indexing, and structured spatiotemporal memory—for future video MLLMs.