Personal AI Agent for Camera Roll VQA

Abstract: We study the personal camera roll visual question answering setting. In this setting, a conversational AI assistant can access a user's personal camera roll and retrieve relevant photos to answer queries, ranging from simple factual questions (e.g., Name of the food I tried yesterday?'') to more open-ended ones (e.g.,Recommend some dishes I have never eaten before''). Given the vast nature of the personal camera roll (i.e., multiple years, hundreds to thousands of photos), a successful AI assistant needs to understand a long-horizon, highly personalized visual content stream in order to navigate and locate the correct and/or relevant information. To support this, we collect and manually annotate questions that mimic real-world usage. The final dataset, camroll, contains 50 users, 31,476 images, and 2,500 QA pairs. We further design camroll-agent, a conversational AI agent equipped with hierarchical memory and a minimal set of tools for efficient navigation over large, personalized visual memory. Experimental results show that camroll-agent outperforms numerous baselines and methods for long-context understanding AI agents system. Together, the camroll dataset and camroll-agent highlight the gap in AI agents' long-context reasoning: personalized visual memory requires different approaches from standard long-context textual memory, especially when consistency, visual details, and user-specific context are present.

• The paper presents a novel camroll dataset and agent architecture for personalized VQA over temporally-rich, user-specific photo streams.
• It employs a hierarchical memory design that integrates raw images, personalized captions, and event-based summaries to enhance context-aware retrieval.
• Experimental results demonstrate significant performance improvements over baselines, highlighting the need for tailored tools and privacy-preserving mechanisms.

Personal AI Agent for Camera Roll VQA: Benchmarking Personalized Long-Horizon Visual Memory Reasoning

Motivation and Problem Formulation

This paper introduces the camroll dataset and camroll-agent, targeting the challenge of visual question answering (VQA) over personal camera rolls—a domain characterized by large-scale, temporally anchored, and user-specific image collections. Existing commercial systems (e.g., Google Gemini, Apple Intelligence) and academic approaches either treat images as isolated instances or rely on basic retrieval paradigms, failing to capture deeper personalized reasoning and narrative structure inherent in personal image streams. The introduction of camroll is motivated by the need for a benchmark that reflects realistic, user-centric questions with diverse answer distributions and complex evidence grounding.

Figure 1: AI assistant VQA over personal camera roll, supporting personalized photo search and contextualized answers.

Dataset Construction and Personalization Analysis

camroll comprises 31,476 images and 2,500 QA pairs from 50 real users, drawing from both in-house smartphone collections and curated subsets of YFCC-100M. Data selection strictly enforces criteria on volume, temporal span, and personal-life signal. Annotation prioritizes human-authored questions that simulate lived experiences, covering semantic and episodic memory facets.

Personalization is quantitatively validated via embedding-level kNN purity and value-level answer uniqueness. Episodic questions exhibit 16.5% purity (well above the random baseline), reflecting highly user-specific linguistic patterns. Value-level analysis reveals 90.2% of gold answer strings are unique to a single user. Compared with VQA and LLaVA, camroll showcases a substantially heavier-tailed answer distribution, indicating the necessity for retrieval directly over user-specific memory rather than generic latent space.

Figure 2: Geographic and temporal coverage of camroll; in-house smartphone users generate denser, less curated photo streams than traditional digital camera users.

Hierarchical Memory Design and Agent Architecture

camroll-agent operationalizes hierarchical personal memory as a three-level pyramid: raw images, personalized captions, and event-based summaries, with cross-linked storage for O(1) navigation. Caption generation is conditioned on user identity and local chronological context, enabling first-person grounding and improved disambiguation. Event segmentation leverages prompt-driven incremental LLM processing to build episodes reflecting real-life activities.

Figure 3: Hierarchical memory for personal camera rolls, organizing images, captions, and events for adaptive retrieval and reasoning.

The agent interacts through a minimal set of domain-specific tools, carefully decomposed along retrieval paradigm and access granularity axes. Tool inventory includes semantic (search), lexical (grep), and metadata-based (list) retrieval, complemented by get and view for full-record inspection and raw pixel analysis. This design enables efficient memory navigation on large-scale, user-centric collections, mitigating common inefficiencies in generic agents.

Experimental Evaluation and Comparative Analysis

Quantitative benchmarks demonstrate camroll-agent outperforms all baselines—including naive MLLMs (plain context injection), RAG-based retrieval, and memory-layer systems—on both multiple-choice and freeform QA. Domain-specific agent tooling is shown to be crucial: ClaudeCode, repurposed from coding settings, incurs high token usage due to reliance on exhaustive visual inspection, whereas camroll-agent efficiently allocates budget to semantic retrieval (53.6% of tool calls), reducing unnecessary image views.

Figure 4: Distribution of camroll-agent tool calls across question types and interaction turns.

Ablation studies verify the importance of hierarchical memory: removing captions or event structuring degrades performance substantially. All tools provide meaningful contributions, with search being most impactful. Closed-source MLLMs (Gemini-3.1-Pro, GPT-5.2) lead in performance, but recent open-source models (Qwen3-VL-8B-Instruct) approach competitive freeform Judge scores, suggesting emerging viability for local, privacy-preserving deployments.

Error analysis categorizes failures and shows most originate from inefficient agent tool usage (exploration, overconfidence, step budgeting) rather than VLM capability, indicating the necessity for targeted post-training and joint retrieval-agent optimization.

Practical Implications and Theoretical Insights

The results reveal that conventional RAG and memory-augmented paradigms for long-context reasoning are insufficient for highly personalized visual question answering. Domain-specific agent architectures and hierarchical memory schemes are critical for efficient, accurate retrieval and reasoning at scale. The heavy-tailed answer distributions and episodic structure of real user camera rolls expose limitations in current multimodal systems, particularly in compositional and longitudinal QA.

For practical AI systems, this work underscores the need to jointly optimize agent reasoning, memory representation, and tool integration, tailored to the unique demands of personal visual streams. Privacy and user-control considerations are paramount as highly personalized and sensitive content is involved, motivating research directions in privacy-preserving retrieval and adaptive consent management.

Future Directions

Future work should focus on learning-driven memory construction, joint fine-tuning of agent reasoning and retrieval modules, and explicit modeling of cross-modal, cross-event dependencies. There is scope for benchmarking broader agent capabilities (e.g., consistent storytelling, persistent memory updates) and exploring scalable local deployment using open-source MLLMs. Additionally, research should prioritize privacy architectures, including memory sandboxing, consent tracking, and user-specified memory constraints.

Conclusion

This paper establishes camroll as a robust benchmark for personalized long-horizon visual QA, and camroll-agent as a reference agent architecture for memory-intensive reasoning in personal camera rolls. The hierarchical memory, domain-tailored tools, and agentic interaction protocols together define a new direction for personalized multimodal AI systems, supporting complex, context-aware applications grounded in real user memory.