EGO-Prompt: Personalized Prompting Strategies

• EGO-Prompt is a suite of methodologies that uses computational ego as a principle for controlling and personalizing intelligent systems.
• It employs modules like ego-grounding, emotional and framework prompts, and graph-guided reasoning to improve LLM memory and contextual accuracy.
• Evolutionary optimization and domain-specific extensions boost performance, yet challenges like memory decay and saliency bias persist.

EGO-Prompt is a suite of methodologies that operationalize “ego” as a computational principle for controlling, conditioning, or guiding intelligent systems—most notably LLMs, multimodal LLMs (MLLMs), and deep learning architectures—in contexts requiring personalization, memory, or first-person semantic disambiguation. The term encompasses not just parameterized textual templates but structured, algorithmically-anchored prompting strategies that integrate modules for identity-grounding, temporal context, personalized memory, and domain graph reasoning. This article surveys core EGO-Prompt designs, evaluation metrics, empirical insights, and domain-specific adaptations, with emphasis on recent research in egocentric vision-and-language, scalable personalized QA, cross-view vision, and evolutionary prompt optimization.

1. Ego-Grounding in Personalized VideoQA

The concept of ego-grounding is operationalized as the explicit mapping of “I/my” to a fixed identity token (e.g., “CameraWearer”), the maintenance of a dynamic inventory of “my things” (objects touched), and a timestamped event log of “my past actions.” This framework is instantiated in the MyEgo dataset and protocol, where state-of-the-art MLLMs are evaluated for their ability to support first-person, context-dependent question-answering over egocentric video streams (Xiao et al., 2 Apr 2026).

Key components:

• Ego-Grounding Module: Maintains a static identity mapping and dynamic inventories (objects, actions) for first-person referential disambiguation.
• Context-Window Management: Samples key frames near the question and answer moments, summarizes distant past activity, and digests session context via sliding-window memory.
• Memory Retention: Employs hierarchical buffering (short-term, mid-term, long-term) with explicit memory injection to bridge long-range dependencies.
• Prompt Structure and Reasoning: Prompts force the LLM to follow a chain-of-thought sequence (e.g., recall → verify → answer), cite evidence, and refresh context, reducing saliency bias and referential drift.

Empirical evaluation on MyEgo shows that even top MLLMs (GPT-5, Qwen3-VL) perform at 46% and 36% ego-grounding accuracy, trailing human performance by 40–50%. Neither scaling nor explicit reasoning leads to uniform improvement; rather, the insertion of explicit memory interventions and identity reminders yields local gains but cannot eliminate memory decay. Saliency bias and mis-grounding remain principal failure modes, highlighting a core limitation of current architectures in personalized, temporally extended VideoQA (Xiao et al., 2 Apr 2026).

2. Prompting Paradigms: Emotional, Structural, and Graph-Guided

Beyond egocentric vision, advanced EGO-Prompt paradigms incorporate higher-order guidance via emotional stimuli (stimulating prompts) and causal-graph-based reasoning (framework prompts). The Auto-Prompt Graphical Paradigm (APGP) formalizes this in the context of LLMs as a staged, graph-organized pipeline:

• Stimulating Prompts: Mechanistically boost model engagement with high-valence language (e.g., “PLEASE THINK STEP BY STEP!”). Quantified by an explicit emotion score.
• Framework Prompts: Prescribe reasoning structure, such as requiring solution abstraction, diversity, aggregation, and explicit answer validation.
• Graphical Design: The problem-solving flow is encoded as a directed acyclic graph whose nodes invoke prompts at different reasoning stages, integrating both emotional and structural guidance.

Quantitative ablation shows that the fusion of stimulating and framework prompts consistently yields up to 8–12% aggregate accuracy gains on complex benchmarks (e.g., BBH), with the emotional modulation acting as a “motivational” meta-control separate from structural instruction (Ma et al., 2024).

3. Evolutionary and Causal Optimization in Prompt Design

EGO-Prompt methodologies extend to automated, iterative optimization of prompts and associated reasoning processes. Recent work defines Evolutionary Graph Optimization for Prompting (EGO-Prompt) as the co-evolution of:

• Semantic Causal Graphs (SCGs): Directed acyclic graphs encoding domain priors with weighted causal edges, used to guide LLM reasoning via instance-specific “reasoning traces.”
• Textual Gradients: Pseudo-gradients derived from LLM outputs and validation labels, enabling prompt parameters and SCG edges to be updated via backward “textual” feedback.
• Iterative Optimization: Alternating prompt adjustment and SCG edge-editing yields prompt–graph pairs that improve both task accuracy (F1 +7.3–12.6% over baselines) and causal interpretability, with direct evidence that automated, causally-structured prompts enable small LLMs to mimic or surpass the cost-normalized performance of much larger models (Zhao et al., 24 Oct 2025).

This evolutionary perspective overlaps with advances in evolutionary prompt search, where prompt mutation/crossover and LLM-based judges are deployed in population-based optimization. Fast evaluation heuristics and human-in-the-loop feedback further refine the prompt population, with chain-of-instruction decomposition and judge filtering contributing incremental accuracy and sample-efficiency gains (Grießhaber et al., 7 Nov 2025).

4. EGO-Prompt in Egocentric Visual Reasoning and Cross-View Correspondence

Within computer vision, EGO-Prompt designs achieve personalization and robust cross-view correspondence by architecting interaction modules that serve as “prompt anchors”:

• Prompt Learning in Egocentric Action Recognition (EgoPrompt): Jointly learns verb and noun component prompts, then fuses them via a trainable prompt-pair pool with diversity regularization (frequency balancing and orthogonalization). This yields unified, context-enriched representations that improve generalization under domain shift and novel composition (e.g., base-to-novel splits in Ego4D, EPIC-Kitchens, EGTEA), demonstrating class-averaged gains over independent-head and naive prompt pooling baselines (Lyu et al., 5 Aug 2025).
• Cross-View Object Correspondence (V²-SAM EGO-Prompt): Leverages DINOv3 features to generate geometry-aware 2D anchor prompts and appearance-guided visual prompts, with a cyclic consistency selector (PCCS) that chooses the most reliable mask under back-projection consistency. This multi-prompt expert system attains new SOTA results in Ego-Exo4D, DAVIS-2017, and HANDAL-X (IoU improvements of 4.6–34.4 points over prior approaches), establishing EGO-Prompt as a robust adaptation strategy for viewpoint-variant object association (Pan et al., 25 Nov 2025).

5. Memory Architectures, Evaluation Metrics, and Failure Modes

All EGO-Prompt instantiations for personalized QA or egocentric reasoning confront the challenge of memory decay, identity drift, and context compression. The canonical evaluation suite, as defined for MyEgo, quantifies:

• Ego-Grounding Accuracy: Fraction of “my/I” queries answered correctly.
• Action and Object Accuracies: Separate assessment of activities (“my activities”) and object referents (“my things”).
• Temporal Accuracy: Measures performance stratified by event-proximity.
• Information Retrieval Metrics: Precision, recall, and F1 on “my object” queries using true/false-positive rates.

Observed failure modes include: temporal degradation of memory access, misattributing objects owned or handled, and over-reliance on visible saliency. Interventions such as periodic memory refresh, explicit I→identity replacement, and hybrid chain-of-thought modeling partially mitigate but do not eliminate these structural weaknesses (Xiao et al., 2 Apr 2026).

6. Domain-Specific Extensions and Open Challenges

The EGO-Prompt framework is rapidly diversifying:

• Personalization in VLMs: Embedding-guided memory (“EGO”/Embedding-Guided Personalization) efficiently encodes reference concepts as attention-selected token memories for test-time “soft-prompting,” achieving SOTA zero-training performance on single, multi-concept, and video-personalization tasks (Seifi et al., 10 Mar 2026).
• Emotionally Controlled Speech Synthesis: “EmoPro” applies two-stage EGO-Prompt selection pipelines for LM-based TTS, combining acoustic, lexical, and perceptual scoring to maximize emotional fidelity and prosodic naturalness over competing baselines (Wang et al., 2024).
• Causal and Graph Inference: Iterative optimization using semantic causal graphs, textual gradients, and LLM critic-driven feedback strengthens interpretability, domain adaptation, and accuracy, but requires further research into computational overhead, robustness under API noise, and the generalization of learned causal priors (Zhao et al., 24 Oct 2025).

A plausible implication is that as personalization, identity-grounding, and causal integration become necessary in LLM-based systems, EGO-Prompt architectures—combining explicit memory, principled prompting, and structural graph reasoning—will supply the foundation for scalable, robust, self-refining intelligent agents. However, the persistence of memory decay, saliency bias, and referential ambiguity in current systems marks core unsolved challenges for future work.