General User Models (GUMs) Overview

• General User Models (GUMs) are computational frameworks that continuously update and integrate user knowledge from diverse digital traces to enable proactive applications.
• They employ a modular architecture—including observation, propose, retrieve, revise, and privacy audit modules—to infer dynamic, context-driven propositions.
• GUM systems demonstrate high calibration and effective context integration, enhancing proactive assistance in adaptive interfaces and notification management.

A General User Model (GUM) is a computational framework that abstracts, infers, and continuously updates knowledge, beliefs, preferences, and behavioral tendencies of an individual user from multimodal, unstructured observations. GUMs aim to resolve the fragmentation of traditional user models—which are typically confined to specific applications or contexts—by enabling flexible, context-aware, and semantically grounded representations that can drive reasoning and anticipation across an unlimited range of tasks and interactive scenarios (Shaikh et al., 16 May 2025). GUMs not only capture static facts or narrowly defined preferences, but also learn complex, temporally-evolving propositions reflecting the user's intentions, emotional state, situational need, and latent knowledge, while maintaining calibration (well-quantified uncertainty) and respecting contextual privacy boundaries.

1. GUM Architecture and Core Components

The GUM architecture is designed to operate on unstructured, multimodal digital traces—screenshots, notifications, file modifications, email content—without reliance on predefined schemas. GUM instantiates a pipeline of modules:

• Observation Modules: These include Screen Observers (for capturing and transcribing visual content), Notification Observers (for extracting OS alerts), and in principle, any observer for streams of digital or sensor data.
• Propose Module: For every observation, this module generates confidence-weighted natural language propositions by constructing a contextual reasoning trace (chain-of-thought) connecting the evidence to the inferred statement (e.g., "User is preparing for a wedding" inferred from viewing event invitations and dress rental searches). Each proposition is paired with a normalized confidence score in [0,1], representing subjective certainty from the generative model.
• Retrieve Module: Based on recency- and context-aware relevance scores, this module selects prior propositions that could inform the current context. Retrieval is implemented using BM25 scores modulated by time decay,

$$\gamma_i = \exp(-\alpha_i \cdot k \cdot \text{age}(d_i)), \qquad \tilde{r}_i = r_i \cdot \gamma_i$$

and Maximum Marginal Relevance (MMR)

$$\mathrm{MMR}(d_i,S) = \lambda \tilde{r}_i - (1-\lambda) \delta_i$$

balancing similarity to the query and diversity among selected propositions.

• Revise Module: Upon generation of new or overlapping inferences, this component merges, updates, and recalibrates existing propositions by adjusting confidence and temporal decay (rate at which statements become stale).
• Privacy Audit Module: All inferences and observations are subject to a privacy screening process that enforces contextual appropriateness and blocks sensitive topics using contextual integrity norms.

This hierarchical, modular pipeline enables continuous ingestion, reasoning, retrieval, and revision, maintaining an evolving, queryable store of high-level user knowledge.

2. Representation and Inference Mechanisms

Rather than encoding users as fixed-dimensional vectors or user IDs, GUMs represent a user as a dynamic set of language-based propositions with associated confidence and decay measures. Propositions capture states ("User is preparing for a wedding"), temporal intent ("User intends to submit a grant proposal soon"), affect ("User is frustrated about feedback"), and inferred needs ("User lacks formal attire for an upcoming event").

The mapping from multimodal observations to propositions is performed using large pre-trained models (e.g., Qwen 2.5 VL for vision-language, Llama 3.3 70B for natural language), followed by prompt-engineered reasoning chains. Confidence scores are generated via explicit textual prompts, then rescaled for consistency. Decay rates—affecting retrieval ranking and proposition lifespan—result from either empirical heuristics or model-predicted importance/urgency.

Updating is continual and fine-grained. Retrieval allows new inferences to consider the context of prior beliefs and intentions, while revision ensures that stale inferences are downgraded unless recurrent evidence supports their persistence. Privacy audit occurs at each stage, flagging (and blocking if necessary) inferences about sensitive topics before they can influence application-facing outputs.

3. Applications Across HCI and AI Systems

GUMs serve as an abstracted "user knowledge layer" supporting multiple downstream and cross-app scenarios:

• Context-augmented Assistants: LLM-based chat interfaces, when augmented by GUM context, deliver substantially more relevant and proactive responses (e.g., proactively mentioning ongoing edit tasks or upcoming events).
• Notification Management: OS-level GUMs can prioritize, suppress, or cluster notifications based on inferred user context (high-stakes document editing, scheduled meetings, detected focus states).
• Proactive Agents (Gumbo): Autonomous assistants leveraging GUM inferences can execute or suggest actions (e.g., "Search for suit rentals nearby" as soon as a wedding plan is inferred) before explicit user prompting.
• Resource Management, Email Organization, Cross-application Adaptation: GUMs enable intelligent orchestration of application behaviors, from task-aware summaries to personalized reminders and context-sensitive UI adaptation.

In controlled studies, GUMs demonstrated high calibration (Brier score 0.17 for high-confidence inferences) and user-rated accuracy nearing 100% for high-confidence statements (Shaikh et al., 16 May 2025). Users also indicated a preference for the full architecture over ablations, particularly appreciating the just-in-time, contextually aligned suggestions.

4. Evaluation Methodology and Empirical Calibration

GUMs are evaluated along both objective calibration and subjective utility dimensions:

• Accuracy and Calibration: In controlled deployments (e.g., with email logs), high-confidence propositions correlated with nearly perfect subjective accuracy. The full pipeline exhibited a lower Brier score (0.17) compared to ablated variants, indicating more reliable uncertainty quantification.
• Retrieval and Revision Effects: Comparative studies showed that retrieval-based context integration and cumulative revision yielded more contextually aligned, up-to-date, and less redundant inferences—reflected in higher pairwise win rates in user ratings.
• End-to-End Proactive Assistance: Field deployments with Gumbo as a proactive agent resulted in high-rated utility, timing, and acceptability of unsolicited suggestions powered by the underlying GUM.
• Privacy Safeguarding: The system's privacy audit identified only a small fraction of generated propositions as inappropriate, with participants affirming respect for contextual integrity.

5. Innovations, Limitations, and Future Directions

Significant innovations include (a) the abstraction of user state into language-based, confidence-calibrated propositions, (b) the integration of flexible, in-context retrieval and revision to maintain relevance and factuality, and (c) the operationalization of privacy via context-aware prefilters. These design choices enable semantic, longitudinal, and multimodal user modeling far beyond rigid vector embeddings or log-summary features.

Challenges and open research areas include:

• Mitigating hallucinations and prompt-injected proposition generation, particularly around adversarial or biased content.
• Extending architecture to new modalities (audio, sensors, physical context), scale (cross-device, federated deployments), and communication with other autonomous agents.
• Improving on-device efficiency for privacy, latency, and resource-constrained use cases.
• Refining revision and calibration through more end-to-end approaches (beyond modular prompt chaining).
• Enhancing transparency and user control (e.g., user-editable proposition stores, explicit feedback on calibration/confidence).
• Addressing potential risks associated with over-prediction, biased inference, or user discomfort with generated insights.

6. Implications for General User Modeling

GUMs redefine the technology landscape for user modeling by bridging the gap between rich, context-sensitive human needs and the computational machinery underpinning interactive AI systems. By leveraging multimodal, continuous observation and explicit reasoning grounded in natural language, GUMs fulfill long-standing visions of adaptive, anticipatory, and privacy-aware human-computer interaction (Shaikh et al., 16 May 2025). The ability to continuously refine, revise, and contextually surface user knowledge without rigid schema or task-bound limitations positions GUMs as a cornerstone for the next generation of context-aware operating systems, assistants, and collaborative AI.

The practical realization of GUMs sets a foundation for intelligent agents capable not only of reactive support but of proactive, semantically-driven action aligned with individual users' evolving intentions and life situations, while simultaneously embedding privacy, transparency, and calibration into the core of user-modeling practice.