Agent Experience (AX) Overview

• Agent Experience (AX) is the structured accumulation and application of an agent's operational memories, enabling robust reasoning and continual self-improvement.
• AX frameworks integrate hierarchical memory architectures, experience replay, and cross-agent knowledge sharing to optimize decision-making processes.
• By employing API-first control and automated skill synthesis, AX enhances reliability, reduces task completion times, and improves system efficiency.

Agent Experience (AX) refers to the structured accumulation, representation, retrieval, and application of knowledge, memories, and action traces generated by autonomous or semi-autonomous agents during their operation. AX frameworks enable agents not just to act, but to learn from, generalize, and reuse their own and others’ trajectories—advancing reliability, efficiency, continual self-improvement, and the emergence of agentic collective intelligence. AX encompasses system-level design, memory architectures, experience replay methods, cross-agent knowledge propagation, and human-agent-computer interaction paradigms.

1. Conceptual Foundations and Design Principles

AX is characterized by the formalization and systematization of an agent’s internal workspace—distinct from both user experience (UX) and developer experience (DX)—governing what information enters the model prompt, its structure, degree of compression, and extensibility. Key goals include:

• Conciseness and stability: AX subsystems minimize verbosity, selectively preserve key facts and decision points, and maintain long-horizon context for robust reasoning (Wang et al., 11 Dec 2025).
• Interpretability and structure: Agent-facing context is presented as structured (tagged, hierarchical, or templated) memory objects: e.g., <file_edit …>, indexed action–reasoning traces, experience fragments or skill libraries (Wang et al., 11 Dec 2025, Tang et al., 8 Jul 2025, Cai et al., 9 Nov 2025).
• Extensibility and retrieval: AX frameworks are designed for plug-and-play integration with external tools (e.g., semantic indices, RAG modules, reasoning engines), facilitating collective agent intelligence and seamless cross-domain knowledge sharing (Tang et al., 8 Jul 2025).
• Action-centric control: Experience representation supports prioritized tool choice (APIs over UI wherever possible) and guarantees optimal selection for latency and reliability (Lu et al., 2024).

2. AX Representation and Memory Architectures

AX entries (sometimes called "fragments," "skills," or "experience units") are formalized as multi-field tuples:

• Confucius SDK: AX organized as hierarchical working memory, with persistent, typed nodes (decisions, error snippets, file edits), adaptive compression/summarization, and cross-session retrieval. Decay policies demote stale items, critical observations are pinned for long-context reasoning (Wang et al., 11 Dec 2025).
• Agent KB ("AX unit"): Each experience is a quadruple $E = \langle \pi, \gamma, S, \mathcal{C} \rangle$—with $\pi$ a problem embedding, $\gamma$ a set of constraints, $S$ an action–reasoning trace, and $\mathcal{C}$ metadata for cross-framework compatibility. Stored experiences are indexed lexically and semantically for hybrid retrieval (Tang et al., 8 Jul 2025).
• FLEX: Non-parametric, human-readable library $\mathcal{E}$, hierarchically partitioned into strategic principles, reasoning templates, and concrete instances, with zones for "golden" successes and "warning" failures. Growth follows predictable scaling laws with logistic dynamics across epochs (Cai et al., 9 Nov 2025).
• GoalfyMax XP: Layered memory system distinguishing short-term buffers for recent context and long-term stores of vetted fragments, each scored for trust, annotated with embeddings, and indexed for fast retrieval (Wu et al., 13 Jul 2025).
• ReMe: Fine-grained experience pool of $E=\langle\omega,e,\kappa,c,\tau\rangle$ units, supporting context-adaptive reuse and utility-based pruning to avoid memory stagnation or overfitting (Cao et al., 11 Dec 2025).
• MUSE: Three-level memory hierarchy (strategic, procedural/SOP, tool memory) integrates reflection, retrieval, and dynamic updating to enable "on-the-job" agent self-evolution (Yang et al., 9 Oct 2025).

Framework/Paper	AX Data Unit Structure	Memory Shape/Compression
Confucius SDK (Wang et al., 11 Dec 2025)	Typed hierarchy: decision, error, edit	Tree; adaptive summaries
Agent KB (Tang et al., 8 Jul 2025)	$\langle$task emb., constraints, trace, meta$\rangle$	JSON, semantic index
FLEX (Cai et al., 9 Nov 2025)	(level, zone, text)	Hierarchy, golden/warning
GoalfyMax XP (Wu et al., 13 Jul 2025)	WHY/HOW/CHECK fragments	Short + long-term memory
ReMe (Cao et al., 11 Dec 2025)	$\langle\omega,e,\kappa,c,\tau\rangle$	Embedding-pool, utility
MUSE (Yang et al., 9 Oct 2025)	Strategic, procedural, tool memories	3-level hierarchy

These representations support both granular recall (e.g., top-k experience retrieval by relevance or trust) and abstract generalization (e.g., rediscovery of long-horizon strategies, cross-task SOPs).

3. Integration of Experience Replay and Knowledge Evolution

AX systems operationalize agent memory by enabling prioritized, structured experience replay:

• Regret-minimizing replay: Schemes like MAC-PO assign replay sampling weights by minimizing expected policy regret, integrating Bellman error, proximity to optimal $Q^*$, on-policy likelihood, and joint-action diversity (Mei et al., 2023).
• Cache-locality prioritization: AccMER further exploits hardware-level cache performance by reusing transition batches with high weights for $n$ steps, yielding significant speedups without loss of convergence (Gogineni et al., 2023).
• Gradient-free learning and inheritance: FLEX and related frameworks maintain forward-evolving experience pools with actor–critic loops. New experiences are captured, hierarchically merged, and transferred bidirectionally among agents, supporting population-level AX inheritance (Cai et al., 9 Nov 2025, Cao et al., 11 Dec 2025).
• Automated distillation and refinement: ReMe's mechanisms include multi-faceted extraction (success, failure, comparative insights), scenario-aware kNN retrieval, and utility-based deletion, enabling continual evolution towards compact, high-quality experiential knowledge (Cao et al., 11 Dec 2025).

4. Action Selection, Skill Synthesis, and Cross-Agent Knowledge Transfer

AX advances agent reliability and efficiency through formalized skill selection and propagation:

• API-first action prioritization: AXIS ensures that, for every subtask, API-based actions are favored over UI interactions when possible, with formal selection via cost-minimizing algorithms (Lu et al., 2024).
• Automated skill discovery: AXIS combines doc-guided and heuristic exploration, generating and validating new skills, translating UI traces into API calls, and continuously expanding the skill library (Lu et al., 2024).
• Collective knowledge sharing: Agent KB unifies cross-framework AX by storing experiences as indexed knowledge graphs, supporting plug-and-play integration and hybrid retrieval pipelines for planning and feedback. Disagreement gates prevent negative interference during cross-model transfer (Tang et al., 8 Jul 2025).
• Multi-agent coordination and memory reuse: GoalfyMax XP aggregates structured rationale (“why”) and procedural (“how”) fragments, scored and validated, enabling continual learning and robust protocol-driven collaboration among agents (Wu et al., 13 Jul 2025).
• 360° assessment for multi-agent systems: Frameworks like 360°REA combine self, peer, and supervisor feedback to generate dual-level experience pools, improving draft quality and generalizability through fine-grained evaluation (Gao et al., 2024).

5. Empirical Evaluation and Performance Impact

AX architectures consistently yield measurable improvements in agent performance across domains:

• Task completion and efficiency: AXIS reduces completion time by 65–70%, lowers cognitive workload by 38–53%, and reaches 97–98% task accuracy compared to human users (Lu et al., 2024).
• Long-horizon reasoning: Hierarchical working memory in Confucius SDK improves Resolve@1 by 6.6–12.4 points over ablative baselines, reduces prompt lengths, and enables more robust multi-file edits (Wang et al., 11 Dec 2025).
• Continual and transfer learning: MUSE demonstrates continuous learning on productivity benchmarks, with >10 point improvement in partial score and striking zero-shot gains when transferring AX to previously unseen tasks (Yang et al., 9 Oct 2025).
• Scalable RL via synthetic experience: DreamGym synthesizes reasoning-grounded experiences, matching or exceeding RL baselines while reducing GPU time 3–5× and lowering sample complexity in sim-to-real transfer (Chen et al., 5 Nov 2025).
• Multi-agent pathfinding: exRHCR achieves up to 39% faster planning by leveraging experience seeds to warm-start priority searches (Madar et al., 2022).
• Replay optimization: AccMER delivers 17–25% reduction in training time via cache-locality-aware prioritization, with preserved or even improved learning curves (Gogineni et al., 2023).

6. UI, Operating System, and Tooling Implications

AX systems are redefining human-agent-computer interaction and software architecture:

• Agent-Centric Operating System (Agent OS): AXIS proposes a paradigm shift to API-first applications, flattening UI hierarchies and repositioning apps as agent enclaves managed by a central kernel hosting orchestration and skill management layers (Lu et al., 2024).
• Scaffolds for agent experience prototyping: Tools such as AgentBuilder democratize AX prototyping, supporting no-code workflows, debugging, live execution, scenario testing, and toggles between developer and user views (Liang et al., 6 Oct 2025).
• Design guidelines: AX-driven design principles advise exposing all key app functions as composable API skills, minimizing nested UI, and providing uniform registries for easy parsing and orchestration (Lu et al., 2024).
• Challenges: Scaling AX tooling from linear flows to branching tasks, supporting collaborative workflows among designers and QA, and formalizing guarantees of agent-safe operation (e.g., via executable contracts) remain open areas for future work (Liang et al., 6 Oct 2025).

7. Future Directions and Open Research Questions

Research on AX is converging on several fundamental themes:

• Experience abstraction and generalization: New mechanisms for aggregating, compressing, and recombining fine-grained experience units—balancing specificity and transferability—are advancing zero-shot and continual learning (Cai et al., 9 Nov 2025, Cao et al., 11 Dec 2025, Yang et al., 9 Oct 2025).
• Collective agent intelligence: Universal KBs, plug-and-play experience pools, and cross-platform retrieval pipelines lay the technical groundwork for emergent agent societies capable of sharing, disputing, refining, and inheriting knowledge (Tang et al., 8 Jul 2025).
• Computational efficiency over scale: The memory-scaling effects observed in ReMe and FLEX suggest that sufficiently robust experience-driven evolution can allow lightweight agents to outperform much larger, memoryless baselines—potentially shifting the emphasis from ever-larger LLMs to smarter, memory-rich orchestration (Cao et al., 11 Dec 2025, Cai et al., 9 Nov 2025).
• Safety, validation, and dynamic adaptation: Utility-pruning, trust scoring, and 360° assessment techniques (e.g., peer and supervisor feedback, contextual validation) are central to maintaining high-quality experience pools, supporting reliability and adaptation in unpredictable environments (Gao et al., 2024, Wu et al., 13 Jul 2025).
• Open questions: Theoretical limits of non-parametric AX, optimal compression and decay policies, experience transfer across heterogeneous agent types, and strong guarantees for “never-delete” or “scoped-action” behavior require further investigation.

Agent Experience thus encapsulates a multi-faceted, rapidly evolving paradigm for constructing autonomous systems that not only act but continuously learn, refine, and transmit knowledge at runtime—integrating insights across memory architectures, action selection, performance optimization, and collective intelligence.