Papers
Topics
Authors
Recent
Search
2000 character limit reached

Multi-Agent Transactive Memory

Published 18 Jun 2026 in cs.AI, cs.CL, and cs.IR | (2606.19911v1)

Abstract: The decentralized deployment of LLM agents with diverse capabilities across diverse tasks motivates infrastructure for knowledge sharing across heterogeneous agent populations. Just as search engines index human-generated artifacts to support human problem solving, retrieval systems can organize agent-generated artifacts for reuse across agent populations. We extend retrieval-augmented generation - which demonstrates the value of human-authored artifacts to individual agents - to retrieval of agent-generated artifacts supporting a population of agents. In particular, agent trajectories encode reusable procedural knowledge, yet these artifacts are typically discarded after a single use or retained only by the producing agent, forcing newly instantiated agents to repeatedly rediscover existing solutions. We propose Multi-Agent Transactive Memory (MATM), a framework for population-level storage and retrieval of agent-generated trajectories, where producer agents contribute trajectories to a shared repository and consumer agents retrieve them to improve task execution. We focus on interactive environments (ALFWorld and WebArena), where trajectories are long and encode especially rich procedural structure. Our experiments demonstrate that retrieving trajectories from MATM improves downstream task performance and reduces interaction steps without coordination or joint training. These results position MATM as a design pattern for population-level experience sharing in open agent ecosystems.

Summary

  • The paper introduces MATM to enable agents to share and retrieve procedural trajectories through state-conditioned key-value indexing.
  • The paper demonstrates significant improvements in task success rates (up to 64.3% in ALFWorld) and efficiency by integrating dense retrieval with LTR reranking.
  • The paper establishes transactive memory as a key design for scalable multi-agent systems, supporting immediate procedural transfer and collective problem solving.

Multi-Agent Transactive Memory: Population-Level Artifact Sharing for Open Agent Ecosystems

Motivation and Context

The decentralization of LLM agents and their diverse deployment across multiple domains amplifies the need for infrastructures that facilitate population-wide knowledge sharing. While retrieval-augmented generation (RAG) validates the utility of retrieving human-authored artifacts for individual agents, a considerable inefficiency persists: agent-generated artifacts, especially procedural trajectories in interactive environments, are typically discarded or retained in agent-local silos. This results in redundant rediscovery and limits the evolution of collective behaviors.

The proposed Multi-Agent Transactive Memory (MATM) framework targets this inefficiency by enabling agents to both contribute and retrieve interaction trajectories from a shared repository, bridging the gap between individual agent memory and collective procedural intelligence. MATM thereby operationalizes the “transactive memory” concept in multi-agent learning, supporting open-ended, heterogeneous, and continually expanding agent populations operating in interactive domains such as ALFWorld and WebArena.

Figure 1

Figure 1: MATM expands traditional and RAG retrieval paradigms to encompass agent-generated artifacts, enabling organic growth and continual service for distributed agent populations.

MATM Architecture and Indexing

Within MATM, a population of LLM agents operates on one or more environments, producing rich trajectories of action-observation pairs during sequential task execution. Unlike prior per-agent memory or case-based reasoning, MATM treats these trajectories as first-class artifacts available for population-level reuse. Agents act as both producers and consumers: successful trajectories are contributed to MATM, while consumer agents query the memory for trajectories with high contextual relevance to their current state.

Key design elements include:

  • State-Conditioned Key-Value Indexing: Recent interaction history forms the key; subsequent action-observation continuations form the value. This allows agents to condition retrieval not merely on initial task specification but on their live state sequence.
  • Producer-Consumer Attribution: Retrieval events retain provenance, permitting trust and reputation modeling over contributed artifacts.
  • Temporal Memory Growth: The population-level memory grows both by bootstrapping from publicly available trajectories and through incremental online updates as agents solve new tasks.

Retrieval and Learning to Rank

Agent consumption of MATM is realized via a cascaded retrieval-and-ranking pipeline. An initial dense retriever surfaces top candidate trajectory chunks. To optimize retrieval quality, a learning-to-rank (LTR) stage uses features spanning producer metadata, consumer IDs, retrieval scores, trajectory/query lengths, and various embedding or overlap-based similarities.

The reranker is trained via marginal utility labels: the improvement in agent performance when a candidate trajectory is injected versus baseline behavior without retrieval. Multiple architectures are benchmarked for reranking, including feed-forward networks, LambdaMART, and SVMRank. The framework thus enables both population-level sharing and principled ranking of diverse procedural artifacts.

Experimental Evaluation

Benchmarks and Setup

MATM is instantiated in ALFWorld (text-based household tasks) and WebArena (web navigation tasks), with producer and consumer populations comprising over 30 heterogeneous LLM agents. The MATM indices are pre-populated with trajectories from strong open checkpoints and further expanded via systematic allocation and contribution from all agents during training phases. LTR reranker supervision is collected by rolling out retrieval-augmented episodes from trajectory branching points, enabling precise measurement of retrieval utility.

Effectiveness and Efficiency Gains

Retrieval from MATM consistently yields substantial improvements in both downstream task effectiveness (success rate, SR) and efficiency (steps per episode). For ALFWorld, SR increases from 47% (no retrieval) to 55% (dense retrieval) and up to 64.3% under SVMRank reranking; average episode steps decrease from 11.77 to 10.35. On WebArena, gains are more modest (SR increase from 18% to 20%), reflecting the greater complexity and sensitivity to early errors in web tasks, but a positive trend holds.

Figure 2

Figure 2

Figure 2: MATM memory scaling curves on ALFWorld (top) and WebArena (bottom), showing monotonic improvement in success rates and step reduction as memory size increases.

Reranking Improves Retrieval Utility

Reranking with LTR models significantly augments gains over single-stage retrieval. On ALFWorld, SVMRank reranking delivers an additional +9.2 percentage points (SR) over dense retrieval. Feature importance analyses show producer agent metadata (e.g., benchmark competence scores) has high predictive value, enabling implicit trust modeling and personalized retrieval.

Distribution and Generalization Effects

The population-level sharing does not result in benefit concentration among specific agent pairs. Retrieval is advantageous regardless of the capability gap between producer and consumer, with only a weak (statistically insignificant) positive correlation between capability gap and retrieval advantage.

Figure 3

Figure 3: Each point denotes retrieval advantage versus producer-consumer capability gap in ALFWorld, illustrating broad distribution of benefits across agent pairings.

Furthermore, retrieved trajectories generalize beyond their original task type. Restricting MATM retrieval to same-task candidates reduces effectiveness relative to unconstrained retrieval, and even cross-task retrieval confers substantial gains over the no-retrieval baseline.

Scalability

As MATM memory size increases, consumer effectiveness and efficiency scale monotonically in ALFWorld and, with minor non-monotonicities, in WebArena. Critical mass in memory size is necessary to ensure the diversity and coverage required for consistently relevant retrieval.

Theoretical and Practical Implications

MATM establishes transactive memory as a viable architectural primitive for distributed agent ecosystems. Its empirical results challenge the sufficiency of agent-local memory paradigms and emphasize artifact sharing as foundational for scalable, continually improving populations. It breaks with knowledge distillation and transfer learning paradigms by enabling immediate procedural transfer without the alignment or retraining overheads.

Producer-consumer attribution, population-level trust, and personalization of retrieval emerge as core future directions. Scaling MATM to more heterogeneous, potentially adversarial open agent societies will also require robustness to malicious inputs and fair attribution mechanisms. The generalization results motivate further investigation into artifact types with maximal cross-task transferability and into incentivization mechanisms for artifact contribution in open agent markets.

Conclusion

MATM implements a scalable, attribution-preserving, population-level memory where agents both contribute and retrieve procedural trajectories, enabling collective problem-solving capability that grows with agent activity. Demonstrated on complex interactive environments, MATM increases both task success and efficiency without joint training or centralized coordination, generalizes across tasks, and benefits from index scaling. The results position shared artifact storage and retrieval as a critical design pattern for next-generation multi-agent systems and a foundational substrate for open-ended, continual collective intelligence.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Explain it Like I'm 14

Easy Explanation of “Multi-Agent Transactive Memory (MATM)”

What this paper is about

This paper introduces a shared “memory” for AI agents called Multi-Agent Transactive Memory (MATM). Think of many different AI helpers working on lots of tasks. As they work, they create useful “how I solved it” records. Instead of throwing those records away or keeping them private, MATM stores them in a shared library so other agents can look them up later. This helps new agents avoid repeating the same trial-and-error and solve problems faster and better.

The main questions the paper asks

  • Can agents do better on tasks if they can look up helpful “how-to” records made by other agents?
  • Can we make the lookup smarter so agents find the most helpful records, not just the most similar-sounding ones?
  • Do both weaker and stronger agents benefit from sharing, or just some?
  • Does this sharing help across different kinds of tasks, not only the exact same type?
  • Does the system work better as the shared library grows larger?

How the system works (in everyday language)

What’s a “trajectory”?

A trajectory is a step-by-step record of what an agent did and what happened next. Imagine a student’s worked solution that shows:

  • what they tried (the action),
  • what they saw after trying (the observation),
  • and the next step they tried.

These chains of steps are valuable because they show the actual process, not just the final answer.

What is MATM?

MATM is a shared library where agents:

  • contribute their step-by-step solution records (producer agents), and
  • look up records that could guide them on a new problem (consumer agents).

This is inspired by “transactive memory” in human groups, where people don’t remember everything themselves—they remember who knows what and how to find it.

How do agents look things up?

  • The agent takes a short window of its recent steps (like the last few moves it made) and turns that into a search query.
  • The system finds the top candidate solution snippets from the shared library that look relevant to the agent’s current situation.
  • A small “decision helper” inside the agent chooses when to search so it doesn’t overload itself with random tips.

In simple terms: the agent says, “Given what I’ve just done, has anyone solved something like this before? If so, what did they do next?”

Picking the best advice: “reranking”

Finding similar records is helpful, but similar isn’t always best. The paper adds a second step called “reranking”:

  • It scores the candidates not only by similarity but also by:
    • the past reliability of the producer agent,
    • how well a candidate fits the current agent,
    • how long and complete the candidate is,
    • and whether similar candidates actually improved results in past tests.
  • This is like asking, “Whose advice has worked well before for problems like this?” and then choosing the top one.

The authors try several reranking methods and train them using a simple idea: if adding a specific retrieved snippet led to a better outcome than not retrieving anything, that snippet gets a higher score. This is called measuring “marginal utility,” which means “how much extra help did this actually give?”

Where they tested it

They tested MATM in two interactive environments where agents must take many steps:

  • ALFWorld: a text-based household environment (like “pick up the soap from the bathroom sink”).
  • WebArena: a web-browsing environment where agents click and type to complete website tasks.

What they found (and why it matters)

Here are the key takeaways, presented simply:

  • Better results and fewer steps:
    • With MATM, agents solved more tasks and took fewer steps on average. For example, in ALFWorld, success rates rose from about 47% without MATM to about 64% with the best reranking. Fewer steps mean the agent is more efficient, like finishing a maze faster.
  • Smarter picking helps:
    • Using the reranking step made results even better than using similarity alone. It’s not just about finding “similar” past attempts; it’s about picking “helpful” ones.
  • Benefits across the board:
    • The improvements weren’t limited to weak agents. Both weaker and stronger agents benefited from the shared memory, and not just by copying the strongest agent. Advice from many different producers could help.
  • Helps across different task types:
    • Even when the retrieved records came from a different type of task, they still helped. That means the shared library carries reusable patterns, not just one-off tips.
  • Grows stronger as it grows larger:
    • As the library contained more records, performance generally improved. With more experiences to draw from, agents are more likely to find a good fit for their current situation.

Why this is important

  • Saves time and compute: Agents don’t waste effort rediscovering solutions that others have already figured out.
  • Encourages collaboration: It turns many individual agents into a community that learns together.
  • Works in open ecosystems: New agents can join at any time, contribute their experiences, and immediately benefit from what’s already there—without needing special training to fit in.
  • Lays groundwork for trustworthy sharing: Because reranking can consider producer “reputation,” future systems could build stronger models of trust, safety, and personalization.

Simple conclusion

The paper shows a practical way for AI agents to share their experiences in a common memory and use that shared knowledge to solve tasks better and faster. By storing step-by-step solutions and teaching agents how to find the most helpful ones, MATM turns a collection of separate agents into a community that steadily gets smarter together.

View Paper Prompt  View All Prompts 

Knowledge Gaps

Knowledge gaps, limitations, and open questions

Below is a concise, actionable list of what remains missing, uncertain, or unexplored in the paper, framed to guide follow-up research.

  • External validity: MATM is evaluated only on ALFWorld and WebArena; its effectiveness on other interactive settings (e.g., embodied robotics, code execution, multimodal tool use, long-horizon planning) is untested.
  • Cross-benchmark generalization: Rerankers are trained and evaluated within the same benchmark; whether a reranker trained on one environment transfers to another remains unknown.
  • Cold-start consumers: Personalization features use consumer metadata, but how MATM helps brand-new or unseen consumer models with no prior metadata is not evaluated.
  • Cold-start producers: The reranker leverages producer metadata (e.g., benchmark scores); how to rank or trust trajectories from new producers without reputational signals is open.
  • Producer-side welfare: The work measures consumer gains but not producer incentives, credit/attribution, or fair reward allocation in the proposed two-sided marketplace.
  • Adversarial resilience: There is no defense or evaluation against malicious or poisoned trajectories, prompt-injection artifacts, or spam contributions to the repository.
  • Provenance and trust: Mechanisms for verifying trajectory provenance, linking outcomes to specific producers, and auditing for manipulation are unspecified.
  • Privacy and governance: The repository may contain sensitive or proprietary content (especially in web tasks); data privacy, access controls, redaction, and licensing/compliance policies are not addressed.
  • Negative transfer detection: The system does not detect when retrieval harms performance; policies for suppressing misleading or out-of-date trajectories are unstudied.
  • Retrieval planner design: The RetrievalPlanner’s decision policy (when to retrieve) is not analyzed or optimized; no ablation on retrieval frequency, triggers, cost-benefit thresholds, or failure modes.
  • Retrieval budget and fusion: Only a single top-1 chunk is injected; the benefits/risks of multi-chunk retrieval, late fusion, or iterative re-retrieval are unexplored.
  • Chunking choices: The effect of trajectory window size (l=5), key/value definitions, and variable-length segmentation on retrieval quality is not analyzed.
  • Embedding model choice: Retrieval uses E5-Base without comparison to other dense retrievers, cross-encoders, or trajectory-specialized encoders; no end-to-end retrieval tuning is attempted.
  • Reranker modeling space: Only FFN, LambdaMART, and SVMRank are studied; the value of transformer cross-encoders, listwise objectives, or differentiable retrieval remains untested.
  • Labeling noise and variance: Marginal-utility labels depend on roll-in/out randomness and sparse rank sampling; robustness of labels, inter-run variance, and confidence calibration are not reported.
  • Feature dependence and drift: Reranker feature importance is environment-specific; how to adapt features online to distribution shift, new agents, or evolving environments is open.
  • Online learning: Procedures for continual reranker updates, safe deployment, and guardrails against catastrophic shifts as the index grows are unspecified.
  • Deduplication and quality control: Controls for near-duplicate trajectories, repeated patterns, or noisy low-quality contributions are not discussed.
  • Index growth vs. latency/cost: The paper shows accuracy scaling with memory size but does not quantify retrieval latency, memory footprint, or compute costs, nor propose indexing/sharding strategies.
  • Robustness to distribution shift: Test tasks may share layouts/domains with training; how MATM behaves under truly novel tasks, domains, or UI changes is unexamined.
  • Heterogeneous action spaces: The framework assumes compatible action-observation formats; how to normalize trajectories when agents/environments use different tool APIs or ontologies is unclear.
  • Integration policy: How retrieved chunks are incorporated into the agent’s policy (prompt structure, conflict resolution with current plan, context-window budgeting) lacks systematic study.
  • Safety and misuse: Risks from retrieving trajectories that encode unsafe behaviors (e.g., credential handling, harmful web actions) and corresponding safeguards are not evaluated.
  • Cross-task transfer granularity: While cross-task gains are shown, the specific structural factors enabling transfer (e.g., subgoal schemas, UI patterns) are not identified or exploited.
  • Task-type taxonomy reliance: Retrieval scope ablations use coarse task-type labels; finer-grained, learned task taxonomies or skill ontologies for better generalization are not explored.
  • Measuring compute savings: Although reuse is motivated by cost-efficiency, quantitative analysis of token/step/compute savings from MATM (including retrieval overhead) is missing.
  • Negative example curation: The index does not explicitly store or use “anti-trajectories” (known-bad paths) to steer consumers away from pitfalls.
  • Fairness across consumers: Whether MATM systematically helps or harms certain consumer segments (model sizes, families, or capability bands) beyond average gains is incompletely analyzed.
  • Update policies and forgetting: Strategies for pruning stale/misaligned trajectories, handling environment updates, and preventing repository bloat are absent.
  • Standardization: A common schema for trajectories (metadata, actions, observations, outcomes) that supports interop across heterogeneous agents/tools is not specified.
  • Hybrid artifacts: The paper focuses on raw trajectories; how to jointly index and retrieve higher-level skills, code, and plans alongside trajectories (and when each is preferable) is open.
  • Evaluation breadth: Beyond SR, steps, and RPP, there is no human preference evaluation, error taxonomy, or causal analysis linking retrieval decisions to specific outcome changes.
  • Theoretical underpinnings: Formal guarantees about when transactive memory helps (e.g., conditions on similarity, coverage, noise) are not provided.
  • Marketplace mechanisms: Protocols for contribution curation, reputation, pricing/rewards, and governance in the proposed two-sided market are unimplemented and unevaluated.
View Paper Prompt  View All Prompts 

Practical Applications

Immediate Applications

Below is a focused list of deployable use cases that leverage the paper’s core innovations: population-level storage/retrieval of agent-generated trajectories, state-conditioned retrieval, and learning-to-rank (LTR) rerankers with producer trust and consumer personalization.

  • Enterprise web RPA acceleration (software, operations)
    • What: Share and retrieve successful web-navigation trajectories (logins, form-fills, KYC, invoicing, procurement) across a fleet of enterprise agents to improve success rates and reduce steps.
    • Tools/products/workflows: MATM Memory Server (vector DB + trajectory schema), Producer/Consumer SDKs for logging and retrieval, RetrievalPlanner policy, cascaded retriever→LTR reranker, observability with RPP metric.
    • Assumptions/dependencies: Consistent action semantics (e.g., browser APIs), privacy filtering/PII redaction in trajectories, robust HTML/state embeddings, governance for trajectory attribution and rollback when sites change.
  • Customer support automation with shared resolution flows (software, CX)
    • What: Chat/agent systems retrieve proven escalation and troubleshooting trajectories to resolve tickets faster with fewer tool calls.
    • Tools/products/workflows: Ticket-to-trajectory linker, domain-specific feature engineering (error codes, device types), trust modeling for producer teams with high historical resolution rates.
    • Assumptions/dependencies: Integration with CRM/ITSM systems, de-identification of user data, versioning of runbooks as websites/products evolve.
  • SOC runbook retrieval for incident response (security)
    • What: SOC copilots retrieve agent-executed runbooks (isolation, triage, containment) conditioned on current telemetry to reduce time-to-mitigation.
    • Tools/products/workflows: Telemetry-to-state keying, producer trust via historical MTTR, gated insertion with quality thresholds.
    • Assumptions/dependencies: Strict access control and audit logs, redaction of sensitive indicators, adversarial submission detection.
  • CI/CD and SRE auto-remediation reuse (software infrastructure)
    • What: Agents reuse prior rollback/patch/infra change trajectories that successfully fixed classes of failures, reducing experimentation steps.
    • Tools/products/workflows: Pre-population from incident postmortems, utility-labeled reranker training from A/B runs, RPP dashboards by service.
    • Assumptions/dependencies: Safe execution sandboxes, strong provenance and revert mechanisms, environment drift detection.
  • EHR/health IT admin workflow assistance (healthcare)
    • What: Administrative agents retrieve EHR navigation trajectories (prior-authorization, coding, scheduling) to reduce clicks and errors.
    • Tools/products/workflows: HIPAA-compliant trajectory storage, consumer personalization by role (coder vs. scheduler), environment-specific action adapters.
    • Assumptions/dependencies: Compliance and consent, rigorous PII/PHI scrubbing, vendor API stability.
  • Enterprise knowledge operations copilots (knowledge management)
    • What: Cross-team agents reuse trajectories for document lifecycle tasks (classification, retention, entitlements) to standardize processes.
    • Tools/products/workflows: Producer reputation scores, policy-aware reranking (department, sensitivity), retrieval scope controls (same- vs cross-task).
    • Assumptions/dependencies: Data governance policies encoded in retrieval/ranking, lineage tracking and auditability.
  • Education platforms with solution-strategy retrieval (education)
    • What: Tutoring/assessment agents retrieve procedural solution trajectories to guide students through comparable problems while adapting steps to the current state.
    • Tools/products/workflows: Problem-state keying, cross-task retrieval for transferable heuristics, guardrails to avoid verbatim answer leakage.
    • Assumptions/dependencies: Academic integrity controls, domain adaptation (math, CS, physics) of action semantics.
  • In-product automation for SaaS (B2B software)
    • What: Product-embedded agents retrieve previously successful user-journey trajectories (workspace setup, data import, permissioning) to onboard users faster.
    • Tools/products/workflows: On-device trajectory cache + organization-level MATM, retrieval personalization by tenant and plan tier.
    • Assumptions/dependencies: Multi-tenant isolation, opt-in consent for sharing anonymized trajectories.
  • Browser automation assistants for consumers (daily life)
    • What: Personal agents reuse community trajectories for common tasks (travel bookings, returns, benefit applications), improving reliability.
    • Tools/products/workflows: Browser extension with local+community MATM, drift alerts when site flows change, one-click report/rollback.
    • Assumptions/dependencies: Permissioned sharing, terms-of-service compliance, sensitivity to dynamic CAPTCHAs and geo-variations.
  • Research and evaluation infrastructure (academia, IR/agents)
    • What: Benchmarks, datasets, and tooling to study trajectory retrieval, marginal-utility labeling, and RPP-based welfare metrics at population scale.
    • Tools/products/workflows: Open MATM trajectory datasets, LTR pipelines (FFN/LambdaMART/SVMRank), feature catalogs (producer metadata, query-trajectory features).
    • Assumptions/dependencies: Clear artifact schemas, reproducible evaluation splits, budget for utility-labeled data collection.

Long-Term Applications

These require additional research, scaling, standardization, or regulatory alignment, but are directly implied by the paper’s methods and findings (population-level experience sharing, trust-aware retrieval, cross-task generalization, scaling curves).

  • Open agent ecosystem marketplaces with attribution and incentives (software platforms)
    • What: Cross-vendor markets where producers publish trajectories with provenance; consumers pay per-use, with trust and personalization in ranking.
    • Tools/products/workflows: Standardized trajectory schemas/APIs, attribution and remuneration protocols, fairness-aware ranking, reputation systems.
    • Assumptions/dependencies: Interoperability standards, economic and legal frameworks for IP and liability, adversarial resilience.
  • Cross-institution healthcare trajectory networks (healthcare, policy)
    • What: Privacy-preserving sharing of admin and clinical workflow trajectories across institutions to reduce administrative burden and error rates.
    • Tools/products/workflows: Federated MATM with differential privacy, institution-scoped trust models, regulator-auditable lineage.
    • Assumptions/dependencies: Regulatory approval (HIPAA/GDPR), robust de-identification, vendor-neutral EHR action semantics.
  • Multi-robot shared procedural memory (robotics, manufacturing, logistics)
    • What: Robots share and retrieve manipulation/navigation trajectories; consumers adapt retrieved segments to current state for faster task completion.
    • Tools/products/workflows: State-conditioned keying over sensor embeddings, sim-to-real alignment, safety-rated execution policies.
    • Assumptions/dependencies: Common action abstractions across platforms, high-fidelity embeddings for physical state, certified safety constraints.
  • Grid operations and incident response playbooks (energy, critical infrastructure)
    • What: Agents share real incident trajectories (dispatch, load-shed, restoration) with trust-weighted retrieval to improve resilience.
    • Tools/products/workflows: Real-time state keying from SCADA/EMS, regulator-approved audit trails, counterfactual rollouts in simulators.
    • Assumptions/dependencies: Secure enclaves for sensitive data, rigorous red-teaming against adversarial submissions, sector standards.
  • Financial compliance and onboarding copilot networks (finance, regtech)
    • What: Cross-firm sharing of compliant trajectories for onboarding, AML/KYC checks, reporting; retrieval adapts to jurisdiction/state.
    • Tools/products/workflows: Jurisdiction-aware reranking, policy constraints embedded in ranking features, regulator portals for inspection.
    • Assumptions/dependencies: Data-sharing agreements, explainability mandates, continuous monitoring for rule changes.
  • Safety- and robustness-optimized trajectory retrieval (AI safety)
    • What: Ranking policies that penalize risky producers, detect distribution/drift, and down-weight trajectories with harmful side effects.
    • Tools/products/workflows: Counterfactual risk scoring, causal features in LTR, sandbox evaluation before live execution, kill-switch governance.
    • Assumptions/dependencies: High-quality labels for adverse events, standardized risk taxonomies, cost-effective pre-deployment testing.
  • Federated and on-device MATM for privacy-first applications (edge AI)
    • What: Devices/organizations keep local MATM while sharing only secure summaries or model updates for global improvements.
    • Tools/products/workflows: Federated LTR training, privacy-preserving feature engineering, periodic synchronization.
    • Assumptions/dependencies: Communication-efficient protocols, defensible privacy guarantees, robust aggregation against poisoning.
  • Cross-domain skill induction atop trajectories (skill libraries)
    • What: Induce reusable skills/workflows from trajectories, publish as skills with provenance; agents retrieve either raw trajectories or distilled skills.
    • Tools/products/workflows: Skill extraction pipelines, dual-index for trajectories and skills, automatic skill aging/retirement.
    • Assumptions/dependencies: Reliable induction quality, alignment between skill interfaces and consumer agents, continual revalidation.
  • Standards and policy for artifact governance (policy, standards bodies)
    • What: Schemas for trajectories, provenance, consent, attribution, and audit; certification for MATM services and rerankers.
    • Tools/products/workflows: Open specs for action-observation schemas, evidence logs for marginal utility, conformance suites.
    • Assumptions/dependencies: Multi-stakeholder governance, harmonization across jurisdictions, compatibility with data and model cards.
  • Scientific automation and lab orchestration (science/biotech)
    • What: Agents share lab execution trajectories (pipetting, assay protocols, data QC), speeding up reproducibility and method transfer.
    • Tools/products/workflows: Instrument-specific adapters, error-aware retrieval (e.g., contamination risks), provenance to publications/LIMS.
    • Assumptions/dependencies: Vendor cooperation on APIs, safety and biosecurity review, ontologies for experimental states.
  • Large-scale programmatic accessibility assistants (public sector, daily life)
    • What: Community MATM of successful navigation through government benefits, accessibility accommodations, and forms.
    • Tools/products/workflows: Public-good trajectory repositories with curation, locale-aware reranking, offline-first modes.
    • Assumptions/dependencies: Government APIs and consent, safeguards against misuse, continuous updates for policy changes.

Notes on feasibility and dependencies across applications:

  • Data quality and privacy: Trajectories must be de-identified, filtered, and versioned; sector-specific compliance applies.
  • Interoperability: Consistent action semantics and schemas across tools/environments are critical for cross-agent reuse.
  • Utility-labeled data: Effective reranking benefits from marginal-utility labels; bootstrapping via A/B or selective labeling can reduce cost.
  • Robustness to drift: Websites, policies, and systems change; monitoring, rollback, and revalidation workflows are needed.
  • Safety and security: Adversarial submissions and harmful trajectories require trust modeling, anomaly detection, and sandboxing.
  • Compute and context budgets: RetrievalPlanner and reranker should manage inference-time costs; chunk sizes and top-k must be tuned per domain.
  • Incentives and attribution: Marketplaces and cross-org sharing require attribution, remuneration, and fair ranking policies.
View Paper Prompt  View All Prompts 

Glossary

  • action-observation trajectories: Sequences of alternating actions taken by an agent and the resulting observations from an environment, used as reusable procedural knowledge. "reuse action-observation trajectories for downstream decision-making."
  • ALFWorld: A text-based interactive benchmark for household tasks used to evaluate agents in simulated environments. "ALFWorld \citep{shridhar2021alfworld}, a text-based household-task environment,"
  • candidate generation: The first-stage retrieval step in a ranking pipeline that selects a pool of potentially relevant items for further reranking. "Learning to rank pipelines consist of a retrieval stage known as candidate generation, followed by a feature-based ranking stage"
  • cascaded retrieval pipeline: A multi-stage retrieval setup where initial retrieval is followed by a reranking model to refine results. "or a cascaded retrieval pipeline combining an initial retriever with a reranker."
  • consumer agents: Agents that retrieve shared artifacts from a repository to aid their own task execution. "we refer to agents that contribute trajectories to DD as producer agents and those that retrieve from it to aid their own task-solving as consumer agents"
  • dense index: A vector-based retrieval index storing dense embeddings of documents for similarity search. "inserted into the dense index D0D_0"
  • dense retriever: A retrieval model that uses dense vector embeddings to find semantically similar items. "may be instantiated as a dense retriever, or a cascaded retrieval pipeline"
  • E5-Base: A specific sentence embedding model used to encode queries and documents for retrieval. "we use the E5-Base embedding model \citep{wang2022text} as the shared embedding function ff."
  • embedding function: A model that maps text (e.g., queries and documents) to vector representations for retrieval. "encoded with the shared embedding function ff"
  • feature map: A function that extracts numerical features from a query–document pair for use by a ranking model. "Let ϕ(q,d)∈Rz\phi(q, d) \in R^{z} be a feature map that extracts features by inspecting the query qq and document key dd."
  • inference-time scaling: Increasing computational effort at inference (e.g., more steps or samples) to improve agent performance without additional training. "Many modern agents rely on inference-time scaling and generate a number of intermediate artifacts,"
  • LambdaMART: A popular gradient-boosted decision tree algorithm for pairwise learning-to-rank. "pairwise LambdaMART~\citep{wu2010adapting}"
  • learning-to-rank (LTR): A set of machine learning techniques that learn to order items by relevance for retrieval tasks. "using lightweight learning-to-rank (LTR) rerankers \citep{cao2007ltr}"
  • Learning To Rank Trajectories (LTRT): An LTR-based approach specialized for scoring and reranking retrieved trajectory chunks by predicted utility. "Learning To Rank Trajectories (LTRT)"
  • marginal utility: The incremental improvement in task performance attributable to adding a retrieved item compared to a baseline without it. "we label trajectory chunks by their marginal utility"
  • Multi-Agent Transactive Memory (MATM): A shared, population-level repository where agents contribute and retrieve trajectories to improve collective performance. "We propose Multi-Agent Transactive Memory (MATM), a framework for population-level storage and retrieval of agent-generated trajectories,"
  • Pareto-dominance: A comparison notion where one solution is better if it is at least as good in all metrics and strictly better in at least one (e.g., success and steps). "which measures the Pareto-dominance of trajectories between a candidate model and a fixed baseline"
  • producer agents: Agents that contribute their generated trajectories or artifacts to a shared repository. "we refer to agents that contribute trajectories to DD as producer agents and those that retrieve from it to aid their own task-solving as consumer agents"
  • producer-agent metadata: Attributes describing the producing agents (e.g., capability signals) used to model trust and improve retrieval. "Producer-agent metadata is designed to enable a form of trust modeling,"
  • Retrieval-augmented generation (RAG): A paradigm where LLMs condition generation on retrieved external documents. "retrieval-augmented generation (RAG) \citep{lewis2020retrieval}"
  • RetrievalPlanner: An LLM component that decides when an agent should issue retrieval calls during interaction. "Each consumer agent is equipped with a RetrievalPlanner LLM"
  • return-paired preference (RPP): A metric that compares pairs of trajectories by success and efficiency to assess overall improvement over a baseline. "we adopt return-paired preference (RPP) \citep{diaz2026rpp}"
  • roll in: Executing a policy up to a certain step to create a partial trajectory prefix for intervention or evaluation. "we roll in to the corresponding prefix ht=(τ1,…,τt)h_t = (\tau_1, \dots, \tau_t)"
  • roll out: Continuing from a prefix to complete trajectories, often to evaluate the impact of an intervention. "We then roll out ∣I∣|\mathcal{I}| one-shot trajectory-augmented generations"
  • state-conditioned key-value indexing scheme: An indexing method where recent interaction history forms the key and subsequent steps form the value for retrieval. "we adopt a state-conditioned key-value indexing scheme"
  • state-conditioned retrieval: Retrieval that conditions on an agent’s current interaction state rather than only the initial task instruction. "but state-conditioned retrieval over action-observation histories has received much less attention"
  • SVMRank: A pairwise ranking algorithm based on support vector machines used for learning-to-rank. "pairwise SVMRank~\citep{joachims2006training}"
  • transactive memory: A theory of distributed group memory where members rely on each other’s knowledge and retrieval mechanisms. "based on the concept of transactive memory \citep{wegner1987transactive}"
  • trust modeling: Estimating the reliability or competence of information sources (e.g., producers) to prioritize high-quality artifacts. "including agent-specific personalization, producer trust modeling, and periodic update of retriever"
  • two-sided marketplace: A platform dynamic where producers supply artifacts and consumers retrieve them, creating mutual value. "This producer-consumer structure induces a two-sided marketplace for agent-generated procedural knowledge,"
  • WebArena: A web navigation benchmark used to evaluate agents in realistic browser-based tasks. "WebArena \citep{zhou2024webarena}, a web navigation-based task environment."
View Paper Prompt  View All Prompts 

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Tweets

Sign up for free to view the 6 tweets with 49 likes about this paper.

Sign Up for Free