Trajectory Memory Component

• Trajectory Memory Component (TMC) is a dedicated module that records, structures, and retrieves past trajectories using advanced embedding and associative memory techniques.
• It is applied in sequential decision-making, planning, and prediction to enhance sample-efficient learning and robust adaptation to distribution shifts.
• TMCs utilize architectures like key-value associative memory, hierarchical memory, and vector quantization to effectively balance generalization with scenario-specific details.

A Trajectory Memory Component (TMC) is a dedicated module within machine learning systems—particularly those addressing sequential decision-making, planning, or prediction—whose purpose is to record, structure, and retrieve historical trajectory information for use in inference or control. In modern AI and robotics, TMCs are engineered to bridge the gap between parametric models’ generalization and the specificity of previously encountered scenarios. This is achieved through mechanisms such as non-parametric retrieval, structured embedding storage, hierarchical or associative memory, and similarity-based reasoning. Recent research has demonstrated that TMCs are critical enablers of sample-efficient policy learning, contextualized multi-step planning in unfamiliar domains, and robust adaptation to distributional shift.

1. Core Architectures and Design Patterns

TMC architectures vary across domains, but share essential design elements: encoding raw trajectories into vector representations, structuring memory with appropriate indexing for fast retrieval, and defining read/write/update rules.

• Key-Value Associative Memory: In settings such as task automation and trajectory prediction, historical sequences (e.g., GUI action-observation pairs, agent tracks) are encoded by neural networks (GRUs, LSTMs, or Transformers) into fixed-dimensional keys or feature vectors. These keys are then stored alongside values, which may encode future goals, intentions, or full sequence outcomes. For instance, MapAgent’s page-memory database stores vectorized JSON summaries of GUI pages for each app, indexed in Milvus for rapid cosine similarity retrieval (Kong et al., 29 Jul 2025). MANTRA encodes past and future trajectory pairs as key-value slots for flexible multi-modal trajectory decoding (Marchetti et al., 2020).
• Hierarchical or Structured Memory: To address both short- and long-term dependencies, architectures such as Tree Memory Networks (TMN) (Fernando et al., 2017) and Structured Memory Networks (SMN) (Fernando et al., 2018) encode temporal sequences into trees or grid-based tensors, recursively aggregating local embeddings via LSTM or specialized St-LSTM gating layers.
• Discrete and Quantized Memory: FMTP (Guo et al., 2024) implements a vector-quantized memory, compressing trajectory features into a discrete codebook and using index sequences as input to transformer-based reasoning. This reduces redundancy and accelerates retrieval.
• Actor- or Instance-specific Tokens: T4P (Park et al., 2024) maintains an actor-specific token memory, where each actor carries a learnable adaptation vector updated online, crucial for handling non-stationary test-time environments.
• Clustered Pattern Memory: Memory banks constructed by clustering trajectory embeddings, as in MP²MNet (Yang et al., 2024), facilitate retrieval of motion pattern priors and enable robust conditional generation via diffusion models.

2. Mathematical Formulations and Retrieval Mechanisms

Mathematical formalism underpins TMC operation:

• Encoding and Storage: Given a trajectory $h = \{o_0, a_0, o_1, a_1, ..., o_t\}$ (where $o_i$ is an observation and $a_i$ an action), encoders such as $E(\cdot)$ (LSTM/GRU/Transformer) project raw sequences into feature vectors $v \in \mathbb{R}^d$ for storage.
• Similarity Search: Retrieval typically employs nearest neighbor selection: for a query $q$ (embedding of a current situation or subtask), memory slots $v_j$ are ranked via $$\mathrm{sim}(q, v_j) = \frac{q \cdot v_j}{\|q\|_2 \|v_j\|_2}$$ (cosine similarity) or, in discrete codebooks, by Euclidean distance or negative log-likelihood in latent space. MapAgent uses top-$k$ cosine retrieval per app collection (Kong et al., 29 Jul 2025); MANTRA ranks key similarities for multi-modal forecast (Marchetti et al., 2020).
• Trainable Memory Addressing: Some models, e.g., MemoNet (Xu et al., 2022), employ learned addresser networks to optimize memory lookup in terms of downstream trajectory or intention reconstruction, combining hard slot selection with pseudo-labeling via reconstruction error.
• Hierarchical Aggregation: Recursive Tree-LSTM or St-LSTM gating functions recursively merge sets of embeddings, maintaining both fine-grained and global temporal-spatial context (Fernando et al., 2017, Fernando et al., 2018).

3. Integration with Planning, Prediction, and Control

TMCs are tightly coupled to downstream inference modules, often as follows:

• Context Injection for LLM-driven Planning: MapAgent’s framework retrieves top-$k$ contextually similar memory “page chunks” and injects them as examples into the coarse-to-fine LLM planner, thus anchoring high-level plans in real UI contexts (Kong et al., 29 Jul 2025).
• Multi-modal Trajectory Prediction: Memory retrieval yields multiple plausible “future” trajectories for each situation; the decoder fuses observed past, retrieved future encodings, and scene context (e.g., semantic maps) for diverse forecasting (Marchetti et al., 2020, Xu et al., 2022).
• Hybrid Model-based and Episodic RL: Episodic memory modules such as in MBEC (Le et al., 2021) estimate action values from stored trajectory outcomes, then dynamically blend them with parametric DQN values via learned mixing networks, enhancing sample efficiency and robustness in high-noise or non-Markovian environments.
• Pattern-conditional Diffusion or Generative Models: In MP²MNet (Yang et al., 2024), retrieved cluster priors modulate both the score function in diffusion trajectory sampling and the local target distributions for forecasted motion.

4. Training, Update, and Optimization Strategies

TMCs depend on bespoke training schemes and updating logic:

• Autoencoder and Reconstruction Losses: Encoders and decoders for both past and future are trained jointly to minimize MSE or cross-entropy losses on reconstructed trajectories, anchoring embeddings in predictive utility (Marchetti et al., 2020, Xu et al., 2022).
• Write Controllers and Memory Growth: Writing to memory is governed by learned controllers that monitor reconstruction or prediction errors, selectively registering only “surprising” or under-represented patterns to prevent redundancy and memory bloat (Marchetti et al., 2020).
• Online and Incremental Updates: In domains requiring lifelong or adaptive learning (e.g., traffic scenarios), per-actor tokens (T4P) and memory slots can be continually refined online by gradient descent on empirical reconstruction/prediction loss, with appropriate learning rate schedules and episodic averaging (Park et al., 2024). Some models allow for optional incremental updates to cluster centers or slot statistics (Yang et al., 2024).

5. Empirical Impact and Benchmarks

Multiple studies have empirically isolated the significance of TMCs:

• Task Success and Planning Quality: MapAgent's ablations show that enabling the trajectory memory component increases single-app end-to-end task success rates from 0.531 to 0.627 (+9.6 points, English), and from 0.433 to 0.553 (+12.0, Chinese). For cross-app tasks, memory elevates English/Chinese success from ≤0.20/≤0.10 to 0.35 (Kong et al., 29 Jul 2025).
• Prediction Accuracy: FMTP achieves >10% lower ADE and FDE on ETH-UCY and >50% improvement on the inD dataset over previous memory-based methods (Guo et al., 2024). MemoNet yields 20–28% FDE reductions versus parameter-only models (Xu et al., 2022).
• Adaptation to Distribution Shift: T4P reports 40–60% reductions in mADE6 for cross-dataset shifts, sustaining ≥10 FPS inference (Park et al., 2024).
• Planning Under Noise and Non-Stationarity: MBEC’s TMC enables sample-efficient policy learning in noisy, dynamic, or partially observable environments, outperforming DQN and pairwise episodic memories (Le et al., 2021).
• Memory Efficiency: Discrete or quantized TMCs offer >10× compression and up to 2× lower per-agent inference time with minimal loss in accuracy (Guo et al., 2024).

6. Representative Implementation and Hyperparameter Choices

Recent high-impact systems detail key parameters:

System	Memory Slot Dimension	Memory Size	Retrieval k	Storage Backend
MapAgent	$d$ ≈ 1536	per-app, unbounded	k=3	Milvus 2.x, IVF index
MANTRA	$d_k=d_v=48$	~2% of dataset	k (multi-)	Array/list with controller
T4P	$D=256$ per actor	≤# active actors	n/a	Dict (actor id keyed)
FMTP	$d=32-128$	K≈768 (ablation)	hard-quantized	Learnable codebook
SMN/TMN	$l=30$/ $k=300$	spatial grid/tree	soft-attn	Grid / Tree structure

Parameters such as learning rates ($\eta=0.01$ network, $\eta_{tok}=0.5$ tokens, (Park et al., 2024)), batch size, and masking ratios are application dependent. Memory updates may be strictly online (per step/episode) or batch/epochal.

7. Challenges, Limitations, and Outlook

Despite their advantages, TMCs face challenges including:

• Redundancy Management & Scalability: Methods such as write controllers (Marchetti et al., 2020) and codebook quantization (Guo et al., 2024) are designed to minimize memory size without sacrificing coverage, but scaling to persistent lifelong agents remains an open issue.
• Retrieval Latency and Compatibility: Embedding and lookup schemes must balance speed against semantic fidelity; vector DB backends (e.g., Milvus) and quantized arrays each impose constraints.
• Domain and Task Dependence: The optimal architecture (flat, tree, spatial, associative) is domain- and task-sensitive—there is no universal recipe.

The TMC paradigm is now a foundational element of retrieval-augmented sequence modeling, RL, and planning, with emerging relevance for cross-app automation, agent adaptation, and efficient lifelong learning (Kong et al., 29 Jul 2025, Xu et al., 2022, Marchetti et al., 2020, Guo et al., 2024, Park et al., 2024).