Neural Procedural Memory: Empowering LLM Agents with Implicit Activation Steering

Abstract: While LLMs excel as static solvers, transforming them into autonomous agents remains challenging. This transition requires continuous environmental interaction, yet current agents lack the necessary persistent procedural memory. Existing approaches predominantly employ Retrieval-Augmented Generation (RAG) to inject explicit textual guidelines into model contexts. However, relying solely on symbolic instructions can introduce a text-action disconnect, frequently failing to activate the internal representations necessary for correct task execution. To address this, the paper introduces Neural Procedural Memory (NPM), a training-free framework that represents agent memory through implicit activation steering rather than explicit instructions. By distilling procedural skills from historical contrastive experiences into steering vectors in the activation space, NPM directly activates the task-relevant neural mechanisms to guide task execution. Evaluations across four agent benchmarks show that NPM performs comparably to baselines using explicit textual instructions. Furthermore, the results show that combining implicit steering with explicit workflows provides complementary advantages, leading to more robust task execution. Representational analyses indicate that these steering vectors encode consistent task logic, forming organized structures within the activation space. These findings suggest that implicit activation steering provides a promising approach for managing agent memory.

• The paper introduces a training-free framework that extracts task-specific steering vectors from successful and failed trajectories, using contrastive experience analysis and inference-time residual-stream intervention.
• NPM outperforms static activation-steering baselines and often matches or exceeds explicit textual memory, while the hybrid NPM-plus-Workflows method achieves the best average scores across MiniCPM3-4B, Qwen3-4B, and Qwen3-8B.
• The results show that implicit procedural memory can improve execution without adding context tokens or meaningful latency, but deployment requires white-box model access, successful experience data, and better stage-aware intervention methods.

Motivation: from textual to implicit procedural memory

LLM-based agents increasingly operate in long-horizon interactive environments, where persistent procedural memory—the ability to convert past trajectories into reusable execution skills—is a prerequisite for autonomy. The dominant approach encodes such memory as explicit text retrieved into context (e.g., Reflexion-style insights or Agent Workflow Memory), but this suffers from a text-action disconnect: agents may comprehend retrieved instructions yet fail to execute them faithfully, omitting intermediate steps over extended horizons. The paper argues, drawing on cognitive neuroscience's distinction between declarative and non-declarative (non-verbalizable) memory systems, that procedural skills are better represented as modulations of internal neural activity than as symbolic instructions. It proposes Neural Procedural Memory (NPM), a training-free framework that stores agent experience as steering vectors in activation space and injects them into the residual stream at inference time (2606.29824).

Framework design

NPM operates through a three-phase pipeline of contrastive experience construction, vector extraction, and inference-time intervention.

Contrastive experience construction. The framework builds paired reasoning segments $(x^+, x^-)$ at two granularities. Inter-trajectory contrast aligns successful and failed full trajectories for a given task, capturing global behavioral shifts toward success. Intra-trajectory contrast addresses the sparse-success regime common in complex tasks by exploiting failures alone: within a single failed trajectory, steps are partitioned into degenerate sets (detected via redundancy heuristics for repeated commands and invalidity checks against environment error feedback) and effective sets, and the two groups are contrasted collectively rather than step-by-step to reduce noise.

Vector extraction. Hidden states are extracted per layer—last-token states for inter-trajectory pairs, mean-pooled token representations for intra-trajectory step groups—and stored as task-specific contrastive pair sets. At inference, a dense retriever fetches the top-$K$ similar historical tasks, pools their contrastive pairs, mean-centers them, and extracts the first principal component as the steering direction $\mathbf{v}_l(q)$. During generation, activations are intervened via $$\tilde{\mathbf{h}}_{l,t} = \mathbf{h}_{l,t} + \alpha \cdot \mathbf{v}_l(q)$$, with $\alpha$ calibrated online using a KL-divergence constraint adapted from dynamic activation composition work so that intervention strength stays below a divergence threshold.

Granularity selection. A heuristic routes between granularities based on estimated task complexity: if the average length of retrieved successful trajectories exceeds $1.5\times$ the benchmark average, intra-trajectory (local error-correction) steering is used; otherwise inter-trajectory (global planning alignment) steering applies. The authors concede this heuristic occasionally underperforms an oracle granularity choice in boundary cases.

Main results

Evaluations cover MiniCPM3-4B, Qwen3-4B, and Qwen3-8B on ALFWorld (success rate), WebShop, ScienceWorld, and BabyAI (average reward), against no-memory, explicit-textual (Insights from ExpeL; Workflows from Agent Workflow Memory), and static implicit baselines (CAA, Mass-Mean).

Method	MiniCPM3-4B Avg	Qwen3-4B Avg	Qwen3-8B Avg
No Memory	22.60	28.14	30.63
Insights (explicit)	28.53	29.86	31.84
Workflows (explicit)	32.68	34.23	37.90
CAA (static implicit)	18.58	16.84	22.79
Mass-Mean (static implicit)	24.71	26.04	31.89
NPM (implicit)	28.87	31.39	36.32
NPM + Workflows (hybrid)	34.47	37.60	41.89

Three findings stand out. First, NPM consistently outperforms static steering baselines, which can degrade performance below the no-memory baseline (CAA drops Qwen3-4B to 16.84 on average); this indicates that fixed dataset-wide mean differences fail to capture multi-step procedural skills, whereas retrieval-conditioned, task-specific synthesis provides more accurate behavioral correction. Second, NPM is competitive with explicit textual memory: it beats Insights on average across all backbones and exceeds Workflows in some settings (e.g., WebShop with Qwen3-4B: 48.00 vs. 45.73), while consuming zero context-window tokens. Third, the hybrid configuration achieves the best results everywhere—for instance, ALFWorld success rises to 66.42% on Qwen3-8B versus 62.69% for Workflows alone—supporting the claim that symbolic workflows supply high-level planning structure while implicit steering enforces adherence during execution.

Ablations show that neither granularity dominates universally: intra-trajectory steering favors step-intensive environments like ALFWorld, inter-trajectory steering favors macro-planning tasks like WebShop, and the dynamic selection mechanism reduces variance relative to either alone.

Representational analysis

The paper provides several analyses supporting the claim that steering vectors encode structured procedural knowledge rather than noise:

• Linear separability. Linear SVMs trained on hidden states distinguish successful from degenerate reasoning modes with very high accuracy (e.g., 99.99% for intra-trajectory contrasts on ALFWorld with Qwen3-4B; ≥88.46% even for the harder inter-trajectory case on WebShop), justifying PCA-based directional extraction.
• Geometric consistency. Pairwise cosine similarity heatmaps reveal diagonal block structure when vectors are grouped by task type, with related tasks sharing sub-actions (PickCool/PickHeat/PickTwo) showing elevated cross-task similarity. Intra-trajectory vectors also cluster by interaction target (DeskLamp, Drawer), whereas inter-trajectory vectors remain less structured due to contextual variance in global reasoning paths.
• Mechanistic decomposition. Sparse dictionary learning yields interpretable behavioral primitives annotated via mutual information with action types plus LLM-based labeling (e.g., "premature task completion," "systematic cabinet searching"). In a PickHeat case study, inter-trajectory steering amplified early-planning features and shortened a 33-step trajectory to 14 steps, while intra-trajectory steering amplified search/container features and suppressed premature-termination signals. Entropy analysis confirms distinct mechanisms: inter-trajectory vectors are more distributed (mean normalized entropy 0.919 vs. 0.885), intra-trajectory vectors more concentrated (top-3 concentration 43.6% vs. 38.0%).
• Retrieval scaling. General behavioral features grow monotonically with retrieval pool size, but task-specific primitives peak at moderate scale before declining under cross-task interference—empirically motivating the dynamic, contextually constrained synthesis.
• Efficiency. Storage is fixed per trajectory (~45 KB for Qwen3-4B across three layers), independent of trajectory length; prefill latency (71.09 ms) is nearly identical to the no-memory baseline (63.46 ms) versus 279.89 ms for textual memory, since steering avoids expanding the attention context.

Limitations and open questions

The paper is explicit about several constraints. NPM requires white-box access to the residual stream, excluding closed API models. Repository construction depends on occasionally sampling successful trajectories, creating a cold-start problem in environments where successes are rare. Degenerate-step identification relies on deterministic heuristics (redundancy loops, format violations) that miss implicit logical fallacies without explicit environment errors—an LLM-judge audit measured an overall false negative rate of 6.34%, reaching 9.30% on WebShop, meaning some genuine failures contaminate the positive pool. Finally, the synthesized vector is a single static modulation applied uniformly throughout generation, lacking stage-aware flexibility; the authors identify dynamically adaptive interventions across execution phases as an open problem, alongside better granularity-allocation strategies than the current length-threshold heuristic.

Conclusion

NPM demonstrates that procedural memory for LLM agents can be carried by implicit activation steering rather than explicit text: dual-granularity contrastive extraction produces task-conditioned steering vectors that match textual-memory baselines, combine synergistically with them, and decompose into interpretable, behaviorally consistent primitives—all without parameter updates, context expansion, or meaningful latency cost. The framework's dependence on open-weight architectures, successful-trajectory availability, and static per-task modulation delineates the boundary conditions within which this result holds.