Neural Garbage Collection (NGC)

• Neural Garbage Collection (NGC) is a set of advanced techniques that manage memory in neural systems using learned, policy-driven strategies for optimized context retention.
• It integrates indexed object tracking and decision mechanisms like reinforcement learning and planner-based lifecycle management to dynamically govern memory usage.
• Empirical benchmarks demonstrate that NGC significantly reduces stored context while maintaining high downstream performance in tasks like program synthesis and long-horizon reasoning.

Neural Garbage Collection (NGC) encompasses a set of methodologies for managing memory or context resources in neural systems—ranging from program synthesis, foundation models, to long-horizon language agents—by learning or governing when, what, and how to forget. Unlike classical garbage collection in programming language runtimes, which relies on reachability, NGC techniques leverage supervised, reinforcement, or planner-driven strategies to optimize resource retention and reclamation over the system’s reasoning or operational horizon. Key formulations appear in program synthesis with learned slot dropping, end-to-end cache management in reasoning LLMs, and runtime lifecycle control over indexed objects in tool-using LLM agents (Hao et al., 1 Jul 2026, Li et al., 20 Apr 2026, Zohar et al., 2018).

1. Distinction from Heuristic Pruning and Traditional Garbage Collection

Traditional garbage collection algorithms—such as generational or mark-and-sweep GC in runtime systems—or naïve in-context pruning heuristics (e.g., oldest-k token removal, tool-output masking, final self-summary) operate on syntactic or structural attributes, typically trading low complexity for blind spots regarding dependencies and efficiency. For example, chronological pruning drops the oldest k turns but cannot account for whether those turns encapsulate objects or evidence needed for future steps. Similarly, one-shot summaries preserve narrative but lose byte-exact artifacts and live object handles critical to downstream operations (Hao et al., 1 Jul 2026). In contrast, Neural Garbage Collection systems seek to manage resource retention as a policy that is context-, dependency-, or reward-aware, learning to govern the object lifecycle beyond static position or recency.

2. Indexed Object Reification and Memory Representation

A hallmark of advanced NGC approaches is the explicit mapping of memory units, context spans, or intermediate states into indexed objects. In agentic systems, Self-GC reifies each user turn and tool span as a first-class context object with a stable, session-local identifier—e.g., conversation:user:k for the k-th user turn, function:tool:n for tool spans—enabling the planner and the harness to refer and operate on objects by id without fuzzy text-matching (Hao et al., 1 Jul 2026). In program synthesis, NGC interprets the executor state as a bounded vector of variable slots, embedded and pooled across input-output examples, with learnable indicators for type and content (Zohar et al., 2018). This indexed view decouples memory management operations from surface ordering, allowing richer dependency and lifecycle analysis.

3. Decision Mechanisms: Planning, Learning, and Policy Optimization

NGC decision policies span explicit planning, supervised classification, and reinforcement learning:

• Planner-Based Lifecycle Management: In Self-GC, a side-channel planner is triggered on governance rounds and proposes structured GC actions for each indexed object: <fold> (migrate payload to sidecar with a recovery pointer), <mask> (drop low-signal content, retain handles), or <prune> (remove from active view). Decisions leverage dependency tests (will a future turn need this?), granularity rules (prefer tool-level action if context-carrying), and content signals (e.g., logs vs. obsolete artifacts) (Hao et al., 1 Jul 2026).
• Neural Drop Policies in Synthesis: In program synthesis (PCCoder), a “drop head” is trained jointly with next-statement and operator heads, outputting a v-way sigmoid for each variable slot indicating “safe to drop”—with losses reflecting binary future-use labels collected over program traces (Zohar et al., 2018).
• Reinforcement Learning for Forgetting: In resource-constrained reasoning tasks, the NGC policy in (Li et al., 20 Apr 2026) operates over both next-token actions and resource-management (KV cache eviction) actions. Cache-eviction is treated as a discrete decision sampled at periodic intervals; the model is optimized via outcome-based RL, with token and memory losses each modulated by group-normalized task reward. This joint policy learns which cache blocks to retain or forget to maximize downstream correctness.

4. Runtime Enforcement and Commit Policies

A robust NGC system must enforce safety, recoverability, and runtime constraints:

• Recoverable Sidecars and Commit Barriers: In Self-GC, folded payloads are moved to recoverable side storage (“sidecars”). Plans are rehearsed via dry-run—normalizing invalid edits, computing projected token savings—and only committed at safe turn boundaries if a compression threshold is met and the projected object view is valid (Hao et al., 1 Jul 2026).
• Cache-Aware Commit: Commit is further gated by a cost-benefit function:

$$\mathrm{CommitBenefit} \approx N_{\mathrm{future}} (C - C') - L_{\mathrm{cache\_break}} - L_{\mathrm{GC}}$$

where $N_{\mathrm{future}}$ is the expected number of reuse events, $C$/$C'$ the token cost before/after, and $L_{\mathrm{cache\_break}}$ and $L_{\mathrm{GC}}$ are latency/cost overheads (Hao et al., 1 Jul 2026).

• Replay Masking in RL: In end-to-end KV cache learning regimes, training with exact replay attention masks ensures proper gradient computation and contextual alignment after each eviction step (Li et al., 20 Apr 2026).

5. Empirical Benchmarks and Comparative Performance

NGC techniques have been quantitatively benchmarked in several regimes:

Setting	Approach/Baseline	Prune Rate	No-Impact (Retention) Rate	Peak KV Compression
Self-GC (33 Hard Sessions)	Oldest-turn fold	63.45%	66.67%	—
	Self-GC	43.95%	84.85%	—
Self-GC (332 Prod Sessions)	Self-GC (Qwen 3.7)	33.98%	94.58%	—
Program Synthesis (t₂=8)	DeepCoder (baseline)	—	11.2% solve rate	—
	NGC (PCCoder₈)	—	90.0% solve rate	—
Reasoning (Countdown, 50%)	No Eviction	—	53.2% accuracy	—
	NGC (RL, 2.4× reduction)	—	49.6% accuracy	2.4× peak cache reduction

Self-GC achieves a 30–44% reduction in context tokens while preserving 91–95% of real downstream continuations in production-like regimes, substantially raising the efficiency-utility Pareto front over heuristic baselines (Hao et al., 1 Jul 2026). RL-driven NGC nearly matches full-cache performance on arithmetic and math reasoning tasks at 2–3× cache compression, while statically configured heuristics degrade sharply (Li et al., 20 Apr 2026). In program synthesis, learned drop policies allow problem lengths 2–3× longer than non-NGC models, with 90%+ solve rates at moderate lengths (Zohar et al., 2018).

6. Implementation Details and Key Algorithms

Notable implementation aspects include:

• Indexed Object Tracking: Session-local monotonic allocators (for IDs) and formal object-state maps

$$\mathcal{O} = \{ o_i=(\mathrm{id}_i, \mathrm{type}_i, \mathrm{payload}_i, \mathrm{state}_i) \}$$

enable deterministic referencing and modification (Hao et al., 1 Jul 2026).

• Planner Protocols: XML-formatted action plans align fold/mask/prune actions with precise object ids, facilitating dry-run rehearsals and safe merges prior to commit (Hao et al., 1 Jul 2026).
• Policy Gradient with Replay: Eviction decisions in RL NGC are trained with Gumbel-top-k block selection over coarsened cache intervals, using replay attention masks to maintain training/inference alignment (Li et al., 20 Apr 2026).
• Environment Embedding in Synthesis: Variable slots are embedded, pooled via a 10-layer DenseNet, and processed jointly for statement/function/drop outputs—enabling concurrent learning of memory retention and operation generation (Zohar et al., 2018).
• Curriculum and Regularization: RL-based eviction policies use curriculum schedules (staircase in ε) to avoid training collapse under aggressive memory pressure (Li et al., 20 Apr 2026).

7. Limitations, Extensions, and Broader Implications

NGC approaches, while effective, have inherent trade-offs:

• Dependency on Predictive or Learned Policies: Mistakes in object-drop prediction (supervised or RL-based) can force expensive backtracking or degrade future performance, particularly when future dependencies are non-local or soft (Zohar et al., 2018, Li et al., 20 Apr 2026).
• Curriculum and Stability: RL-driven forgetting requires careful curriculum tuning (eviction rates), replay mechanisms, and reward normalization to ensure convergence and avoid catastrophic forgetting (Li et al., 20 Apr 2026).
• Potential Extensions: Promising directions include (a) meta-token “gist” compression (forcing the model to emit summaries before eviction), (b) dynamic or adaptive budget schedules, (c) resource-aware routing for modeling compute precision or expert invocation, and (d) soft memory attention for partial retention (Li et al., 20 Apr 2026, Zohar et al., 2018).

A plausible implication is that as large models and long-horizon interactive agents become increasingly common, explicit neural or planner-driven garbage collection—viewing the context as a heap of governed, recoverable objects—will be central to sustained, resource-efficient cognition.

---

Key References:

• Self-GC for LLM agents (Hao et al., 1 Jul 2026)
• RL-based cache management for chain-of-thought reasoning (Li et al., 20 Apr 2026)
• Learned variable slot dropping in neural program synthesis (Zohar et al., 2018)