Selective Foveation Memory

• Selective foveation memory is a design pattern that allocates high-fidelity retention to salient events while using compressed representations for less critical data.
• It employs gating mechanisms and context-sensitive criteria to determine which experiences are stored in durable substrates like language adapters and visual tokens.
• Empirical results demonstrate that selective retention enhances goal-conditioned behavior and efficiency compared to exhaustive memory storage methods.

Searching arXiv for the cited papers to ground the article in current literature. to=arxiv_search.search 天天彩票中奖{"query":"id:(Han, 25 Jun 2026) OR id:(Min et al., 22 Apr 2026) OR id:(Kang et al., 13 Apr 2026) OR id:(Reddy, 28 Jun 2026) OR id:(Wu et al., 24 Apr 2026) OR id:(Gu et al., 22 Apr 2026) OR id:(Fioresi et al., 5 Feb 2026) OR id:(Bursa, 4 Jan 2026) OR id:(Bhat, 2024) OR id:(Jin et al., 2020) OR id:(Killick et al., 2023)","max_results":10,"sort_by":"submittedDate"}]} Searching broader arXiv for additional recent uses of foveation-selectivity terminology in memory, reasoning, and agent contexts. to=arxiv_search.search 】【。】【”】【json {"query":"(\"foveation\" OR \"foveated\") AND (memory OR agent OR retrieval OR reasoning)", "max_results":10, "sort_by":"submittedDate"}{Jsii Selective foveation memory denotes a family of memory designs in which an agent does not preserve all experience at uniform fidelity. Instead, a small subset of events, observations, or latent updates judged salient for future behavior is retained in a durable or high-resolution form, while the remainder is left in shallow retrieval stores, compressed summaries, decaying representations, or omitted entirely. In recent work, this pattern appears in language-agent memory as a distinction between memory access and memory depth (Han, 25 Jun 2026), in vision-language reasoning as stateful acquisition of high-resolution evidence within a single decoding trajectory (Min et al., 22 Apr 2026), and in social robotics as context-selective storage of emotionally salient or novel episodes (Kang et al., 13 Apr 2026).

1. Conceptual scope and defining distinctions

A central distinction in the contemporary literature is between access and depth. External retrieval systems answer what can be fetched from a store, whereas memory depth concerns what continues to shape behavior after the relevant text or observation is no longer present in working context. The formulation in long-running language agents makes this explicit: retrieval can preserve factual access, but durable goal-conditioned tendencies require selective parametric consolidation into weights or adapters that survive context unload and long-loop interference (Han, 25 Jun 2026).

Within this framing, “foveation” is best understood as selective allocation of representational fidelity rather than as a purely spatial mechanism. In visual systems, foveation classically denotes high-acuity processing of a small region with lower-resolution peripheral coverage. Recent agent-memory work extends the same organizational principle to events, reasoning steps, and retrieved traces. A small subset of inputs receives privileged retention, while the majority remains available only through cheaper or less behavior-shaping channels. This suggests that selective foveation memory is not a single architecture but a design pattern spanning parametric memory, KV-cache accumulation, vector-store retention, graph-structured context construction, and forgetting policies.

The scope of selection also varies by substrate. In language agents, the selected object can be an event admitted into a write buffer and then consolidated into a LoRA adapter. In vision-LLMs, it can be a crop of the image whose tokens are injected back into the autoregressive context. In multimodal social robotics, it can be an episodic scene retained because it exceeds emotion or novelty thresholds. In retrieval systems with bounded capacity, it can be a memory entry protected from eviction because it has high observed utility, success, or specificity (Han, 25 Jun 2026, Min et al., 22 Apr 2026, Kang et al., 13 Apr 2026, Reddy, 28 Jun 2026).

2. Selection signals, gates, and write criteria

Selective foveation memory is operationalized through gating rules that decide which inputs deserve durable or high-fidelity retention. In the EVAF mechanism for long-running language agents, the gate is explicitly multiplicative over surprise and valence. For event $x_t$, surprise $s_t$ is derived from token negative log-likelihood and valence $v_t$ from embedding similarity to the user’s durable goal and preferences, with

$$g_t = \sigma(k_s(s_t-\tau_s))\cdot\sigma(k_v(v_t-\tau_v)).$$

If $g_t>\tau_w$, the event is admitted to a small write buffer. The reported constants include per-user median warmup for $\tau_s$, fixed gate slopes $k_s=1$, $k_v=10$, and $\tau_v=0.5$, $\tau_w=0.5$ across seeds (Han, 25 Jun 2026).

A related but non-parametric formulation appears in context-selective multimodal memory for social robots. SUMMER encodes only moments that are emotionally salient or novel. Emotional salience is derived from emotion intensities $s_t$0 and per-emotion thresholds $s_t$1, with the intended normalization

$s_t$2

followed by

$s_t$3

Novelty is defined as the minimum cosine distance from the current scene embedding to previously stored scenes for the same user, and storage occurs when emotional salience is positive or novelty exceeds $s_t$4 (Kang et al., 13 Apr 2026).

In dynamic retrieval memory, selectivity can be expressed through remembrance and decay rather than a one-shot write gate. ARM maintains per-item usage count $s_t$5, last-access time $s_t$6, and a binary remembered flag. Retrieved items are reinforced by incrementing $s_t$7; once $s_t$8, the item becomes remembered and is protected from decay. Unremembered items older than a grace period $s_t$9 are multiplicatively decayed by $v_t$0, with default $v_t$1, $v_t$2, and $v_t$3 (Bursa, 4 Jan 2026).

Selective forgetting frameworks invert the same logic. FSFM computes a weighted importance score

$v_t$4

with pilot weights $v_t$5 and strong negative security penalties such as dangerous $v_t$6 and sensitive $v_t$7. Under capacity pressure, low-scoring memories are pruned or deleted (Gu et al., 22 Apr 2026).

These mechanisms differ in substrate, but they share a common structure: salience is decomposed into interpretable factors; admission or preservation is sparse; and persistence is not equivalent to mere availability. This suggests that foveation memory is fundamentally a policy over which experiences deserve durable influence rather than only a policy over where to look.

3. Memory substrates and architectural realizations

Selective foveation memory has been instantiated across several substrates, each defining a different notion of what it means for a selected item to persist.

In language agents, EVAF uses a LoRA adapter as a compact writable substrate. Consolidation occurs only on buffer trigger and includes replay of prior consolidated events plus an L2 anchor to restrain adapter drift:

$v_t$8

where $v_t$9 (Han, 25 Jun 2026). The retained content is therefore not explicit text but durable parametric biasing toward goal-conditioned behavior.

In Foveated Reasoner, the memory substrate is the autoregressive context itself. The system begins with a low-resolution image and instruction tokens, maintains a memory buffer $$g_t = \sigma(k_s(s_t-\tau_s))\cdot\sigma(k_v(v_t-\tau_v)).$$0 implemented as the Transformer KV-cache, and injects selected high-resolution evidence into the same decoding trajectory when the token policy emits <fov>. The environment returns $$g_t = \sigma(k_s(s_t-\tau_s))\cdot\sigma(k_v(v_t-\tau_v)).$$1, and the memory updates by $$g_t = \sigma(k_s(s_t-\tau_s))\cdot\sigma(k_v(v_t-\tau_v)).$$2 (Min et al., 22 Apr 2026). Persistence is thus stateful but bounded by context growth.

In SUMMER, the retained unit is a multimodal episodic trace stored in ChromaDB. Each stored memory can include the raw image frame, a Moondream2 caption, text and image embeddings, timestamps, and user identity, with retrieval conditioned on the current identified user (Kang et al., 13 Apr 2026). The retained object is explicit and queryable rather than parametric.

ContextWeaver uses a graph of reasoning steps. Each node is

$$g_t = \sigma(k_s(s_t-\tau_s))\cdot\sigma(k_v(v_t-\tau_v)).$$3

and retrieval is graph traversal rather than similarity lookup. The ancestry set of the current node is kept at full fidelity, while non-ancestors retain Thought and Action but have Observation compressed by controlled elision (Wu et al., 24 Apr 2026). Here foveation is dependency-structured rather than salience-scored in the conventional retrieval sense.

TraceRetain, LTS, and related systems operate over bounded external memory. TraceRetain scores episodic summaries by interpretable features such as success, age, access frequency, redundancy, specificity, and downstream utility, then evicts the lowest-scoring items at capacity (Reddy, 28 Jun 2026). LTS uses a global shared memory bank of key–value pairs $$g_t = \sigma(k_s(s_t-\tau_s))\cdot\sigma(k_v(v_t-\tau_v)).$$4 for parallel agent teams, with a lightweight controller making a binary YES/NO admission decision after every agent step (Fioresi et al., 5 Feb 2026).

System	Selective unit	High-fidelity retention form
EVAF	Goal-relevant event	LoRA adapter update
Foveated Reasoner	High-resolution crop	Injected tokens in KV-cache
SUMMER	Emotionally salient or novel scene	Image, caption, embeddings, timestamps
ContextWeaver	Dependency-critical step	Full Thought–Action–Observation node
TraceRetain	Episodic trajectory summary	Protected entry in bounded memory
LTS	Cross-team useful intermediate step	Shared key–value memory item

This diversity shows that selective foveation memory is not tied to a single implementation choice. What unifies these systems is asymmetric fidelity allocation: a small set of retained items receives privileged future influence, while the rest persists only in reduced, delayed, or query-contingent form.

4. Empirical trade-offs: access, persistence, efficiency, and noise

The most explicit empirical split between shallow access and deep persistence appears in the loop-drift protocol for long-running language agents. With the retrieval index intact but working context unloaded, RAG is strongest on short-fact accuracy, reaching 0.973 on TinyLlama and 0.956 on GPT-2, whereas EVAF is strongest on goal persistence and post-unload recovery: 0.833 and 0.812 on TinyLlama, and 0.904 and 0.900 on GPT-2. EVAF does this with only about 2.4–2.6 writes per 200 events, compared with 200 writes for Naive-LoRA; corresponding L2 drift is 21.5 and 29.0 for EVAF versus about 67.5 and 119.3 for Naive-LoRA on TinyLlama and GPT-2 respectively (Han, 25 Jun 2026).

The same study also shows complementarity rather than substitution. Routed EVAF+RAG sends factual probes to retrieval and goal probes to EVAF. On GPT-2 it achieves short-fact 0.935 with goal/post-unload 0.927/0.925, while on TinyLlama it reaches short-fact 0.975 with goal/post-unload 0.773/0.750 (Han, 25 Jun 2026). A plausible implication is that selective foveation memory is most effective when paired with broad-access retrieval instead of replacing it.

In stateful visual reasoning, FoveateR quantifies the computational side of selective acquisition. At $$g_t = \sigma(k_s(s_t-\tau_s))\cdot\sigma(k_v(v_t-\tau_v)).$$5 input, FoveateR-3B uses approximately $$g_t = \sigma(k_s(s_t-\tau_s))\cdot\sigma(k_v(v_t-\tau_v)).$$6 visual tokens on average versus 1152 for multi-pass baselines; at $$g_t = \sigma(k_s(s_t-\tau_s))\cdot\sigma(k_v(v_t-\tau_v)).$$7, it uses approximately $$g_t = \sigma(k_s(s_t-\tau_s))\cdot\sigma(k_v(v_t-\tau_v)).$$8 versus 1152. The method is trained by a two-stage pipeline of cold-start supervision followed by reinforcement learning, and the paper reports stronger accuracy under tight visual-token budgets across multiple vision-language benchmarks (Min et al., 22 Apr 2026).

In social robotics, selectivity is evaluated both at storage time and retrieval time. SUMMER’s selective storage mechanism reaches a Spearman correlation of $$g_t = \sigma(k_s(s_t-\tau_s))\cdot\sigma(k_v(v_t-\tau_v)).$$9 against human memorability ratings, exceeding human consistency at $g_t>\tau_w$0. In multimodal retrieval, fusion improves Recall@1 by up to 13\% over unimodal text or image retrieval; concrete examples include Flickr30k with CLIP ViT-L/14 plus jina text improving from text 59.6 to fusion 72.6 and from image 64.7 to 72.6. Runtime remains below the two-second conversational threshold, with total interaction around $g_t>\tau_w$1 s and perception-side memory encoding around $g_t>\tau_w$2 s (Kang et al., 13 Apr 2026).

In bounded episodic memory for frozen LLM agents, TraceRetain shows that retention policy matters primarily under pollution. On clean ALFWorld at $g_t>\tau_w$3 to $g_t>\tau_w$4, differences among bounded policies lie within Wilson 95\% CIs. Under a 75\% synthetic distractor stress, however, unbounded memory and FIFO-K50 degrade on Precision@5 from 20.2\% to 12.4\% and from 15.8\% to 3.8\%, while TraceRetain-CEM remains essentially unchanged at 16.9\% to 16.6\% and preserves 97/100 task success (Reddy, 28 Jun 2026).

Parallel agent memory exhibits a similar pattern. LTS learns selective shared-memory admission for parallel teams and reduces runtime while matching or improving accuracy. On AssistantBench with Qwen3-32B, mean runtime falls from 2239 s to 1479 s; on GAIA with the same backbone, from 1569 s to 892 s. For GPT-5.1, runtime falls from 1389 s to 882 s on AssistantBench and from 1005 s to 781 s on GAIA, with corresponding accuracy improvements from 24.0 to 26.7 and from 47.9 to 49.1 (Fioresi et al., 5 Feb 2026).

Dynamic RAG memory provides a lighter-weight retrieval analogue. ARM reports NDCG@5 $g_t>\tau_w$5 and Recall@5 $g_t>\tau_w$6 with approximately 22M embedding parameters, while GPT-4o with a dynamic selective retrieval policy reaches 8.2 s average response time with key-term coverage 58.7\%; by contrast, Llama 3.1 with static RAG achieves the highest key-term coverage at 67.2\% with moderate latency (Bursa, 4 Jan 2026).

5. Theoretical interpretations and formal abstractions

Several recent formalisms treat selective memory as controlled compression rather than mere storage. In selective state space models, the hidden state $g_t>\tau_w$7 is a compressed summary of past inputs, and selectivity is introduced by a gate $g_t>\tau_w$8 that modulates which dimensions are updated:

$g_t>\tau_w$9

The paper defines an effective dimensionality

$\tau_s$0

and analyzes the compression–retention trade-off through mutual information, rate–distortion, and information bottleneck formulations (Bhat, 2024). This provides a formal analogue of foveation: only a subset of state dimensions is actively updated, while the rest is retained with lower effective write bandwidth.

The language-agent literature adds a second axis absent from standard compression analyses: actuation. EVAF’s mechanism audits separate selection—which events are admitted—from actuation—how strongly admitted events are written into the adapter. Matched random gates show that sparsity alone does not explain performance, while fixed-inner controls across GPT-2, TinyLlama, and Mistral-7B show that inner-loop write strength is model-dependent (Han, 25 Jun 2026). This suggests that selective foveation memory has at least two orthogonal control problems: identifying the correct target for retention and calibrating how deeply that target should alter the memory substrate.

A third abstraction is the center–periphery dynamic found in remembrance and forgetting systems. ARM’s remembered items form a protected core, while unremembered stale items undergo geometric decay; FSFM similarly treats importance as a dynamically updated score subject to exponential forgetting and capacity-constrained pruning (Bursa, 4 Jan 2026, Gu et al., 22 Apr 2026). Although these systems are not framed as foveation in a visual sense, they instantiate the same asymmetry: central traces remain stable and high influence, peripheral traces fade or are removed.

Graph-structured memory introduces yet another interpretation. ContextWeaver’s ancestry set is the high-resolution region, and dependency summaries act as compressed causal paths. The selective object is not a raw observation but a logically justified subgraph of prior reasoning steps (Wu et al., 24 Apr 2026). This suggests that foveation in memory can be topological as well as statistical: what is preserved at full detail is determined by dependency structure, not only by feature-level salience.

6. Misconceptions, unresolved boundaries, and research directions

A common misconception is that retrieval alone solves long-term memory for agents. The loop-drift results do not support that view: retrieval preserves short-fact access, but durable goal-conditioned behavior can drift once working context unloads, even with the index intact (Han, 25 Jun 2026). Another misconception is that sparse writing is sufficient. EVAF’s matched random-gate controls show that selection is not equivalent to sparsity, and the Mistral-7B inversion demonstrates that miscalibrated actuation can make random matched gates outperform salience-based selection (Han, 25 Jun 2026).

A related confusion is to equate high similarity with high utility. TraceRetain reports that unbounded memory under noise has the highest mean retrieval similarity, 0.87, but the lowest precision, because failed distractors cluster near the query in embedding space. Selective retention improves Precision@5 by suppressing redundant or failed traces rather than by maximizing similarity alone (Reddy, 28 Jun 2026).

It is also inaccurate to restrict foveation to spatial ROI selection. Foveated Reasoner is spatial and stateful, but SUMMER is event-level and user-conditioned, ContextWeaver is dependency-structured, and retention systems such as ARM, FSFM, TraceRetain, and LTS are organized around frequency, utility, safety, or cross-team reuse rather than image coordinates (Min et al., 22 Apr 2026, Kang et al., 13 Apr 2026, Wu et al., 24 Apr 2026, Bursa, 4 Jan 2026, Gu et al., 22 Apr 2026, Reddy, 28 Jun 2026, Fioresi et al., 5 Feb 2026). This suggests that “foveation” is most productively understood as an allocation principle over limited memory resources.

Several boundaries remain unresolved. EVAF’s Memora diagnostic shows stale-memory invalidation as an unsolved problem: on the full 10-persona setting, Frozen gives 91/222 false-answer outcomes versus 95/222 for EVAF, with McNemar $\tau_s$1; on the high-mutation slice, 49/126 becomes 52/126 with $\tau_s$2, both not significant (Han, 25 Jun 2026). SUMMER lacks explicit decay, consolidation, or conflict resolution, and its selective-storage evaluation uses 81 synthetic social-interaction images rather than large naturalistic corpora (Kang et al., 13 Apr 2026). FoveateR notes that memory grows linearly with injected tokens in the KV-cache, and suggests compact alternatives such as state-space models for long horizons (Min et al., 22 Apr 2026). ContextWeaver is sensitive to the quality of parent linking and is strongest in structured, test-driven domains (Wu et al., 24 Apr 2026). LTS has no explicit eviction policy and leaves retrieval specifics implicit (Fioresi et al., 5 Feb 2026). ARM does not learn $\tau_s$3, $\tau_s$4, or $\tau_s$5 from feedback, and FSFM reports domain-specific results from telecom interactions with longer-horizon longitudinal effects not yet measured (Bursa, 4 Jan 2026, Gu et al., 22 Apr 2026).

Taken together, these open problems point toward a more complete selective foveation memory architecture with four properties: calibrated selection, calibrated actuation, explicit lifecycle management for deletion and reconsolidation, and substrate-aware control over memory growth. Current systems typically solve only a subset of these. The literature therefore converges on a stable core claim rather than a finished architecture: under bounded compute, bounded context, or bounded storage, selective retention of a small, behaviorally meaningful subset is often more important than universal access to everything.