Context-Folding in Sequential Data Compression

• Context-Folding is a paradigm that compresses sequential data into concise, multi-scale summaries while preserving critical operational details.
• It utilizes structured transformations such as agent and latent space folding, enabling long-term coherence in LLM-based systems with sub-linear context growth.
• The approach spans diverse domains—from dialogue systems and cognitive memory graphs to protein folding and formal languages—demonstrating tangible gains in efficiency and performance.

Context-Folding is a cross-domain paradigm for actively and efficiently compressing, abstracting, or structuring sequential information, enabling tractable reasoning, persistent memory, and robust generation in settings where context (input/output, history, or state) would otherwise grow without bound. The principle has been instantiated in LLM-based agent workflows, cognitive memory graphs, formal languages, high-dimensional embedding spaces, and biophysical molecular folding, with each field developing precise mechanisms and analytic perspectives on when, how, and why to fold. Context-Folding techniques balance compactness with preservation of critical, actionable detail, and facilitate multi-scale, dynamic organization of information.

1. Foundational Definitions and Problem Setting

At its core, Context-Folding denotes an operation (deterministic or learned) that replaces a segment of history or sequential data by a concise, potentially multi-scale summary, with the objective of bounding working memory while avoiding irreversible loss of essential information. In LLM agent environments, the context at step $t$ is recursively managed via operators such as:

$$\text{Context}_t = (\text{Query}, \text{Tools}, \mathbf{S}_{t-2}, \mathbf{I}_{t-1}),$$

where $\mathbf{S}_{t-2}$ is an ordered list of summaries $(s_{x_1,y_1}, s_{x_2,y_2}, ...)$. Folding at step $t$ executes a transformation:

$$f_t = \{\text{"range"}: [k, t-1],\ \text{"summary"}:\sigma_t\},$$

retracting overlapping summary blocks and inserting a new summary $s_{k,t-1}$ (Ye et al., 28 Oct 2025). Granular condensation ($k = t-1$) preserves recent detail, while deep consolidation ($k < t-1$) abstracts over sub-trajectories.

In generic theoretical and computational settings, Context-Folding formalizes the compositional reduction or abstraction of working context — for example, the map $\mathcal{F}(\tau_{<t})$ in agentic RL (Sun et al., 13 Oct 2025), the folding function $$\text{Context}_t = (\text{Query}, \text{Tools}, \mathbf{S}_{t-2}, \mathbf{I}_{t-1}),$$0 in latent space (Harcourt et al., 13 Feb 2025), or the partitioned summary-extraction operators in cognitive architectures (Wang et al., 13 May 2026). Each framework defines precise invariants and quantitative trade-offs: retention of task-relevant actionability, minimization of redundancy, control over context growth, and convergence/completeness conditions.

2. Context-Folding in LLM-Based Long-Horizon Agents

Context-Folding provides a structured remedy to the “context saturation” and “catastrophic forgetting” phenomena in LLM-driven agents. ReAct-style agents that append every reasoning step to context suffer $$\text{Context}_t = (\text{Query}, \text{Tools}, \mathbf{S}_{t-2}, \mathbf{I}_{t-1}),$$1 context-length growth, leading to degraded performance when working memory exceeds the model’s attention window. Full-history summarization inversely risks irreversible omission of critical, fine-grained operational detail.

AgentFold introduces a two-scale, proactive Context-Folding procedure (Ye et al., 28 Oct 2025): at each step, it learns (via supervised fine-tuning) when to condense only the latest atomic step (granular) versus when to collapse an entire resolved sub-problem (deep consolidation). The folding policy is formalized as an operator $$\text{Context}_t = (\text{Query}, \text{Tools}, \mathbf{S}_{t-2}, \mathbf{I}_{t-1}),$$2 and is encoded as JSON directives that govern context sculpture in real time.

Empirical evaluation indicates that AgentFold-30B-A3B achieves 36.2% on BrowseComp and 47.3% on BrowseComp-ZH, outperforming much larger proprietary and open baselines, while keeping context usage sub-linear in trajectory length (context grows from 3.5k to just 7k tokens in 100 turns, compared to $$\text{Context}_t = (\text{Query}, \text{Tools}, \mathbf{S}_{t-2}, \mathbf{I}_{t-1}),$$390k for ReAct). AgentFold enables long-horizon coherence up to 500 steps, while traditional agents saturate or fail when working memory is exceeded.

The framework is further extended by FoldAct (Shao et al., 28 Dec 2025), which situates Context-Folding in RL settings and directly addresses three challenges: (1) gradient dilution (summary tokens receive insufficient credit), (2) self-conditioning and non-stationary observation distributions (due to summaries determining future context state), and (3) computational cost (unique, compressed contexts per time step breaking KV-cache sharing). FoldAct introduces separated loss computation for summary/action tokens, full context consistency loss, and selective segment training, resulting in stable RL optimization and $$\text{Context}_t = (\text{Query}, \text{Tools}, \mathbf{S}_{t-2}, \mathbf{I}_{t-1}),$$4 speedup over full-context training.

3. Hierarchical and Latent-Space Folding Principles

Beyond sequential context logs, Context-Folding has been formalized as a structured transformation in latent spaces, notably for LLMs. Hierarchical Latent Space Folding (Harcourt et al., 13 Feb 2025) introduces an operator $$\text{Context}_t = (\text{Query}, \text{Tools}, \mathbf{S}_{t-2}, \mathbf{I}_{t-1}),$$5, parameterized per layer, which shapes token embeddings dynamically to enforce multi-scale organization:

$$\text{Context}_t = (\text{Query}, \text{Tools}, \mathbf{S}_{t-2}, \mathbf{I}_{t-1}),$$6

with $$\text{Context}_t = (\text{Query}, \text{Tools}, \mathbf{S}_{t-2}, \mathbf{I}_{t-1}),$$7 a clustering “potential” and $$\text{Context}_t = (\text{Query}, \text{Tools}, \mathbf{S}_{t-2}, \mathbf{I}_{t-1}),$$8 learned cluster centers. Energy minimization guides each representation toward compactness and coherent clustering, balancing local similarity (penalizing distortion of adjacent tokens) and long-range semantic grouping. Quantitatively, hierarchical folding yields up to 48% variance reduction in token representations across layers, 2.5–4.0 perplexity improvement, increased attention-head utilization, and a 3–5% inference speedup at the cost of $$\text{Context}_t = (\text{Query}, \text{Tools}, \mathbf{S}_{t-2}, \mathbf{I}_{t-1}),$$94.7% extra training time.

Context-Folding at the latent level thus extends the principle of compression/abstraction from explicit agent memory to neural representations, producing interpretably multi-scale, context-sensitive feature geometries.

4. Specialized Variants: User-Centric Dialogue, Always-On Memory, and Domain Applications

Context-Folding methods have been adapted to diverse architectures and problem settings, demonstrating their versatility and structural commonality:

• U-Fold (Su et al., 26 Jan 2026): In user-centric, tool-augmented dialogues, prior context-folding methods fail by irrevocably discarding constraints and missing evolving user intent. U-Fold introduces an intent-aware, evolving dialogue summary $\mathbf{S}_{t-2}$0 and a compact, task-relevant tool log $\mathbf{S}_{t-2}$1 at each turn, both dynamically updated via lightweight prompt modules and explicit to-do tracking. Compression is optimized against a performance trade-off and achieves up to 27% improvement over baselines in long-context, noisy dialogue settings.
• CogniFold (Wang et al., 13 May 2026): As a brain-inspired always-on agent memory, CogniFold organizes fragmented event streams into a multi-layer self-organizing graph (Event, Concept, Intent nodes). Folding here is a continuous operator that incrementally consolidates, merges, and decays knowledge, leading to emergence of goal-directed Intent structures as concept density rises. This cognitive folding yields strong auditability, proactivity (as measured by Intent Proactivity $\mathbf{S}_{t-2}$2), and robust benchmark performance across multi-hop QA, streaming QA, narrative comprehension, and theory of mind.
• Cloth Manipulation (BiFold) (Barbany et al., 12 May 2025): In visual policy learning for dynamic, highly-deformable objects, context-folding is instantiated as temporal context fusion (via a cross-modal transformer) over recent keyframes. This enables the policy to disambiguate occlusions and maintain implicit, persistent object state, nearly doubling pick-and-place accuracy over static input models.
• Biophysical Folding (Wells et al., 10 Jul 2025): In molecular biology, “context-folding” refers to the dependence of protein folding intermediates’ structures on the biological context (co-translational on the ribosome vs post-translational in vitro). Crystal, NMR, or cryo-EM studies confirm that intermediate-state geometries are context-dependent and that current equilibrium-based predictors (such as AlphaFold2) poorly capture transient, non-native intermediates, motivating next-generation, context-sensitive structure predictors.

5. Formal Language Theory: Folding Systems and Combinatorial Consequences

In formal language theory, context-folding is precisely instantiated by folding systems (F-systems) (Lucero, 2019). Here, an F-system is defined as a pair $\mathbf{S}_{t-2}$3 of a core language and a folding-procedure language over a “fold-up/fold-down” alphabet, where the $\mathbf{S}_{t-2}$4-step folding function $\mathbf{S}_{t-2}$5 recursively constructs the string by sequentially folding each symbol according to the procedure.

Folding classes $\mathbf{S}_{t-2}$6 (where $\mathbf{S}_{t-2}$7 and $\mathbf{S}_{t-2}$8 are language families, e.g., regular or context-free) are characterized by necessary pumping lemmas: any infinite language in $\mathbf{S}_{t-2}$9 must admit a multiblock decomposition under which repeated “pumping” in designated substrings yields new members. Lemmas are established for $(s_{x_1,y_1}, s_{x_2,y_2}, ...)$0, $(s_{x_1,y_1}, s_{x_2,y_2}, ...)$1, $(s_{x_1,y_1}, s_{x_2,y_2}, ...)$2, and $(s_{x_1,y_1}, s_{x_2,y_2}, ...)$3, with closure and necessary condition analyses underpinning the combinatorial boundaries of context-folding generativity.

6. Empirical Results, Limitations, and Prospective Directions

Empirical studies across domains demonstrate the impact of context-folding:

• On long-horizon agent benchmarks, folding-based agents (AgentFold and FoldGRPO) yield up to $(s_{x_1,y_1}, s_{x_2,y_2}, ...)$4 context efficiency, outperform context-size-matched ReAct agents, and maintain solution rates as task complexity increases (Ye et al., 28 Oct 2025, Sun et al., 13 Oct 2025).
• In dialogue, U-Fold reduces omission errors by over 50% and halves the number of redundant tool calls, with best improvements on noisy, multi-turn tasks (Su et al., 26 Jan 2026).
• For memory graphs, CogniFold uniquely achieves both high “Purity” (event-to-concept alignment) and “Proactivity,” outperforming retrieval-augmented (RAG) and episodic-only baselines (Wang et al., 13 May 2026).
• In protein dynamics, current folding predictors fail to resolve non-native intermediates, confirming a fundamental context-dependence not captured by equilibrium-based models (Wells et al., 10 Jul 2025).

Limitations include possible irreversibility of overly aggressive folding (catastrophic forgetting), path-dependence (order sensitivity) in hierarchical graph folding, and the need for more adaptive or reward-driven folding policies. Future directions emphasize adaptive triggers for folding, RL-driven folding strategies, hierarchical or multi-layer folding, integration with external memory structures, and expanding experimental and computational benchmarks, especially in molecular and high-dimensional representation settings.

7. Summary Table: Representative Context-Folding Methods and Domains

Method/Domain	Folding Operator	Compression Mechanism	Key Metrics/Results
AgentFold	$(s_{x_1,y_1}, s_{x_2,y_2}, ...)$5 (JSON)	Multi-scale in-agent context	$(s_{x_1,y_1}, s_{x_2,y_2}, ...)$610% token growth, 36-47% benchmark score
FoldAct	Separate-loss, summary action	Context-token compression in RL	$(s_{x_1,y_1}, s_{x_2,y_2}, ...)$7 speedup; ablation validates methods
Hier. Latent Folding	$(s_{x_1,y_1}, s_{x_2,y_2}, ...)$8 (layerwise)	Clustered latent representation	$(s_{x_1,y_1}, s_{x_2,y_2}, ...)$9 variance reduction, $t$0 latency gain
U-Fold	$t$1	Intent-aware summary + tool log	$t$2 avg improvement on VitaBench
CogniFold	Graph-based, merge/decay	Event $t$3 Concept $t$4 Intent	4.6$t$5 comp., Proactivity 0.61
Biophysical (Protein)	N/A (context dependence)	Biological/experimental state	Native predictor TM $t$60.5 on intermediates
Formal Language (F-sys)	$t$7 procedure-based	Folding procedure language	Pumping lemma characterizations

Each method implements domain-specific mechanisms for context reduction, memory consolidation, and multi-scale structural preservation, showing that context-folding constitutes a unifying paradigm for efficient, scalable, and robust handling of sequential and high-dimensional information across computational and physical systems.