Delethink: Scalable Long-Horizon Reasoning

• Delethink is a paradigm that redefines long-horizon reasoning in LLMs by segmenting chains into fixed-size chunks with brief, transferable summaries.
• It employs Chain-of-Deliberation and Self Distillation in dense retrieval to generate multiple intermediate embeddings that enhance semantic matching.
• In reinforcement learning tasks, Markovian chunking ensures constant memory usage and improved compute efficiency compared to traditional LongCoT methods.

Delethink is a paradigm and methodology advancing the scalability, efficiency, and cognitive quality of long-horizon “thinking” in LLMs. Delethink refers to mechanisms in both dense retrieval and reinforcement learning (RL) reasoning tasks that restructure the model’s reasoning environment: either by enforcing a stepwise, deliberative representation (as in dense retrievers) or by Markovian chunking (as in RL), with shared goals of enhancing representation fidelity and compute efficiency. Key facets include Chain-of-Deliberation, Self Distillation, and Markovian Thinking environments, all designed to ensure nuanced chain-of-thought reasoning with linear compute and robust downstream utility.

1. Reconfiguring the Reasoning Environment

Delethink redefines how LLMs process and represent extended chains of thought. In RL applications, the standard Long Chain-of-Thought (LongCoT) environment concatenates all prior reasoning tokens with the initial query, creating an unbounded state and incurring quadratic compute and memory costs due to the self-attention mechanism. In contrast, Delethink segments reasoning into fixed-size “chunks”—at each chunk boundary, the context resets, retaining only a short, textual carryover (“Markovian state”) that summarizes the necessary information for seamless continuation. The process is governed by:

$$T_{max} = \mathcal{C} + (I - 1) \cdot (\mathcal{C} - m)$$

where $\mathcal{C}$ is the chunk context size, $m$ the carryover length, and $I$ the number of chunks. This formulation allows the total reasoning trace $T_{max}$ to be arbitrarily large while capping the per-step context size.

In dense retrieval (DEBATER), Delethink refers to multi-step deliberative encoding (Chain-of-Deliberation, CoD), producing several intermediate embeddings rather than only one per sequence. This process better captures multiple perspectives and facets within documents.

2. Markovian Thinking and Chunked Reasoning

Markovian Thinking, as instantiated in the Delethink RL environment (Aghajohari et al., 8 Oct 2025), requires that at the end of each chunk, the model compresses the necessary information into a short textual state used as input for the next chunk—akin to a Markov process where future states depend only on the present and not the full history. Formally, for chunk $l$:

$q_l = \text{Query} \oplus s_{l-1}[-m:]$

where $s_{l-1}[-m:]$ is the carryover (last $m$ tokens of the previous chunk). This structure forces the policy during RL to learn to write intermediate summaries that encode all essential reasoning up to that boundary, supporting both seamless continuation and optimal future reward.

Empirically, this architecture results in constant memory usage per chunk and throughput advantages: at 96K “thinking” token average length, compute requirements for Delethink-trained models are 7 H100-months, versus 27 for LongCoT-RL. Throughput measurements during RL confirm that fixed KV cache size maintains high training and inference efficiency.

3. Deliberative Representation and Embedding Selection

In dense retrieval, the DEBATER model (aka Delethink) (Ji et al., 18 Feb 2025) employs the Chain-of-Deliberation to iteratively encode documents by producing a sequence of intermediate embeddings $h^d_i$ corresponding to stepwise refinement. The ultimate relevance score for a query-document pair is the maximum similarity across steps:

$$f_{\max}(q, d) = \max_{i=1,\dots,m} \operatorname{sim}(h^q, h^d_i)$$

This selection acts as a dynamic match between queries and the most semantically pertinent “thought” about the document, outperforming static [EOS]-based embeddings across retrieval tasks (BEIR, TREC-COVID, NFCorpus, HotpotQA, FiQA).

Self Distillation ensures the final document embedding retains information from the most informative intermediate steps, using KL divergence between ranking probabilities:

$$L_t = \operatorname{KL}(P(\tilde{\mathcal{D}}|q) \| Q(\tilde{\mathcal{D}}|q))$$

The contrastive training objective incorporates the strongest matches from among all intermediate states, refining both local (per-step) and global (final representation) matching fidelity.

4. Performance, Scalability, and Empirical Evaluation

Delethink achieves notable empirical improvements in both RL and dense retrieval domains. In RL environments, Delethink-trained models (e.g., R1-Distill 1.5B) sustain reasoning for chains up to 24K tokens at parity or better than baselines trained on full-length LongCoT. Test-time scaling reveals continued gains as reasoning length grows, unlike LongCoT-RL which plateaus.

In dense retrieval, DEBATER implemented with small LLMs (e.g., MiniCPM-2.4B) achieves performance comparable to much larger 7B models. Ablation studies confirm that neither CoD nor vanilla LLMs alone match the combined effect of deliberative embeddings with self distillation. The architecture supports robust performance across diverse benchmarks and domains.

Table: Compute and Performance Comparison

Method	Reasoning Length (tokens)	Compute Cost (H100-mo)	Benchmark Performance
LongCoT-RL	96K	27	Baseline
Delethink-RL	96K	7	$\geq$ Baseline
DEBATER 2.4B	N/A	N/A	$\approx$ 7B Model

This suggests practical feasibility for deploying Delethink methodologies in settings with restricted computational resources or extreme output lengths.

5. Cognitive Analysis and Iterative Feedback

While Delethink’s dense retrieval and RL environments both emphasize deliberate, staged reasoning, related frameworks such as THiNK (Yu et al., 26 May 2025) highlight the importance of iterative, agent-driven feedback in advancing higher-order cognitive skill assessment. THiNK’s multi-agent system, grounded in Bloom’s Taxonomy, evaluates LLM outputs across cognitive levels from remembering to creating. Structured feedback loops guide “think-aloud” revisions, moving models from surface-level accuracy towards deeper abstraction and application.

In contrast, Delethink’s Markovian chunking implicitly encourages similar reflective intermediate summarization, as the model must continually compress its reasoning to fit within a short, transferable state. This parallel suggests an architectural alignment between RL environment design and cognitive evaluation principles.

A plausible implication is that mechanisms like Markovian chunking and agent-guided reflection may mutually reinforce long-horizon reasoning and higher-order cognitive competence.

6. Theoretical Foundations and Key Equations

Delethink’s core theoretical innovations lie in its redefinition of state, memory, and reward across chunk boundaries. The RL environment leverages fixed chunk context, Markovian carryover, and distillation of key reasoning traces—all encoded in the update and reward structure of the RL policy.

Key equations defining the process include:

• Chunked token budget: $$T_{max} = \mathcal{C} + (I - 1) \cdot (\mathcal{C} - m)$$
• Markovian prompt update: $q_l = \text{Query} \oplus s_{l-1}[-m:]$
• Retrieval relevance score: $$f_{\max}(q, d) = \max_{i=1,\dots,m} \operatorname{sim}(h^q, h^d_i)$$
• Self distillation loss: $$L_t = \operatorname{KL}(P(\tilde{\mathcal{D}}|q) \| Q(\tilde{\mathcal{D}}|q))$$
• Contrastive retrieval loss:

$$L_c = -\log \left[ \frac{\exp(f_{\max}(q, d^+))}{\exp(f_{\max}(q, d^+)) + \sum_{d^-} \exp(f_{\max}(q, d^-))} \right]$$

These constructs formalize the relationship between local chunk states and global reasoning chains, ensuring both computational tractability and representational robustness.

7. Practical Applications, Limitations, and Future Directions

Delethink’s chunked reasoning environments and multi-perspective document embeddings have direct implications for web search, question answering, fact-checking, and biomedical information retrieval. The methods support accurate, facet-rich matching between queries and documents and empower models with limited parameter count to achieve performance levels previously restricted to larger architectures.

The chunk-reset mechanisms enable inference for chains reaching tens or hundreds of thousands of tokens, previously intractable due to quadratic attention scaling. Since Delethink relies on environment design rather than model modification, it is compatible with existing transformer architectures and may be further optimized in conjunction with non-quadratic attention variants.

Future development directions include explicit learning of Markovian state representations, direct manipulation of KV caches, and harmonized integration with agent-driven feedback loops for cognitive advancement. These possibilities indicate a path toward both more intelligent and more efficient “thinking” in next-generation LLMs.

A plausible implication is the convergence of RL environment design and dense representation learning as dominant levers—not only for scaling and efficiency but also for cognitive quality and real-world utility in artificial reasoning systems.