Token Recycling: Mechanisms & Applications

• Token Recycling is a design pattern where previously generated tokens, contexts, or assets are retained and reused across AI tasks and blockchain protocols.
• It improves efficiency by caching useful artifacts—e.g., reusing prompt tokens in long-context QA can yield up to +16 absolute points and speculative decoding achieves roughly 2× speedup.
• Beyond AI inference, token recycling applies to token-economic and identity-sharing systems, enabling conversion or exchange of tokens under constrained budgets.

Searching arXiv for papers on “token recycling” to ground the article in the current literature. Searching for exact phrase and related contexts: token recycling, context recycling, TORE, speculative decoding. Token recycling denotes a family of reuse mechanisms in which previously produced tokens, token representations, contexts, or utility tokens are retained and redeployed rather than discarded, replayed, or reissued from scratch. In recent arXiv literature, the term appears in at least four technically distinct settings: long-context ICL for QA, speculative decoding for LLM inference, ViT-based active visual exploration, and long-horizon conversational memory; related protocol and token-economic literatures use the same phrase for token conversion, exchange, or burn-and-remint cycles (Cattan et al., 2024, Luo et al., 2024, Olszewski et al., 2023, Thomas, 1 May 2026, Teutsch et al., 2019).

1. Conceptual scope and recurring structure

The term is not monosemous. In some works, a “token” is a transformer symbol or patch embedding; in others, it is a prompt-budget unit; in others, it is a cryptographic or governance asset. Taken together, these works suggest a common operational pattern: useful artifacts generated in an earlier step are preserved in a compact state and later reused under a resource constraint, typically compute, latency, context-window size, or market liquidity.

Domain	Recycled object	Representative paper
Long-context QA	Demonstrations generated from the same context	"Can Few-shot Work in Long-Context? Recycling the Context to Generate Demonstrations" (Cattan et al., 2024)
LLM inference	Discarded top-$k$ candidate tokens	"Turning Trash into Treasure: Accelerating Inference of LLMs with Token Recycling" (Luo et al., 2024)
ViT-based AVE	Midway patch and $[CLS]$ tokens	"TORE: Token Recycling in Vision Transformers for Efficient Active Visual Exploration" (Olszewski et al., 2023)
Long-horizon dialogue	Retrieved branch content and compacted history	"Context Recycling for Long-Horizon LLM Inference" (Thomas, 1 May 2026)
Blockchain computation	TRU recycled into CPU; DAO dissolved into utility tokens	"Bootstrapping a stable computation token" (Teutsch et al., 2019)
Federated identity	Identity-share token exchanged for a fresh access token	"Trust-Based Identity Sharing For Token Grants" (Dodanduwa et al., 2018)

A recurring structural decomposition is visible across the computational variants. A large or expensive substrate is processed once or incrementally; a smaller reusable state is cached; a later stage consumes the cache instead of repeating the full upstream computation. In prompt-centric variants, the reusable state is not an internal hidden state but a prompt artifact, such as demonstrations or retrieved branch content. In protocol-centric variants, recycling is literal token conversion or exchange.

2. Recycling context in long-context in-context learning

In "Can Few-shot Work in Long-Context? Recycling the Context to Generate Demonstrations" (Cattan et al., 2024), token recycling is defined for long-context QA as reusing the same long input context $D$ both to generate few-shot demonstrations and to answer the target query. Given a long input context of $1\text{–}3\text{k}$ tokens and a query, the method generates additional query-output pairs from the given context as few-shot examples, while introducing the context only once. The demonstrations therefore add only a question-answer pair and a paragraph identifier, instead of repeating the entire context for every shot.

The prompt construction is explicitly budgeted. The final prompt is assembled exactly once as system or instruction tokens, the full context $D$, $k$ recycled demos, a final instruction to “Identify the supporting paragraph(s), then answer.”, and the user query $q$. The token accounting is given as

$$T_{\mathrm{prompt}} = T_D + \sum_{i=1}^k T_{\mathrm{demo},i} + T_{\mathrm{query}} + T_{\mathrm{instr}},$$

with $T_D = |D|$ and $T_{\mathrm{demo},i} = |q_i| + |a_i| + |[id_i]|$, which is typically much smaller than $[CLS]$0. The contrast case is standard ICL, where the context would be replicated across shots.

A second defining component is explicit paragraph identification. In both demo generation and final answering, the model is prompted to emit an “Evidence: [$[CLS]$1para-ID$[CLS]$2]” line before the answer. The paper states that this “flattens the classic U-shaped performance curve” and provides fine-grained attribution. Quantitatively, the method reports “+16 absolute points on average across models” on long-context QA, and on Lost-in-the-Middle with a $[CLS]$3-token context it moves Gemini Pro from approximately $[CLS]$4 to $[CLS]$5, with accuracy staying around $[CLS]$6 regardless of whether the answer lies in early, middle, or late paragraphs. The same setup also generalizes from single-hop demonstrations to multi-hop QA, reaching token-F1 $[CLS]$7 on HotpotQA for Gemini Ultra versus a $[CLS]$8 baseline, and F1 $[CLS]$9 on MuSiQue for Gemini Pro versus $D$0 baseline.

The limitations are equally explicit. Demo generation adds an extra inference pass; all experiments are in English and on $D$1-token contexts; random paragraph sampling may be suboptimal; and $D$2 yields diminishing returns in the reported setup. A plausible implication is that context recycling is most effective when the shared document is large enough that repeated inclusion dominates prompt cost, but still structured enough that local evidence pointers remain stable.

3. Recycling discarded candidate tokens in speculative decoding

In "Turning Trash into Treasure: Accelerating Inference of LLMs with Token Recycling" (Luo et al., 2024), token recycling is a train-free speculative-decoding method that stores candidate tokens that would otherwise be discarded during autoregressive decoding. The key observation is that at each step the model computes a top-$D$3 candidate set $D$4, but only the top-1 token is emitted. Token Recycling retains all top-$D$5 candidates in an adjacency matrix and later reuses them as speculative drafts.

The central data structure is

$D$6

where row $D$7 records the current top-$D$8 candidate tokens following vocabulary item $D$9:

$1\text{–}3\text{k}$0

Cold start initializes all entries to zero, while hot start copies the final matrix from the previous query. After verification, the matrix is updated by overwriting each row corresponding to a draft token:

$1\text{–}3\text{k}$1

Draft construction is breadth-first. Starting from the last confirmed token $1\text{–}3\text{k}$2, the algorithm grows a draft tree of depth $1\text{–}3\text{k}$3, with the supplied summary using $1\text{–}3\text{k}$4 and total node count $1\text{–}3\text{k}$5. Candidate lists are gathered in parallel, a static and imbalanced tree template allocates more children to high-confidence parents, and the merged sequence is verified in one pass using tree attention. The tree-attention mask restricts each draft token to attend only to the prefix on its own branch:

$1\text{–}3\text{k}$6

with

$1\text{–}3\text{k}$7

The storage overhead is deliberately small. For $1\text{–}3\text{k}$8 and $1\text{–}3\text{k}$9, storing $D$0-bit integers requires approximately $D$1 MB, i.e. “$D$2MB of additional storage.” Reported speedups are approximately $D$3 across all sizes of LLMs: Vicuna-7B $D$4, Vicuna-13B $D$5, Vicuna-33B $D$6, and Code Llama-7B on MBPP $D$7. The method is described as significantly outperforming existing train-free methods by $D$8 and a widely recognized training method by $D$9. Ablations identify an optimal design around $k$0 nodes and depth $k$1, and shuffling query order reduces speed by less than $k$2, which suggests that the cached token graph is not merely exploiting short-range prompt continuity.

4. Recycling intermediate tokens in Vision Transformers

In "TORE: Token Recycling in Vision Transformers for Efficient Active Visual Exploration" (Olszewski et al., 2023), token recycling addresses repeated processing of previously seen observations in AVE. The paper splits a ViT encoder of depth $k$3 at layer $k$4 into two modules: an iterator $k$5 and an aggregator $k$6. The iterator processes each glimpse $k$7 separately and emits a $k$8 token $k$9 and patch-token set $q$0; the aggregator ingests the union of all cached midway tokens and produces the current prediction.

The cache is updated incrementally:

$q$1

$q$2

This allows the heavy front portion of the model to run only on the new glimpse, while the downstream stage operates on accumulated cached tokens. The cost comparison is stated explicitly. A full $q$3-block ViT over $q$4 tokens has cost $q$5. TORE yields

$q$6

so the quadratic term depends on the fixed glimpse size $q$7, not on the growing total token count $q$8.

The paper additionally reduces the decoder to one transformer block of dimension $q$9, in contrast to an $$T_{\mathrm{prompt}} = T_D + \sum_{i=1}^k T_{\mathrm{demo},i} + T_{\mathrm{query}} + T_{\mathrm{instr}},$$0-block MAE decoder. The supplied summary states that this lowers decoder FLOPs by $$T_{\mathrm{prompt}} = T_D + \sum_{i=1}^k T_{\mathrm{demo},i} + T_{\mathrm{query}} + T_{\mathrm{instr}},$$1 and does not harm accuracy. Quantitatively, TORE reports up to $$T_{\mathrm{prompt}} = T_D + \sum_{i=1}^k T_{\mathrm{demo},i} + T_{\mathrm{query}} + T_{\mathrm{instr}},$$2 reduction in computational overhead, while improving downstream accuracy in several settings: on SUN360, AME (ViT-L) at $$T_{\mathrm{prompt}} = T_D + \sum_{i=1}^k T_{\mathrm{demo},i} + T_{\mathrm{query}} + T_{\mathrm{instr}},$$3 GFLOPs and $$T_{\mathrm{prompt}} = T_D + \sum_{i=1}^k T_{\mathrm{demo},i} + T_{\mathrm{query}} + T_{\mathrm{instr}},$$4 accuracy becomes TORE + AME at $$T_{\mathrm{prompt}} = T_D + \sum_{i=1}^k T_{\mathrm{demo},i} + T_{\mathrm{query}} + T_{\mathrm{instr}},$$5 GFLOPs and $$T_{\mathrm{prompt}} = T_D + \sum_{i=1}^k T_{\mathrm{demo},i} + T_{\mathrm{query}} + T_{\mathrm{instr}},$$6; on CIFAR100, AME (ViT-B) at $$T_{\mathrm{prompt}} = T_D + \sum_{i=1}^k T_{\mathrm{demo},i} + T_{\mathrm{query}} + T_{\mathrm{instr}},$$7 GFLOPs and $$T_{\mathrm{prompt}} = T_D + \sum_{i=1}^k T_{\mathrm{demo},i} + T_{\mathrm{query}} + T_{\mathrm{instr}},$$8 becomes $$T_{\mathrm{prompt}} = T_D + \sum_{i=1}^k T_{\mathrm{demo},i} + T_{\mathrm{query}} + T_{\mathrm{instr}},$$9 GFLOPs and $T_D = |D|$0; on sequential classification, DeiT at $T_D = |D|$1 GFLOPs and $T_D = |D|$2 on ImageNet-1k becomes TORE + DeiT at $T_D = |D|$3 GFLOPs and $T_D = |D|$4. Figure 1 further reports that for $T_D = |D|$5 up to $T_D = |D|$6, TORE saves $T_D = |D|$7 of FLOPs with less than $T_D = |D|$8 percentage point drop in accuracy.

The stated limitations are architectural. The image is divided into fixed patches or glimpses; the aggregator still grows linearly in the number of seen tokens; and a single split point $T_D = |D|$9 must be chosen at inference. This suggests that token recycling in ViTs is most natural when partial observations arrive sequentially and when early-layer tokenization is cheap enough to isolate from deeper global aggregation.

5. Recycling context across long-horizon conversations

"Context Recycling for Long-Horizon LLM Inference" (Thomas, 1 May 2026) generalizes the idea from single-prompt QA to multi-turn dialogue. The paper introduces ContextForge, which maintains task-relevant information across turns by combining structured query generation, external memory retrieval, and controlled synthesis. The explicit problem is that naive replay of the full history incurs quadratic token growth across turns:

$T_{\mathrm{demo},i} = |q_i| + |a_i| + |[id_i]|$0

ContextForge instead enforces a bounded per-turn prompt budget $T_{\mathrm{demo},i} = |q_i| + |a_i| + |[id_i]|$1, giving

$T_{\mathrm{demo},i} = |q_i| + |a_i| + |[id_i]|$2

with $T_{\mathrm{demo},i} = |q_i| + |a_i| + |[id_i]|$3 tokens in the supplied summary.

The implementation is hierarchical. Structured query generation extracts a small query $T_{\mathrm{demo},i} = |q_i| + |a_i| + |[id_i]|$4 from the incoming user message rather than replaying older turns verbatim. Retrieval uses an in-RAM FTS5/BM25 index over summaries and a disk-backed SQLite store of full node content. The highest-scoring branch is loaded into an L1 KV-cache containing approximately $T_{\mathrm{demo},i} = |q_i| + |a_i| + |[id_i]|$5K tokens. Context assembly is then

$T_{\mathrm{demo},i} = |q_i| + |a_i| + |[id_i]|$6

$T_{\mathrm{demo},i} = |q_i| + |a_i| + |[id_i]|$7

$T_{\mathrm{demo},i} = |q_i| + |a_i| + |[id_i]|$8

$T_{\mathrm{demo},i} = |q_i| + |a_i| + |[id_i]|$9

Immediately after generation, Layer 1 tokens are evicted from the prompt buffer and history is compacted when it grows beyond $[CLS]$00K tokens.

The empirical evaluation uses a $[CLS]$01-turn conversational benchmark over structured healthcare queries with multi-turn reasoning, back-references, and domain shifts. On the $[CLS]$02-turn benchmark, ContextForge scores $[CLS]$03 points with $[CLS]$04 s/turn average and $[CLS]$05K total tokens, versus $[CLS]$06 points, $[CLS]$07 s/turn, and $[CLS]$08K tokens for the baseline, corresponding to a $[CLS]$09 token reduction and $[CLS]$10 speedup. On the $[CLS]$11-turn benchmark, ContextForge achieves $[CLS]$12 points ($[CLS]$13), $[CLS]$14 s, and $[CLS]$15 total tokens, versus $[CLS]$16 points ($[CLS]$17), $[CLS]$18 s, and $[CLS]$19 total tokens for the baseline, a $[CLS]$20 token reduction and $[CLS]$21 speedup. By turn $[CLS]$22, the baseline’s history reaches approximately $[CLS]$23K tokens and must be resent every turn, whereas ContextForge keeps the active budget fixed.

A plausible implication is that context recycling in multi-turn systems is not merely prompt compression. It is a scheduling policy over stable state, transient retrieved state, and evicted state, with retrieval and compaction determining which prior computation is reintroduced into the active context window.

6. Protocol, identity, and token-economic meanings

Outside transformer inference, token recycling often denotes literal reuse, exchange, or conversion of issued tokens. In "Bootstrapping a stable computation token" (Teutsch et al., 2019), the Truebit model defines a closed-loop system in which TRU is minted as solver and verifier reward, CPU is burned by task givers, and TRU can be recycled back into CPU through a tasking-conversion contract:

$[CLS]$24

where $[CLS]$25 is the on-chain median local price in TRU per CPU step. The same paper also defines a one-time dissolution of DAO into TRU or CPU via

$[CLS]$26

Here, recycling is not a computational cache but a monetary circulation mechanism under endogenous price formation.

A related identity-management usage appears in "Trust-Based Identity Sharing For Token Grants" (Dodanduwa et al., 2018). The paper extends OpenID Connect so that a client can obtain an identity_share_token from IdP$[CLS]$27 and exchange it at IdP$[CLS]$28 using grant_type=identity_share_token. The exchanged token is a signed JWT with claims such as iss, aud, iat, exp, and sdata. The lifetime relation is explicitly bounded by

$[CLS]$29

so the fresh access token issued by IdP$[CLS]$30 does not outlive the source identity-share token. In this usage, token recycling means cross-domain grant reuse under a trust relation, not efficiency optimization.

An adjacent but distinct literature uses tokens as incentives for real-world recycling. "A Simulation-Based Conceptual Model for Tokenized Recycling: Integrating Blockchain, Market Dynamics, and Behavioral Economics" (Mustafa, 26 Jul 2025) defines a RecycleToken minted by logRecycle, burned by redeem, and valued by

$[CLS]$31

The study remains “theoretical and simulation-based,” with Monte Carlo evaluation over $[CLS]$32 trials, $[CLS]$33 months, and $[CLS]$34 month. This is not token recycling in the sense of reusing transformer tokens, but it shows that the same lexical pattern can denote incentive circulation in cyber-physical systems.

7. Terminological boundaries, discrepancies, and common misconceptions

The literature does not use the term uniformly, and one documented discrepancy is especially notable. The abstract of "ARNet: Self-Supervised FG-SBIR with Unified Sample Feature Alignment and Multi-Scale Token Recycling" (Jiang et al., 2024) states that the method presents a self-supervised Multi-Scale Token Recycling (MSTR) Module “featured by recycling discarded patch tokens in multi-scale features.” However, the supplied manuscript content states that there is no MSTR description and instead describes a Token Representation Sharing Module (TRSM) based on self-attention and cross-attention. In that supplied content, TRSM is inserted after frozen dual weight-sharing encoders, uses one FA-Attention block with $[CLS]$35 heads, operates on the full sequence of patch tokens excluding the $[CLS]$36 token, and yields gains such as QMUL-Chair V2 Acc.@1 from $[CLS]$37 to $[CLS]$38, QMUL-Shoe V2 Acc.@5 from $[CLS]$39 to $[CLS]$40, and Clothes-V1 Acc.@1 from $[CLS]$41 to $[CLS]$42.

This discrepancy matters because it distinguishes three nearby but non-identical operations: recycling discarded tokens, sharing token representations, and multi-scale feature fusion. A common misconception is to treat any token-level reuse mechanism as belonging to a single method family. The papers surveyed here indicate otherwise. In long-context QA, the recycled object is a demonstration generated from the same document; in speculative decoding, it is a discarded candidate token stored in an adjacency matrix; in TORE, it is a midway patch token cached across glimpses; in ContextForge, it is retrieved branch content that is loaded and then evicted; in Truebit and identity-sharing systems, it is a literal token grant or utility token.

Taken together, these works suggest that “token recycling” is best understood as a higher-level design pattern rather than a single algorithm. The unifying principle is selective retention of prior artifacts under a constrained budget. The technical differences arise from what counts as a token, where the retained state is stored, how it is revalidated, and whether recycling preserves semantics exactly, as in lossless speculative decoding, or preserves utility operationally, as in prompt construction, retrieval, or token conversion.