Accordion-Thinking: Adaptive LLM Reasoning

• Accordion-Thinking is a dynamic framework that compresses and expands chain-of-thought reasoning to balance computational efficiency with accuracy.
• It integrates ThinkSwitcher mode selection and self-regulated summarization to adapt reasoning depth based on query complexity.
• Empirical evaluations show up to 3.8× throughput improvements with minimal accuracy loss, proving its scalability for multi-step inference.

Accordion-Thinking is a class of frameworks for LLMs and large reasoning models (LRMs) that regulate the depth, granularity, and context-length of stepwise reasoning dynamically during inference. These frameworks exploit the model’s capacity to compress or expand its explicit chain-of-thought (CoT) reasoning—analogous to folding and unfolding an accordion—so as to balance computational efficiency and answer quality based on problem complexity. Two primary instantiations are the ThinkSwitcher approach, relying on external switching between concise and detailed CoTs per query, and the Accordion-Thinking framework, realizing end-to-end self-regulation of context complexity via learned stepwise summarization and context pruning (Liang et al., 20 May 2025, Yang et al., 3 Feb 2026).

1. Conceptual Foundations and Definitions

Accordion-Thinking seeks to address the limitations of conventional CoT prompting where all tasks, regardless of complexity, induce lengthy multi-step traces with linear or worse context growth. This incurs prohibitive memory and compute scaling due to attention and key-value (KV) cache growth—quadratic in context length for Transformers (Yang et al., 3 Feb 2026). The metaphor of the accordion reflects the core mechanism: compressing the reasoning chain for simple instances to save compute and memory (fold), and expanding reasoning for harder examples (unfold) when additional intermediate inferences are necessary to preserve solution accuracy (Liang et al., 20 May 2025).

Two main operationalizations exist:

• Switch-based Accordion-Thinking: A lightweight module dynamically chooses between short and long CoTs before generating the answer, with the backbone LRM executing accordingly.
• Self-Regulated Summarization (Accordion-Thinking proper): The LLM iteratively generates stepwise derivations and immediate one-step summaries, discarding obsolete derivation tokens and retaining only summaries in context—allowing the model to compress its own "memory" mid-inference.

2. ThinkSwitcher: Query-Driven Mode Selection

The ThinkSwitcher paradigm demonstrates accordion reasoning by equipping any prompt-controllable LRM with a switching module $f_\phi$—a multi-layer perceptron (MLP) that, given the query embedding $x_q$, predicts the empirical pass-rates under short (SC) and long (LC) CoT templates:

$$\left[\hat{y}_{\mathrm{SC}}, \;\hat{y}_{\mathrm{LC}}\right] = f_\phi(x_q)$$

where $\hat{y}_{\mathrm{SC}}$ and $\hat{y}_{\mathrm{LC}}$ estimate the likelihoods of a correct answer from short and long CoT, respectively. The margin $$\Delta\hat{y} = \hat{y}_{\mathrm{LC}} - \hat{y}_{\mathrm{SC}}$$ is compared to a threshold $\tau$ to select the reasoning mode per query:

$$m(q) = \begin{cases} \text{LC} & \Delta\hat{y} \geq \tau \ \text{SC} & \text{otherwise} \end{cases}$$

Prompt engineering enables reliable induction of concise (short) or elaborate (long) CoT outputs by varying the content of special tags (e.g., using an empty `` block for short CoT).

Supervision is constructed directly from LRM accuracy statistics on sampled outputs per mode, and the module is trained using combined mean-square error and margin consistency losses:

$$\mathcal{L}_{\mathrm{switch}} = \mathcal{L}_\mathrm{MSE} + \lambda_{\mathrm{margin}}\,\mathcal{L}_{\mathrm{margin}}$$

where margin loss aligns $\Delta\hat{y}$ with the empirical accuracy gap per query.

3. Accordion-Thinking: Self-Regulated Step Summarization

Accordion-Thinking as introduced in (Yang et al., 3 Feb 2026) generalizes beyond preset CoT templates, enabling any LLM to periodically summarize and compress its contextual reasoning during multi-step inference. The process proceeds as follows:

• Chain structuring: Each reasoning chain is decomposed into $x_q$0 steps, each a tuple $x_q$1, where $x_q$2 is the detailed derivation for step $x_q$3 and $x_q$4 is a concise, lossless summary.
• Generation/summarization cycle:

1. Generate detailed $x_q$5 up to a termination signal (e.g., </step>).
2. Immediately output $x_q$6 conditioned on all prior step summaries and the current $x_q$7.
3. Discard $x_q$8 from memory—only $x_q$9 remain for subsequent steps.

This cyclic folding ensures that at each stage, the model works only with a "compressed" context, keeping the cumulative memory/token length nearly linear rather than quadratic in total derivation length.

Inference supports two modes:

• Unfold: full historical context (all $$\left[\hat{y}_{\mathrm{SC}}, \;\hat{y}_{\mathrm{LC}}\right] = f_\phi(x_q)$$0, $$\left[\hat{y}_{\mathrm{SC}}, \;\hat{y}_{\mathrm{LC}}\right] = f_\phi(x_q)$$1) retained.
• Fold: all $$\left[\hat{y}_{\mathrm{SC}}, \;\hat{y}_{\mathrm{LC}}\right] = f_\phi(x_q)$$2 discarded after corresponding $$\left[\hat{y}_{\mathrm{SC}}, \;\hat{y}_{\mathrm{LC}}\right] = f_\phi(x_q)$$3 generation; only concise summaries maintained.

Reinforcement learning is applied using a variant of Group Relative Policy Optimization (GRPO), with a key reward: solutions in Fold mode earn zero reward if any summary is insufficient and subsequent reasoning fails. Mixed batches interleave Fold and Unfold, allowing the model to align compression skills with high-level task accuracy.

4. Empirical Evaluation: Efficiency, Accuracy, and Compression

Experimental studies evaluated Accordion-Thinking frameworks across a battery of math benchmarks (e.g., GSM8K, MATH500) and backbone sizes (Liang et al., 20 May 2025, Yang et al., 3 Feb 2026). Principal findings include:

Method	Avg. Accuracy	Avg. Tokens (7B)	Relative Throughput vs. Unfold
LC-Only (Unfold)	64.6%	7,617	1.0×
SC-Only	46.0%	1,079	6–7×
ThinkSwitcher	62.8%	5,405	≈1.4×
Accordion Fold	≈Unfold	≈2–3× less	2.7–3.8×

Key points:

• ThinkSwitcher cuts computational tokens by up to 29% relative to always using LC, with only ~1.8% drop in accuracy on difficult benchmarks (Liang et al., 20 May 2025).
• Accordion-Thinking (Fold mode) attains nearly identical accuracy to Unfold mode after sufficient RL-driven training; the initial compression-induced gap vanishes with continued training (Yang et al., 3 Feb 2026).
• Measured on a 48GB GPU, Fold mode achieves ~3–3.8× throughput improvements due to reduced attention and KV cache overhead.
• Summaries generated in Fold mode have high human faithfulness, forming readable, logical proof traces independent of suppressed detailed derivations.

5. Analysis: Mechanisms, Training, and Failure Modes

The efficacy of Accordion-Thinking depends on several core design and training strategies:

• Strict supervision: Stringent summarization prompts and rule-based filtering (e.g., ensuring summaries are >=100 tokens and well-tagged) are crucial; "lax" prompts yield catastrophic information loss (Fold accuracy < 60%), whereas strict supervision reaches >80% (Yang et al., 3 Feb 2026).
• Reinforcement learning: Fold mode RL with explicit penalties for lossy summaries strongly encourages the model to encapsulate all essential interim state in summaries.
• Mixed-mode training: Alternating between Fold and Unfold rollouts prevents degradation of raw reasoning ability while honing compression.
• Ablations: The margin loss in ThinkSwitcher is essential for crisp mode selection; ablation increases decision noise and reduces normalized AUC-AC (Liang et al., 20 May 2025).

Potential limitations include dependency on prompt strictness during synthesis, robustness to adversarial inputs (e.g., ambiguous step boundaries), and possible information bottlenecks if summary formats are too terse.

6. Interpretations and Broader Implications

Accordion-Thinking delivers nearly lossless context compression for multi-step reasoning tasks, allowing LLMs and LRMs to scale to longer, more complex problems without proportionally increasing resource demands. The approach also yields human-understandable reasoning traces, as summaries are both concise and sufficient for full solution reproduction. A plausible implication is that such learned compression mechanisms will be instrumental in aligning future models’ efficiency, transparency, and manipulability. In both ThinkSwitcher-style external mode selection and the direct self-regulating summarization of Accordion-Thinking, the field now possesses robust techniques for computationally adaptive reasoning chains in LLMs (Liang et al., 20 May 2025, Yang et al., 3 Feb 2026).