Slim-SC: Efficient Thought Pruning

• The paper introduces a novel thought pruning method that eliminates redundant reasoning chains during generation, reducing inference latency without sacrificing accuracy.
• The method employs cosine similarity of embedded intermediate thoughts with high thresholds (around 0.98) to identify and prune semantically redundant chains.
• Experimental results on STEM benchmarks demonstrate up to a 45% reduction in inference latency and a 26% reduction in KV-cache usage, maintaining competitive accuracy.

Searching arXiv for the specified Slim-SC paper and closely related self-consistency work. Slim-SC denotes a test-time scaling method for LLM reasoning that modifies Self-Consistency by pruning redundant reasoning chains during generation rather than after all chains have finished. In the formulation presented in "Slim-SC: Thought Pruning for Efficient Scaling with Self-Consistency" (Hong et al., 17 Sep 2025), the method targets a specific inefficiency of standard Self-Consistency: multiple sampled chains often converge to semantically similar intermediate thoughts and ultimately to the same answer, yet conventional decoding still pays the full latency and KV-cache cost of completing all of them. Slim-SC therefore treats inter-chain redundancy as a pruning signal at the thought level, with the aim of preserving the accuracy benefits of Self-Consistency while reducing inference cost.

1. Conceptual setting within test-time scaling

Self-Consistency is defined as sampling $N$ independent reasoning chains

$$\mathcal{R}_N=\{c_i\}_{i=1}^{N}\stackrel{\text{i.i.d.}}{\sim}{P_\theta(c\mid x)},$$

mapping each chain to an answer

$\hat{y}_i=g(c_i),$

and selecting the final output by plurality vote,

$$\tilde{y}=\operatorname*{arg\,max}_{y}\sum_{i=1}^{N}\mathbf 1\!\bigl[\hat{y}_i=y\bigr].$$

Slim-SC preserves this overall Self-Consistency framework but intervenes before all chains terminate. Its central claim is that many chains become redundant early enough that continuing all of them is unnecessary (Hong et al., 17 Sep 2025).

The method belongs to test-time scaling rather than retraining-based alignment or architecture modification. Its scope is therefore operational: it attempts to improve the accuracy–latency trade-off of parallel chain sampling without changing the underlying LLM weights. The paper positions it against other Self-Consistency accelerations that rely on model-based confidence scores or heuristics, and frames its contribution as both theoretical and empirical: it first analyzes why standard Self-Consistency is inefficient, then proposes a pruning rule derived from those observations.

A common misconception is that Slim-SC is simply early stopping for Self-Consistency. The distinction is substantive. Early-stopping methods attempt to terminate the whole process once enough evidence accumulates for an answer, whereas Slim-SC keeps the multi-chain setting but reduces its active population over time by removing chains judged redundant at the thought level. The final prediction still comes from majority voting over the remaining completed chains.

2. Inefficiencies of standard Self-Consistency

The paper identifies several inefficiencies in standard Self-Consistency. The first is high compute cost: generating many full reasoning chains scales inference expense substantially. The second is what the authors call “wait-for-all” latency: voting cannot occur until all sampled chains have completed, even if the decisive answer appears much earlier in one or several shorter chains. The third is diminishing returns: increasing the number of chains improves accuracy only up to a point, while cost grows nearly linearly. The fourth is redundancy: many chains are semantically similar and add little information. The fifth is vote dilution: incorrect chains can dominate the plurality vote even when the correct answer is already present among the candidates (Hong et al., 17 Sep 2025).

The paper makes the vote-dilution problem explicit through a concrete example. On GPQA Diamond with R1-Distill and $N=64$, the correct answer is present in the candidate set in $91.9\%$ of cases, but the final voted answer is correct in only $63.6\%$. This separates “ideal accuracy,” defined by whether the correct answer appears anywhere in the sample set, from “actual accuracy,” defined by whether majority voting selects it. The gap indicates that more chains do not necessarily improve the final decision if they are concentrated in semantically repetitive but incorrect regions.

Latency inefficiency is tied to chain length. For R1-Distill on AIME, the paper reports that incorrect chains are typically longer than correct chains; in some cases an incorrect chain is six times longer than the shortest correct one. Standard Self-Consistency nevertheless waits for the long chain. This observation motivates the idea that pruning should happen during generation rather than only after full completion.

The paper also reports a semantic clustering effect. Using pairwise thought-level similarity, it finds high intra-class similarity for correct–correct and incorrect–incorrect chain pairs, and relatively rare correct–incorrect similarity. This suggests that similarity is not merely a surface property of language generation but a useful proxy for convergence in answer space. The method’s design depends on this empirical regularity.

3. Step-wise thought pruning mechanism

Slim-SC prunes chains step by step during generation when intermediate thoughts from different active chains become highly similar (Hong et al., 17 Sep 2025). At each analysis interval, the method extracts newly generated thoughts from each active chain, embeds them as vectors $e_i$, and compares them to thoughts from other active chains using cosine similarity. If two thought embeddings satisfy

$\text{sim}(e_i,e_j) > \tau_{sim},$

the corresponding chains are treated as redundant, and one of them is pruned.

The threshold is intentionally high. The paper states that $\tau_{sim} > 0.9$, with empirical settings typically in the range $$\mathcal{R}_N=\{c_i\}_{i=1}^{N}\stackrel{\text{i.i.d.}}{\sim}{P_\theta(c\mid x)},$$0–$$\mathcal{R}_N=\{c_i\}_{i=1}^{N}\stackrel{\text{i.i.d.}}{\sim}{P_\theta(c\mid x)},$$1, and often a practical sweet spot at $$\mathcal{R}_N=\{c_i\}_{i=1}^{N}\stackrel{\text{i.i.d.}}{\sim}{P_\theta(c\mid x)},$$2. This design reflects a conservative pruning policy: only very similar chains should be removed, so that diversity is reduced minimally while redundancy is reduced aggressively.

The method also introduces a pruning delay. Pruning is considered only after

$$\mathcal{R}_N=\{c_i\}_{i=1}^{N}\stackrel{\text{i.i.d.}}{\sim}{P_\theta(c\mid x)},$$3

The rationale is that early reasoning often contains generic boilerplate such as restating the question, defining variables, or outlining a plan. If pruning were enabled immediately, chains might be removed before meaningful divergence appears. Experimentally, a 20-step delay is often best, and each step corresponds to about 3 seconds in the implementation described in the paper.

Algorithmically, Slim-SC initializes $$\mathcal{R}_N=\{c_i\}_{i=1}^{N}\stackrel{\text{i.i.d.}}{\sim}{P_\theta(c\mid x)},$$4 chains, maintains a thought embedding index using FAISS, and repeatedly collects new thoughts and checks for nearest neighbors from other active chains once the pruning delay has passed. Pruning stops when only one chain remains active or when all active chains have finished. Majority voting is then applied to the collected answers from the surviving completed chains.

4. Pruning policies, embeddings, and systems considerations

Once a similar chain pair has been identified, Slim-SC uses one of two pruning-selection heuristics. Random Pruning prunes one of the two chains uniformly at random. Diversity-based Pruning keeps the chain with greater internal diversity and removes the one with higher internal self-similarity. In the paper’s interpretation, a chain with higher internal self-similarity is more repetitive and therefore more prune-worthy (Hong et al., 17 Sep 2025).

The reported findings do not treat these heuristics as interchangeable in all respects. Random Pruning is described as simpler and very robust. Diversity-based Pruning can be slightly more token-efficient. This is important because the method’s gains are not attributed merely to reducing the number of active chains; rather, the paper argues that benefits depend on removing redundancy while preserving useful diversity. Supporting this claim, naive baselines behave poorly: Global Random Pruning hurts accuracy, and Least Similar Pruning is especially bad because it destroys useful diversity.

Thought embeddings are produced with all-mpnet-base-v2, and nearest-neighbor search is handled with FAISS. The paper emphasizes that the overhead of this auxiliary machinery is small: the embedding model uses less than $$\mathcal{R}_N=\{c_i\}_{i=1}^{N}\stackrel{\text{i.i.d.}}{\sim}{P_\theta(c\mid x)},$$5 GPU memory, and the added latency is estimated at about $$\mathcal{R}_N=\{c_i\}_{i=1}^{N}\stackrel{\text{i.i.d.}}{\sim}{P_\theta(c\mid x)},$$6. Within the proposed system model, this makes the pruning computation negligible relative to LLM decoding cost.

A further systems point concerns KV-cache measurement. The paper distinguishes mean KV-cache usage, mean peak KV-cache usage, and end-to-end duration. Slim-SC may still exhibit a fairly high initial peak because all chains are launched in parallel, but cache usage drops quickly after pruning. This is contrasted with standard Self-Consistency, which sustains high cache usage longer, and with ESC, which may have lower peak cache but longer total GPU occupation. A plausible implication is that GPU-time cost, not only instantaneous memory peak, is the relevant deployment metric for parallel reasoning workloads.

5. Experimental profile and quantitative results

The evaluation uses three STEM reasoning benchmarks—GPQA Diamond, AIME-2024, and AQuA-RAT—and two recent LLM architectures: DeepSeek-R1-Distill-Qwen-14B and Qwen-QwQ-32B. The baselines are CoT, standard Self-Consistency, ESC, and CCoT-SC. The main metrics are Accuracy (%), Latency (s), Average completion tokens, and Mean KVC usage (%), with Mean Peak KV Cache usage reported in the appendix (Hong et al., 17 Sep 2025).

The baseline Self-Consistency settings are tuned rather than fixed arbitrarily. For R1-Distill, the optimal $$\mathcal{R}_N=\{c_i\}_{i=1}^{N}\stackrel{\text{i.i.d.}}{\sim}{P_\theta(c\mid x)},$$7 is reported as $$\mathcal{R}_N=\{c_i\}_{i=1}^{N}\stackrel{\text{i.i.d.}}{\sim}{P_\theta(c\mid x)},$$8 on GPQA, $$\mathcal{R}_N=\{c_i\}_{i=1}^{N}\stackrel{\text{i.i.d.}}{\sim}{P_\theta(c\mid x)},$$9 on AIME, and $\hat{y}_i=g(c_i),$0 on AQuA. For QwQ-32B, the optimal $\hat{y}_i=g(c_i),$1 is $\hat{y}_i=g(c_i),$2 on GPQA, $\hat{y}_i=g(c_i),$3 on AIME, and $\hat{y}_i=g(c_i),$4 on AQuA. Slim-SC thresholds are then tuned around high-similarity values; Random Pruning usually uses $\hat{y}_i=g(c_i),$5, whereas Diversity-based Pruning uses $\hat{y}_i=g(c_i),$6–$\hat{y}_i=g(c_i),$7 depending on model and dataset.

The core quantitative results are summarized below.

Setting	Reported result	Interpretation
R1-Distill, GPQA, DP vs SC latency	45% reduction	strongest reported latency gain
R1-Distill, AIME, DP vs SC latency	43% reduction	large gain on hard math
R1-Distill, AQuA, DP vs SC latency	11% reduction	smaller gain on easier task
QwQ-32B, GPQA, DP vs SC latency	20% reduction	moderate gain
QwQ-32B, AIME, DP vs SC latency	8% reduction	modest gain
QwQ-32B, AQuA, DP vs SC latency	9% reduction	modest gain
Overall token reduction	up to 34%	lower decoding volume
Overall mean KVC reduction	up to 26%	lower average cache usage

The paper’s abstract and conclusion condense these findings to “up to 45%” inference-latency reduction and “up to 26%” KVC-usage reduction while maintaining or improving accuracy. A concrete example is given for AIME with R1-Distill: token usage drops by $\hat{y}_i=g(c_i),$8 relative to standard Self-Consistency, and mean KV cache drops from $\hat{y}_i=g(c_i),$9 to $$\tilde{y}=\operatorname*{arg\,max}_{y}\sum_{i=1}^{N}\mathbf 1\!\bigl[\hat{y}_i=y\bigr].$$0.

Accuracy is reported as generally matching or exceeding standard Self-Consistency and outperforming ESC and CCoT-SC in most settings. For R1-Distill on AIME, Slim-SC with Random Pruning reaches $$\tilde{y}=\operatorname*{arg\,max}_{y}\sum_{i=1}^{N}\mathbf 1\!\bigl[\hat{y}_i=y\bigr].$$1, which the discussion states matches Self-Consistency. On AQuA, Slim-SC variants are around $$\tilde{y}=\operatorname*{arg\,max}_{y}\sum_{i=1}^{N}\mathbf 1\!\bigl[\hat{y}_i=y\bigr].$$2–$$\tilde{y}=\operatorname*{arg\,max}_{y}\sum_{i=1}^{N}\mathbf 1\!\bigl[\hat{y}_i=y\bigr].$$3, slightly above or on par with Self-Consistency. The GPQA presentation in the supplied text notes that the table formatting is noisy, but describes Slim-SC as slightly improved or very competitive relative to Self-Consistency.

6. Interpretation, limitations, and relation to broader reasoning methods

The significance of Slim-SC lies in its reframing of Self-Consistency inefficiency. Rather than treating the sampled chain set as fixed until voting, the method treats chains as partially exchangeable computational objects that can be removed once they become sufficiently similar. The paper argues that this is justified because correct chains cluster with correct chains, incorrect chains cluster with incorrect chains, and similar chains usually lead to the same final answer (Hong et al., 17 Sep 2025).

This does not imply that any semantic similarity metric is sufficient. The authors explicitly note a metric-design limitation: Slim-SC uses similarity patterns effectively, but a more lightweight and effective metric is still needed to close the gap between oracle and actual accuracy. They also identify a methodological limitation in the prune-or-keep dichotomy. Current Slim-SC prunes chains outright; future work could merge partial reasoning or reuse KV-cache states instead of discarding them. A third limitation concerns scope: experiments are limited to math/STEM reasoning and to R1-Distill-14B and QwQ-32B, so broader generalization remains open.

A second misconception is that Slim-SC is best understood as a memory optimization only. The paper instead stresses latency, token count, and GPU occupation time alongside KV-cache statistics. Another misconception is that the method succeeds because it is aggressive. The empirical evidence presented points in the opposite direction: accuracy is stable for high thresholds $$\tilde{y}=\operatorname*{arg\,max}_{y}\sum_{i=1}^{N}\mathbf 1\!\bigl[\hat{y}_i=y\bigr].$$4, often peaking around $$\tilde{y}=\operatorname*{arg\,max}_{y}\sum_{i=1}^{N}\mathbf 1\!\bigl[\hat{y}_i=y\bigr].$$5, which indicates that conservative pruning of only very similar chains is central to the method’s performance.

Within the broader landscape of test-time scaling, Slim-SC is presented as simple, model-agnostic, and compatible with Self-Consistency-style parallel reasoning. Its practical contribution is therefore not a new reasoning objective, but an intervention on the execution dynamics of chain sampling: it reduces redundant computation early enough to affect both latency and cache usage, while attempting to preserve the diversity needed for effective majority voting.