- The paper introduces a formal taxonomy for CoT compression, distinguishing Explicit, Composed, and Implicit types through controlled experiments on Qwen2.5 and Llama-3.
- It demonstrates that higher CoT compression granularity increases SFT data requirements, highlighting a balance between output efficiency and data demand.
- RLVR is shown to effectively decompose compressed skills, boosting Pass@1 accuracy by over 80 points on tasks needing finer-grained reasoning.
Compressed Reasoning Data in LLM Post-Training: Analysis and Implications
Introduction
This work systematically investigates the impact of Chain-of-Thought (CoT) compression in post-training of LLMs via both supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR). The study proposes a formal taxonomy for reasoning traces—Explicit CoT, Composed CoT, and Implicit CoT—along with novel synthetic tasks designed to measure compositional reasoning and generalization under varying granularity and data regimes. Using controlled experiments on Qwen2.5 and Llama-3, the paper uncovers quantifiable dynamics central to optimizing the cost and efficacy of reasoning in LLMs.
Taxonomy and Experimental Framework
The paper formalizes CoT compression along two axes: the degree of operation aggregation ("granularity") and information occlusion (outputs shown versus hidden). The taxonomy is as follows:
- Explicit CoT: All reasoning steps and intermediate values are enumerated.
- Composed CoT: Multiple operations are grouped into a single step, skipping intermediates but making all applied operations explicit.
- Implicit CoT: Entire chunks of operations are executed in single steps, omitting both intermediate operations and results, emulating a "black box" chunking of skills.
A synthetic arithmetic benchmark is constructed, imposing sequential dependencies among variables with operations performed modulo 23. The dataset's op parameter controls task depth, simulating compositional reasoning. Compression granularity (g) dictates how many atomic moves are aggregated. The robustness of various CoT compressions is examined across a systematic range of model sizes and training configurations.
Main Findings
Compression Granularity and Data Requirements
Empirical results show that increasing CoT compression granularity elevates data requirements for SFT to match baseline accuracy. Specifically, both Composed and Implicit CoT require more training data and steps compared to Explicit CoT, with the effect amplifying for larger chunk sizes (g). This demonstrates a quantitative trade-off between efficiency (shorter outputs) and data demand for learning compressed reasoning policies.
Claim: Coarser-grained (higher g) compressed CoT requires substantially more SFT data to achieve comparable OOD generalization rates.
Data Scaling versus Repetition
The experiments dissect the efficacy of dataset diversity (scaling) versus repeated exposure (repetition) under fixed compute. Composed CoT accuracy benefits disproportionately from diverse, large-scale datasets, whereas Implicit CoT is susceptible to overfitting and memorization under repetitive small-sample regimes. Notably, Explicit CoT is relatively robust to repetition, but scaling produces diminishing marginal gains.
- Diverse data is critical for compressed reasoning learning; repetition aids Composed CoT but causes degradation for Implicit CoT.
Decomposability and RLVR
Crucially, SFT is unable to induce models to internally decompose compositional steps below the supervised granularity—the models fail on tasks requiring finer-grained reasoning than encountered during training. In contrast, subsequent RLVR is shown to break apart such skill "chunks," enabling generalization to tasks of greater compositional complexity without requiring explicit decomposed traces in the reward function.
Numerical Highlight: RLVR improves Pass@1 accuracy by over 80 points on tasks requiring decomposition (e.g., increasing from near chance to above 84% on odd-length composition problems).
- RLVR enables decomposition of compressed skills via on-policy exploration, unlocking atomic-level reasoning not accessible via SFT alone.
CoT Order and Generalization
Ordering of reasoning steps impacts OOD generalization. Forward (data) and backward (inverse) sequential CoT chains produce robust generalization curves on longer, OOD tasks, while hierarchical (tree-based, chunked) CoT formats fail to generalize. Backward CoT, though effective, requires more data for parity performance.
Implications for Practice and Theory
These results hold numerous implications for LLM post-training and data construction:
- Data Design: For resource-constrained SFT, high granularity Composed CoT is preferred with diverse data; otherwise, repetition is preferable only for Composed (not Implicit) CoT. Data diversity is the primary driver of compressed reasoning acquisition.
- RLVR Utility: RLVR is important not merely for reward maximization but for enabling compositional skill decomposition. This supports optimistic perspectives on RL’s utility in skill composition and generalization, contradicting claims that RLVR only sharpens base model distributions without expanding support.
- Inductive Biases: Unidirectional reasoning traces (either forward or backward) fit the inductive biases of transformer LLMs, while non-unidirectional/hierarchical formats fail to extrapolate.
- Token Cost vs. Generalization: There is an inherent Pareto frontier—compression via CoT shortens inference but at the cost of increased SFT data requirements. Compression without sufficient diversity results in memorization or catastrophic forgetting at longer, OOD tasks.
In real-world scenarios, such as arithmetic or code tasks, where the operational space is combinatorial, RLVR-driven recovery of atomic skills is essential. Manual decomposition for all possible compositions is infeasible, so learning via outcome supervision remains vital.
Limitations and Future Directions
While the study achieves high control over task structure and variable granularity by using synthetic modular arithmetic, it remains to be seen whether the findings translate to naturalistic problem spaces (e.g., noisy real-world math, scientific reasoning, or code generation). The experiments are restricted to decoder-only transformers; extension to other architectures and tasks with branching/graph-like compositionality is outstanding work. Mechanistic interpretability of internal circuits acquired through different CoT compressions is highlighted as a future avenue.
Conclusion
This study elucidates the structural requirements, benefits, and limitations of compressing reasoning traces in LLM post-training. The findings refine data efficiency strategies for compact reasoning supervision and clarify the complementary roles of SFT and RLVR: SFT enforces learned chunking but is brittle to unseen compositional generalization, while RLVR can discover and combine atomic skills hidden within these chunks. The paper establishes strong quantitative evidence for the necessity of diverse, well-sized compressed reasoning datasets and precise RLVR signals to balance cost and OOD generalization in LLM deployment.
Reference:
"Zipping the Thought: When and How Compressed Reasoning Data Works in LLM Post-Training" (2605.28008)
