A Decomposition Perspective to Long-context Reasoning for LLMs

Abstract: Long-context reasoning is essential for complex real-world applications, yet remains a significant challenge for LLMs. Despite the rapid evolution in long-context reasoning, current research often overlooks the internal complexity of the long-context reasoning task itself. In this paper, we move beyond this holistic view and decompose long-context reasoning into a set of fundamental atomic skills, and we then automatically synthesize a suite of pseudo datasets, each explicitly targeting a specific atomic skill. Our empirical analysis confirms that proficiency in these atomic skills is strongly correlated with general long-text reasoning performance. Building on this insight, we employ reinforcement learning on these pseudo datasets to sharpen the model's atomic skills, in the hope of boosting its general long-context reasoning ability. Extensive experiments across multiple benchmarks demonstrate the effectiveness of our approach: it outperforms a strong baseline by an average margin of 7.7\% (improving from 46.3\% to 54.0\%) across Loogle, Loong, LongBench-v2, BrowscompLong, Ruler-qa2, and MRCR.

• The paper decomposes long-context reasoning into five atomic skills, addressing retrieval, integration, relational reasoning, and dynamic tracking.
• It introduces an Anchor-based Reasoning pipeline to create scalable, noise-controlled datasets for targeted reinforcement learning training.
• Empirical results demonstrate up to 7.7% performance improvements across benchmarks, highlighting robust gains and effective skill integration.

Decomposition-Driven Enhancement of Long-Context Reasoning in LLMs

Introduction

The persistent challenge of augmenting long-context reasoning in LLMs arises not merely from the expansion of context windows but from the underlying complexity and hierarchical nature of real-world reasoning tasks. Addressing this, "A Decomposition Perspective to Long-context Reasoning for LLMs" (2604.07981) adopts a cognitive, decomposition-based approach: rather than treating long-context reasoning as a monolithic, black-box capability, the authors theorize and empirically justify that such reasoning is the emergent result of a spectrum of atomic cognitive skills. They propose a systematic taxonomy of these skills, devise a scalable pipeline for targeted dataset construction, and empirically demonstrate the gains—over multiple competitive baselines—achievable by reinforcement learning (RL) targeting these atomic skills.

Atomic Skill Taxonomy and Dataset Construction

The decomposition posits that high-fidelity long-context reasoning fundamentally reduces to five atomic skills, ordered by increased cognitive complexity:

1. Foundational Retrieval (Needle-in-a-Haystack, NIAH): Basic ability to locate information within vast, noisy textual environments, addressing the "lost-in-the-middle" failure mode.
2. Anti-Interference: Robust retrieval in the presence of distractors and conflicting or similar entries, assessing precise discrimination and noise immunity.
3. Global Integration: Aggregating information distributed across locations/documents, critical for multi-source evidence synthesis.
4. Relational Reasoning: Logical and structural manipulation, encompassing set operations and relationship inferences over retrieved information (akin to database-style queries).
5. Dynamic State Tracking: Multi-step computational reasoning, requiring intermediate state manipulation—crucial for tasks involving chained arithmetic or logical dependencies.

Dataset synthesis is operationalized through the Anchor-based Reasoning (AbR) framework, which injects algorithmically generated anchor–question pairs into long contexts, fabricating QA tasks that strictly target each atomic skill. This anchor-based, noise-controllable pipeline generates verifiable, scalable, and skill-specific pseudo-datasets. The pipeline enables controlled curriculum design, ranging from context length to distractor complexity, supporting progressive model alignment.

Empirical Validation of Atomic Skill Decomposition

To demonstrate the external validity of the atomic skill decomposition, the authors correlate model performance on atomic probes against real-world long-context benchmarks (e.g., Loogle, LongBench-v2, Loong, BrowsCompLong, Ruler-qa2, MRCR). Spearman correlation coefficients reveal high predictive power: for instance, NIAH and Anti-interfere scores correlate $\rho=0.95$ and $\rho=0.94$ with the average real-world benchmark performance, respectively. Notably:

• High Correlation, Low Performance: Models exhibit strong association between Anti-interfere/Multi-source and benchmark success, but absolute accuracy on these atomic tasks remains low (e.g., for Qwen2.5-32b-instruct: NIAH 37.0%, Anti-Interfere 22.13%, Multi-source 23.88%).
• Retrieval Ceiling: While most models perform adequately on foundational retrieval, competency in advanced atomic skills is the bottleneck for robust long-context reasoning.

This substantiates that long-context model weaknesses are not evenly distributed but are particularly acute in skills beyond basic retrieval.

RL-Driven Atomic Skill Enhancement

RL-based training leverages Group Relative Policy Optimization (GRPO), enhanced by Dynamic Sampling and LLM-as-a-Judge reward modeling with chain-of-thought system prompts for instruction-tuned models. Training is conducted entirely on the atomic datasets—4k synthetic samples—and does not require large-scale real-world curated long-context datasets.

Significant, architecture-agnostic performance improvements are observed. On six rigorous benchmarks, RL-enhanced models outperform backbone and SOTA baselines by average margins of up to 7.7% (DeepSeek-R1-distill-32B: 46.3% $\rightarrow$ 54.0%). Ablation studies indicate that all atomic skills contribute synergistically; omitting any skill causes marked regression, notably in Multi-source and Logic components.

The approach generalizes: gains persist across varying backbone sizes (Qwen2.5-14B/32B). For Qwen2.5-14B-instruct, the mean score increases from 35.59% to 45.83%, with single-benchmark deltas exceeding 20% on challenging tasks (Ruler-qa2: 40.28% $\rightarrow$ 63.21%).

Hierarchical Dependencies and Skill Interactions

Ablative analysis of the training curriculum verifies the non-orthogonality and hierarchy among atomic skills:

• Diagonal Dominance: Removing training on a given atomic skill selectively degrades that skill’s probe performance (e.g., Logic -29 points, Anti-interfere -21.1), indicating skill specificity.
• Hierarchical Dependence: The performance of higher-order skills (Dynamic State Tracking) disproportionately degrades when lower-order skills (Logic, Multi-source) are omitted, evidencing a stacking cognitive dependency. Multi-source integration is foundational: its removal causes uniform performance drops across all capabilities.

Robustness Across Context Lengths and Task Complexity

The curriculum yields not only improved aggregate performance but also robustness across input context lengths, with consistent uplifts from 8k to 128k tokens—and without suffering substantial information degradation. Critically, the RL-atomic method closes the gap between strong retrieval (NIAH, achieved by alternative data curricula) and generalizable logical/computational reasoning, an area where naive augmentation with long-context or synthetic tasks offers negligible benefit.

Implications

This decomposition perspective enables precise diagnosis and remediation of LLMs’ long-context reasoning deficits. Practically, it offers a data-efficient, verifiable, and scalable path for LLM alignment—mitigating typical issues in large-scale hand-curated dataset design, such as misinformation risk and latent conflict.

Theoretically, the work motivates a shift from monolithic capability-centric LLM evaluation towards hierarchy-aware, skill-specific probes and analytics. Future directions include expanding the taxonomy to additional cognitive skills, investigating inter-skill transfer phenomena, and integrating the anchor-based approach with architectural modifications (e.g., explicit memory modules and retrieval-augmented generation).

Conclusion

By decomposing long-context reasoning into five atomic skills and leveraging automated dataset synthesis with RL-based targeted training, the authors establish a framework for systematic, data-efficient enhancement and precise diagnosis of LLM reasoning fidelity in extended contexts. The empirical results demonstrate robust, consistent improvements over strong baselines, with implications for scalable model alignment, curriculum learning, and long-context model evaluation (2604.07981).