Agentar-DeepFinance-300K Dataset

• Agentar-DeepFinance-300K is a large-scale financial reasoning dataset that combines questions, chain-of-thought reasoning, and rich metadata to promote deep financial inference.
• The dataset employs a multi-stage pipeline including Multi-perspective Knowledge Extraction, advanced CoT synthesis, and Self-Corrective Rewriting to enrich reasoning diversity and accuracy.
• Experimental results highlight significant improvements for challenging financial tasks, emphasizing the benefits of long CoTs and careful synthesizer selection for enhanced performance.

Searching arXiv for the target paper and closely related financial reasoning/CoT work to ground the article. Agentar-DeepFinance-300K is a large-scale financial reasoning dataset introduced to improve chain-of-thought (CoT) reasoning in financial LLMs through systematic synthesis optimization rather than simple answer filtering. It is presented as a large-scale financial dataset with 300K scale and is designed around the premise that financial supervision should contain not only correct answers but also deep, diverse, and challenging reasoning trajectories. The dataset combines questions, solutions containing both CoT and final answer, and rich metadata, and it is constructed through a multi-stage pipeline consisting of Multi-perspective Knowledge Extraction (MKE), CoT sampling and verification, and Self-Corrective Rewriting (SCR) (Zhao et al., 17 Jul 2025).

1. Definition, scope, and dataset structure

Agentar-DeepFinance-300K is defined as a large-scale financial reasoning dataset. Its instances include questions, solutions containing both CoT and final answer, and rich metadata. The metadata is multi-dimensional and includes content, ability, complexity, quality, language, and task labels.

The reported annotation schema includes domain-specific financial topic classification under Content; Ability dimensions covering language, reasoning, knowledge, mathematics, code, instruction following, and agents; Complexity scored from 1–10; Quality scored from 1–10; Language such as Chinese (zh) and English (en); and Task as a hierarchical task label, with an example given as Text Creation → Marketing Copy Generation (Zhao et al., 17 Jul 2025). A notable filtering rule is that samples with a quality rating below 8 are excluded from the dataset.

The paper also states that, beyond the main corpus, an additional dataset annotated by financial experts was curated for practical financial capabilities. This situates the resource not merely as a synthetic benchmark artifact but as a dataset intended to better align with real-world financial interaction. A plausible implication is that the metadata and filtering rules are treated as part of the supervision design rather than as ancillary bookkeeping.

2. Motivation and problem formulation

The motivating problem is financial reasoning under LLMs. The paper argues that, although general LLMs and financial LLMs have improved, financial reasoning remains difficult because it often requires multi-step numerical reasoning, domain-specific knowledge, instruction following, reasoning over implicit finance concepts, and robustness to diverse answer formats (Zhao et al., 17 Jul 2025).

A central criticism is directed at existing CoT synthesis methods for finance. These methods usually sample CoTs directly from a reasoning model for selected QA pairs and then apply rejection sampling or filtering. The paper characterizes this regime as shallow CoT sampling and argues that it leaves open the question of how to construct a well-designed knowledge space for finance reasoning. The stated gaps are therefore twofold: knowledge coverage and reasoning depth.

This framing distinguishes Agentar-DeepFinance-300K from datasets organized primarily around answer correctness. A common misconception would be to regard financial CoT synthesis as a straightforward extension of general-domain answer filtering. The reported design instead treats knowledge-space construction as an explicit research problem: the training corpus should be more exhaustive and diverse, and it should expose models to more complex trajectories, not just correct answers.

3. Knowledge-space construction through MKE

The first stage of the construction pipeline is Multi-perspective Knowledge Extraction. MKE is designed to build a richer financial knowledge space by extracting information from seed corpora in three complementary ways: Q2A, A2Q, and T2Q (Zhao et al., 17 Jul 2025). The seed corpora are required to cover diverse financial domains, and the paper notes that existing public datasets are often based on standardized materials like exams, which are less representative of real user interactions. To bridge that gap, the authors curate a large proprietary dataset with real domain expert annotations.

Q2A, described as Direct Curation, is the foundation of the knowledge extraction stage. It directly harvests structured QA pairs from seed corpora, removes duplicates, and filters low-quality content. Its role is to preserve canonical financial question-answer patterns.

A2Q, described as Counterfactual Augmentation, expands the knowledge space by generating questions from perturbed answers. The reported perturbation operations are semantic negation and contextual antonym substitution. These produce adversarial answer variants, after which LRMs generate questions for those answers. The generated QA pairs are then verified through a multi-stage process assessing semantic coherence, logical consistency, and answer faithfulness. The paper describes A2Q as increasing knowledge space density and causal connectivity. In effect, the method explores the neighborhood around core financial concepts.

T2Q, described as CoT Knowledge Mining, targets knowledge latent in the reasoning traces themselves. The paper notes that CoTs often introduce additional knowledge points during the thinking process and that these may be crucial even if they are not common-sense facts. The stated procedure is to summarize CoTs, prompt LRMs to identify key processes, and create QA pairs from those identified processes. This is intended to uncover implicit reasoning dependencies that exist inside long CoTs.

The paper summarizes MKE as achieving exhaustive knowledge modeling: Q2A preserves canonical patterns, A2Q strengthens causal connectivity via counterfactual perturbation, and T2Q uncovers implicit reasoning dependencies. This suggests that the resource is organized around multiple views of financial knowledge rather than a single extraction channel.

4. CoT synthesis, verification, and self-correction

After MKE generates QA pairs, the pipeline samples multiple CoTs and corresponding answers from current advanced large reasoning models and performs answer consistency verification against golden answers (Zhao et al., 17 Jul 2025). The stated goal is to create high-quality, diverse, and challenging reasoning trajectories.

A key implementation detail concerns answer verification. Because finance answers may have variable formats, the paper states that regular expressions are insufficient for verification, especially for monetary expressions. A lightweight model is therefore used to match answers. Only QA pairs satisfying matching numerical precision and logical coherence are retained. This verification step is presented as necessary for downstream reliability.

The final stage is Self-Corrective Rewriting, which is applied to QA pairs for which the model initially fails to sample the correct answer. Rather than discarding such cases, SCR attempts to recover them. The method is organized into a reflection phase and a rewriting phase.

In the reflection phase, the model compares its incorrect answer $A$ with the golden answer $A^{*}$ and generates diagnostic reflections $T_{rf}$ containing potential mistakes. In the rewriting phase, the reflection CoT $T_{rf}$ is merged with the original reasoning trace $T$, after which the model continues generation to produce a new CoT $T_{rw}$ and revised answer $A_{rw}$. The revised answer is then verified against $A^{*}$. If verification succeeds, the process stops; otherwise, the correction cycle continues until the iteration limit.

The paper states that the final output structure alternately appends the original CoT, the reflection CoT $T_{rf}$, and the rewritten CoT $T_{rw}$ across iterations, with the ultimate answer being $A^{*}$0. The reported practical effect is that SCR turns previously unusable hard questions into valuable training data, and that the resulting CoTs are often much longer. A plausible implication is that failed generations are treated not as noise to discard but as difficult supervision to rehabilitate.

5. CoT Cube and the analysis of CoT effectiveness

The paper’s systematic investigation is termed CoT Cube. It analyzes at least three explicit factors affecting CoT effectiveness in financial reasoning: necessity, length, and synthesizer. It also implicitly connects these factors to task type, difficulty, benchmark or domain differences, and response length stability (Zhao et al., 17 Jul 2025).

Under necessity, the paper asks which financial questions actually benefit from CoT. The analysis is conducted along the axes of task type and question difficulty. For difficulty-based analysis, questions are split into simple and hard subsets based on correctness labels from superior LLMs such as Qwen2.5-14B-Instruct, with equal sample sizes maintained. On FinQA, performance changes from 67.65 → 68.96 for simple questions and from 63.03 → 69.31 for hard questions when CoT is used. On Fin-Eva, the changes are 84.21 → 85.01 for simple questions and 75.80 → 85.96 for hard questions. The reported takeaway is that CoT helps both subsets but helps hard problems much more. For task-type analysis, the dataset is partitioned into 6 subsets aligned with the dataset taxonomy. The average rises from 72.32 without CoT to 80.18 with CoT, with especially large gains in financial mathematics, where performance changes from 31.71 → 61.15.

Under length, the paper studies whether longer CoTs are better for training. Using QwQ-Plus as the synthesizer, it controls length through prompt-based variants: long CoTs as original CoTs, middle-length CoTs by adding “Be concise,” and short-length CoTs obtained by manually simplifying overthinking examples and using them as in-context demonstrations. For FinQA, response length drops relative to long CoTs by 25.0% for medium and 29.6% for short. On generated responses, Fin-Eva scores are 90.99 for long, 91.07 for medium, and 89.51 for short; FinQA scores are 72.97 for long, 72.45 for medium, and 72.97 for short. In distillation, the student trained on long, medium, and short corpora receives average CoT lengths of 1352.16, 1248.42, and 1020.26, respectively. The resulting Fin-Eva scores are 85.53, 85.40, and 85.19, while FinQA scores are 68.79, 65.13, and 65.56. The paper’s explicit takeaway is: “Financial reasoning requires long CoTs.”

Under synthesizer, the paper asks which reasoning model is best as a CoT teacher. The compared teachers include DeepSeek-R1, QwQ-Plus, Qwen3-235B-A22B, and Qwen2.5-7B-Instruct. The central finding is that a model that reasons best is not necessarily the best teacher. Even though DeepSeek-R1 shows very strong intrinsic reasoning, QwQ-Plus yields the best distilled student performance overall. Reported distilled improvements of QwQ-Plus over DeepSeek-R1 are +0.2 on Fin-Eva, +2.2 on FinQA, +1.6 on MATH, and +1.0 on GPQA. The paper also observes that QwQ-Plus generates longer CoTs and suggests that this likely contributes to better student training. As a result, QwQ is chosen as the primary generator for financial CoT synthesis.

These analyses collectively support a nontrivial conclusion: teacher quality for distillation is not identical to teacher reasoning ability. They also indicate that CoT utility in finance is contingent on difficulty, task structure, and supervision properties rather than on answer correctness alone.

6. Experimental configuration, results, and limitations

For rapid experiments, the authors randomly select 60K samples from the rigorously quality-filtered dataset. The implementation details reported are the use of the ms-swift framework, Qwen2.5-7B-Instruct as the student model, and 3 epochs of training (Zhao et al., 17 Jul 2025). Benchmarks include FinQA, Fin-Eva, a self-sampled in-domain subset, and the general reasoning benchmarks MATH-500 and GPQA-Diamond. Evaluation is rule-based on Fin-Eva for multiple-choice and true/false; for FinQA and in-domain open-ended questions it uses LLM-as-a-judge; for MATH-500 and GPQA-Diamond it uses pass@1, with 1 response per query for MATH-500 and 8 responses per query for GPQA-Diamond.

The ablation study on MKE reports incremental and joint gains. Q2A alone gives +3.09 on Fin-Eva. Q2A + T2Q gives +1.12; Q2A + A2Q gives +1.47; and Q2A + A2Q + T2Q gives +1.92. The reported interpretation is that multi-perspective knowledge construction contributes additive value.

The SCR analysis is conducted on the FinQA training set. The paper reports that 5K samples were used and that 4 candidate answers per question were sampled with QwQ-Plus. After verification, 1,337 samples lacked correct answers. The authors compare training on these incorrectly sampled cases using self-consistency and random sampling, and they find that training on incorrectly sampled data can still help and that random sampling outperforms self-consistency in that setting. After applying SCR, 862 samples with correct answers are recovered, and performance improves by +1.74. This is presented as evidence that SCR converts failed generations into usable training signal.

The stated future direction is multi-modal financial reasoning, including visual financial content such as charts. This implies that the current dataset is text-centric. The paper also states that metadata is annotated with LLMs using a carefully designed manual prompt, that quality below 8 is filtered out, and that answer verification uses a lightweight model rather than regex. The resource is publicly released, with the explicit statement: “We publicly release Agentar-DeepFinance-300K, hoping to advance the research in financial reasoning models.”

Taken together, the contribution is both a dataset and a methodology for constructing better financial reasoning supervision. The factual record presented in the paper supports four specific conclusions: CoT is especially useful for difficult and reasoning-heavy finance tasks; synthesizer choice matters more than raw teacher intelligence for distillation; long CoTs are often beneficial for finance; and the combination of MKE and SCR improves downstream performance (Zhao et al., 17 Jul 2025).