DAG-MATH CoT Format

• DAG-MATH CoT Format is a structured paradigm that represents chain-of-thought reasoning as explicit DAG traversals with nodes for intermediate results and edges for rule-based inferences.
• It improves evaluation by distinguishing mere answer accuracy from logical consistency through metrics like logical closeness and Perfect Reasoning Rate.
• The format offers actionable diagnostic insights and benchmarking capabilities, aiding the development of robust mathematical reasoning in language models.

The DAG-MATH CoT Format defines a rigorously structured paradigm for representing, generating, and evaluating chain-of-thought (CoT) mathematical reasoning as explicit traversals over directed acyclic graphs (DAGs). Within this framework, every intermediate step of an LLM's solution corresponds to a node, and explicit edges encode rule-based inferences with parent-child dependencies, making both the logical structure of the argument and the derivation trajectory explicit. This approach not only sharpens the distinction between rote final answer accuracy and genuine, stepwise, rule-consistent reasoning but also facilitates actionable diagnostic metrics and cross-model benchmarking in mathematical LLMs (Zhang et al., 19 Oct 2025).

1. Rationale and Formalization

The central motivation for DAG-MATH CoT is the shortcoming of standard free-form CoT outputs: while LLMs achieve high answer accuracy, the logical integrity and reproducibility of the inferred reasoning remains opaque. By structuring CoT as a rule-based stochastic process on a DAG, the format exposes the provenance and dependencies of each intermediate fact, presenting mathematical reasoning as a sequence of atomic inference steps that collectively instantiate a well-defined, acyclic derivation.

Formally, the process is modeled as follows:

• Let the input prompt $P_{in}$ define the problem statement.
• Let $G$ be a DAG whose nodes $v$ represent atomic statements or intermediate results, and edges $(u \to v)$ encode explicit rule-based inferences, i.e. the application of a mathematical rule or operation.
• The LLM’s solution trajectory, $V_1, V_2, \ldots, V_L$, is an ordered walk along this DAG, where each transition is permitted only once all parents of a node are present in the output history:

$$P(V_t = v \mid V_1,...,V_{t-1}, P_{in}) > 0 \iff \{ \text{parents of } v \} \subseteq \{ V_1,...,V_{t-1} \} ,\ v \notin \{ V_1,...,V_{t-1} \}$$

Each step is encoded as a triple (Edge, Parent(s), Node), aligning with the “Edge → Parent(s) → Node” format, distinguishing logical operations and their dependencies from mere sequential text.

2. Logical Closeness and Reasoning Metrics

To quantitatively diagnose reasoning fidelity, the paper introduces logical closeness as a core metric: a trajectory is logically closed if every non-sink node (intermediate result) in the constructed DAG is used as a parent by at least one subsequent node, i.e. for every $v \in G_{gen}$ except the final answer node,

$\deg(v \mid G_{gen}) \geq 1.$

where $\deg(v \mid G_{gen})$ is the out-degree (number of downstream uses) of node $v$.

The Perfect Reasoning Rate (PRR) is defined as the fraction of examples where both (a) logical closeness holds and (b) the final answer (sink node) is correct:

$$\text{PRR}(P_{in}) = \mathbb{E}\left[\delta_{close}(G_{gen}) \cdot \delta_{final}(G_{gen})\right]$$

with $\delta_{close}$ and $\delta_{final}$ being boolean indicators for logical closeness and final answer correctness, respectively.

This metric enables distinguishing solutions that are both correct and logically justified from those that may reach correct answers via exploratory or over-extended reasoning.

3. Benchmark Construction and Empirical Findings

To systematize evaluation, a benchmark of 2,894 gold-standard DAGs was constructed by staged LLM prompting:

• Stage 1: Generating atomic reasoning “Nodes.”
• Stage 2: Assigning minimal “Parent” sets to preserve acyclicity.
• Stage 3: Explicitly annotating “Edges” as justifications for each node.

Empirical analysis across diverse LLMs (e.g., Gemini-2.5-Flash, GPT-4.1, Qwen3-30B) revealed that:

• Final answer accuracy (PASS@1) can be inflated through broad search, but PRR—which also demands logical closeness—remains a more stringent and meaningful measure of true reasoning capacity.
• As mathematical problem difficulty increases, associated DAGs grow larger, more sparse, and exhibit higher maximum in-degree/out-degree, indicating the need for deeper and more distributed reasoning.
• PRR decays exponentially with chain length in toy examples, suggesting that complex, multi-branch reasoning poses significant challenges for autoregressive LLMs.

4. Methodological and Technical Details

The node-level generation process is captured formally:

• The LLM autogenerates $V_1 \sim P(\cdot | P_{in})$, followed by

$$V_t \sim P(\cdot | V_1,...,V_{t-1}, P_{in}),\quad t = 2, ..., L,$$

restricted to nodes whose parent sets are subsets of those already sampled.

The transition is permitted only if:

• $v \notin \{ V_1, ..., V_{t-1} \}$,
• All parents of $v$ are in $\{ V_1, ..., V_{t-1} \}$.

Out-degree ($\deg(v \mid G_{gen})$) is calculated as the number of edges leaving from node $v$, enforcing strict dependency registry.

Structural statistics—nodes, edges, density, in/out-degree—are computed for each gold-standard DAG, revealing alignment with problem complexity and model performance.

5. Impact, Applications, and Context

The DAG-MATH CoT format offers a bridge between unconstrained natural language CoT and fully formalized proof systems. Its applications include:

• Precise LLM evaluation: moving beyond final answer accuracy, explicitly quantifying reasoning path completeness and consistency.
• Diagnostic analysis: revealing patterns of “under-reasoning” (omitted necessary steps) and “over-reasoning” (redundant or exploratory paths), thereby guiding model improvement.
• Interoperability with proof systems: providing an intermediate format for (semi-)formalized reduction of mathematical arguments, paving the way for scalable formal verification.

This method also enables quantitative research into reasoning generalization guarantees and facilitates regularizers (such as minimum description length) to suppress over-reasoning.

6. Future Directions and Resource Availability

Key research avenues include:

• Algorithmic development for automated registration of reasoning guarantees analogous to those in supervised learning.
• Regularization to minimize unnecessary DAG complexity.
• Fine-grained error analysis distinguishing exploration-driven steps from true logical derivations.

Benchmark data and code for DAG-MATH formatted CoT are made publicly available at [https://github.com/YuanheZ/DAG-MATH-Formatted-CoT] for reproducibility and continued development (Zhang et al., 19 Oct 2025).

---

In summary, the DAG-MATH CoT Format rigorously recasts mathematical reasoning as rule-based DAG traversal, grounding each step as an explicit, dependency-checked node with justified edges. This provides a robust abstraction for evaluating, comparing, and ultimately improving logical reasoning in LLMs—filling the space between unconstrained natural language chains and fully formal proofs, and enabling new metrics and diagnostic tools for mathematical AI research.