---
title: CLoT-Instruct Dataset for Hierarchical Reasoning
url: https://www.emergentmind.com/topics/clot-instruct-dataset
type: topic
---

# CLoT-Instruct Dataset for Hierarchical Reasoning

CLoT-Instruct is an instruction-tuning dataset designed for explicit forward and backward reasoning in hierarchical multi-step mathematical problem solving. Released alongside the Cognitive Loop of Thought (CLoT) framework, the dataset provides richly annotated samples where each mathematical problem is decomposed into layered sub-problems. For each reasoning step, CLoT-Instruct offers both the canonical forward inference path and explicit backward (“verification”) annotations, structured to enable self-reflective question answering and rigorous bidirectional checking. This resource is intended to facilitate research and benchmarking in reversible, efficiency-driven, and robust mathematical reasoning for instruction-tuned large language models (LLMs) [2604.06805].

## 1. Dataset Structure and Formal Specification

Each sample in CLoT-Instruct extends the classic Chain-of-Thought (CoT) approach. While standard mathematical datasets represent samples as 
\[
\tau_1 = (q_1,\, s_{1:T},\, a),
\]
where \(q_1\) is the initial problem, \(s_{1:T}\) is a sequence of reasoning steps, and \(a\) the answer, CLoT-Instruct introduces a strictly hierarchical, bi-directional format. Each record is
\[
\mathcal{X} = \left( q_{\mathrm{origin}},\, a_{\mathrm{gt}},\, \{ \tau^{(l)} \}_{l=1}^{L},\, L \right),
\]
where:
- \(q_{\mathrm{origin}}\) is the original problem statement,
- \(a_{\mathrm{gt}}\) is the ground-truth answer,
- \(L\) is the maximum hierarchy depth,
- each layer \(l=1,\ldots,L\) consists of
  \[
  \tau^{(l)} = \left( q^{(l)},\, s^{(l)}_{1:T_l},\, \{ (q^{(l)}_{\mathrm{verify},t},\, a^{(l)}_{\mathrm{verify},t}) \}_{t=1}^{T_l} \right),
  \]
  where \(q^{(l)}\) is a (sub-)question, \(s^{(l)}_{1:T_l}\) forward steps, and, for each step, a backward verification pair.

This explicit structuring enables models to navigate a problem’s decomposition and to check their inferences by reconstructing earlier reasoning from later conclusions at each hierarchical layer.

## 2. Backward Verification and Bidirectional Annotation

Backward verification in CLoT-Instruct annotates every forward step at every hierarchical level with a paired “inversion” query. Specifically, for each forward inference
\[
q^{(l)} \xrightarrow{s^{(l)}_t} q^{(l)}_{t+1},
\]
the dataset provides a logical reversal:
\[
(q^{(l)}_{\mathrm{verify},t},\, a^{(l)}_{\mathrm{verify},t}),
\]
where \(q^{(l)}_{\mathrm{verify},t}\) is a query designed so that, given the conclusion \(q^{(l)}_{t+1}\), a model can reconstruct the premise or a key intermediary. Typically, this means turning a calculated result back into reasoning about its inputs. The annotation protocol thus requires models to “justify” each step in both forward (deductive) and backward (abductive) modes—enabling built-in self-verification [2604.06805].

## 3. Hierarchical Pruning for Efficient Verification

A core feature of the CLoT-Instruct format is its support for hierarchical pruning in reasoning verification. The backward consistency score at layer \(l\) is defined:
\[
\mathcal{V}^{(l)} = \sum_{t=1}^{T_l} \log p^{\leftarrow} (s^{(l)}_t,\, q^{(l)}_t\,|\, q^{(l)}_{t+1}),
\]
where \(p^{\leftarrow}\) is a learned reverse-step probability. The pruning rule is:
```python
if V(L) > τ:
    # Global loop verified; prune all layers l < L
else:
    # Descend to layer L–1 and repeat
```
That is, if the top-level (coarsest) layer’s backward check surpasses a threshold \(\tau\), all subordinate layer verification can be skipped—drastically reducing inference and verification cost. Only if a higher-level consistency check fails do lower (more detailed) layers require verification, mitigating error propagation and maximizing computational efficiency [2604.06805].

## 4. Corpus Composition and Formatting

CLoT-Instruct comprises three primary subsets sourced from widely used grade-school mathematical reasoning benchmarks:

| Subset   | Number of Samples | Answer Type |
|----------|------------------|-------------|
| GSM8K    | 1,319            | Numeric     |
| SVAMP    | 1,000            | Numeric     |
| AddSub   | 395              | Numeric     |

Totaling 2,714 samples, the dataset is not partitioned into train/validation/test; all samples form a monolithic instruction resource. All reasoning steps, both forward and backward, utilize “<answer>…</answer>” tags enclosing the model’s numeric prediction; this format extends, in principle, to multiple-choice by enclosing the selected letter [2604.06805].

## 5. Example Instance

A typical example from the SVAMP-derived subset demonstrates the hierarchical and bidirectional annotation structure:

**Original Problem**  
Q₀: "Kylar went to the store to buy glasses for his new apartment. One glass costs \$5, but every second glass costs only 60% of the price. Kylar wants to buy 16 glasses. How much does he need to pay?"  
Ground-truth answer: 64  
Max layers \(L=2\).  
---
Layer 2 (coarse):
- \(q^{(2)}\) = original problem.
- Forward steps \(s^{(2)}\):  
  1. Compute total cost of 16 glasses, grouping into pairs.
  2. …
- Final answer \(q^{(2)}_{t+1} = 64\)
- Backward-verification:
  - \(q^{(2)}_{\mathrm{verify},1}\): "If the total cost is 64, what is the cost of a single glass at full price X?"  
    \(a^{(2)}_{\mathrm{verify},1} = 5\)  
  - … (one per forward step)

If \(\mathcal{V}^{(2)} > \tau\), all more detailed verifications (layer 1) can be pruned at inference.

## 6. Intended Usage and Model Training

CLoT-Instruct is designed for instruction-tuning LLMs—providing explicit supervision for both structured multi-step problem decomposition and integrated self-consistency checking. Reported models trained on CLoT-Instruct include GPT-4o-mini, GPT-4-1106-preview, and DeepSeek-v3, evaluated on tasks such as AddSub, GSM8K, SVAMP, AQuA, and CommonsenseQA. Exact-match accuracy is the main evaluation metric. On AddSub, GPT-4o-mini fine-tuned with CLoT-Instruct achieves 99.0% accuracy, improving upon standard CoT (94.9%) and CoT-SC (96.1%). Average accuracy across six benchmarks is 89.6% for GPT-4o-mini plus CLoT, about four points higher than CoT-SC for equal token budgets. Hierarchical pruning reduces verification token count by 41.8% (e.g., 325K tokens to 136K on 100 GSM8K problems) [2604.06805].

## 7. Significance and Future Directions

CLoT-Instruct establishes a benchmark for datasets supporting reversible, vision-inspired, and hierarchical reasoning in LLMs. Its format mandates forward and explicit backward annotation, training models for rigorous self-verification and hierarchical inference. The dataset enables research into consistency-based error mitigation, pruning-based efficiency, and reversible chains of thought, forming a foundation for further advances such as more sophisticated bidirectional supervision, extension to non-numeric/multiple-choice problems, and domain-specific hierarchical data design. Its methodological innovations—especially hierarchical pruning—directly address inference cost and error compounding in long CoT protocols [2604.06805].

Source: https://www.emergentmind.com/topics/clot-instruct-dataset