REVEAL: Reasoning Verification Evaluation Dataset

• The REVEAL dataset is a fine-grained resource offering step-level annotations that verify LLM-generated reasoning chains.
• It employs a detailed annotation schema assessing relevance, attribution, and logical correctness across 4,207 reasoning steps.
• Its benchmark framework addresses gaps in traditional QA by evaluating both intermediate steps and overall chain validity using rigorous metrics.

The REVEAL (Reasoning Verification Evaluation) dataset is a step-level, fine-grained resource for benchmarking automated verifiers of complex reasoning chains generated by LLMs in open-domain question answering (QA) contexts. REVEAL was designed to address the critical gap in available evaluation infrastructure, which previously emphasized only end-answer correctness and overlooked the verification of intermediate reasoning steps. It provides comprehensive, per-step annotations for relevance, attribution to evidence, and logical validity, thereby enabling the principled study of chain-of-thought (CoT) verification, fact attribution, and logical soundness in LLM-generated rationales (Jacovi et al., 2024).

1. Motivation and Conceptual Foundations

CoT prompting has emerged as the dominant paradigm for eliciting multi-step, human-like reasoning from LLMs, substantially boosting answer accuracy in complex QA. However, prevailing evaluation protocols have remained fixated on the correctness of the LLM's final answer, failing to interrogate the validity and evidence-groundedness of the stepwise reasoning process. This has impeded the development of robust verification methods for CoT, as no public dataset previously supplied step-level gold standards for task-specific factuality, logical consistency, or evidence attribution. REVEAL was conceived to fill this void by rigorously annotating each reasoning step for its relevance (necessity for answering the question), type (knowledge attribution or logical inference), factual entailment grounded in external evidence, and logical coherence, accompanied by detailed free-text justifications for each label (Jacovi et al., 2024).

2. Dataset Composition and Annotation Schema

REVEAL covers 817 unique, open-domain QA questions, sampled from StrategyQA, MuSiQue, Sports Understanding, and Fermi. The full dataset comprises 1,226 generated CoT answers, totaling 4,207 reasoning steps (average ≈ 3.4 steps/chain). Generation is balanced across three preeminent LLMs: Flan-PaLM-540B, GPT-3 (text-davinci-003), and Flan-UL2-20B. Each step in a reasoning chain is labeled independently by five annotators, following a rigorously piloted instruction set.

The annotation schema is formalized as follows:

• Relevance: Is the step nontrivially helpful for answering the question? (binary)
• Type: Does the step introduce new world knowledge from external evidence ("attribution"), does it follow logically from previous steps ("logical"), or both?
• Attribution Correctness (for attribution steps): Is the claim fully entailed, partially entailed, unsupported, or contradicted by up to three retrieved evidence paragraphs?
• Logical Correctness (for logical steps): Is the inference from previous steps valid (correct) or not (incorrect)? If a logical step depends on earlier incorrect logic, it is marked "undefined".
• Free-text Justification: Each label is explained with a brief rationale.

Annotator agreement is quantified by Krippendorff’s α (0.49 for attribution steps, 0.46 for logical steps). A subset of 161 questions is answered by all three LLMs for cross-model analyses. Ambiguous or disputed steps are released in the REVEAL-Open split alongside the curated REVEAL-Eval data (Jacovi et al., 2024).

3. Data Format and Availability

The dataset is distributed as JSON-lines files, each record corresponding to one model-generated CoT answer. Records include the question ID, question text, dataset source, model name, and an array of annotated steps. Each step details:

• Step text
• Labels for relevance, type, logic
• Evidence items (up to three paragraphs per attribution step) with attribution labels
• Up to five free-text, anonymized annotator justifications per label

The resource is public on Hugging Face (https://huggingface.co/datasets/google/reveal) under a license restricting use in LLM pretraining to minimize contamination (Jacovi et al., 2024).

4. Benchmarking Tasks and Evaluation Metrics

REVEAL targets both step-level and chain-level reasoning verification. Core tasks and their associated metrics include:

• Step-level classification:
  • Relevance (binary accuracy)
  • Type (three-way classification: attribution / logical / both)
  • Attribution entailment (support/contradiction/unsupported), with macro-F1 for imbalanced classes
  • Logical correctness (binary, macro-F1)
• Chain-level classification:
  • Aggregation of stepwise labels to accept or reject a full CoT as a correct justification (binary)

Pipeline verification (deciding chain validity from stepwise decisions) is compared to direct single-shot chain labeling. Baseline macro-F1 on step attribution reaches ≃ 88% for NLI-based classifiers (2-way), but drops to ≃ 55–70% on 3-way (with contradiction as the most challenging). Logical error detection remains a bottleneck (F1 ≈ 33–47% for "incorrect" logic detection by current LLMs), while pipeline chain correctness detection (PaLM-2-L) substantially outperforms direct chain evaluation by LLMs (Jacovi et al., 2024).

5. Insights from Baselines and Analyzer Performance

Multiple empirical findings are surfaced by REVEAL:

• LLM-generated chains frequently include reasoning steps unsupported by retrieved evidence (38.6% of attribution steps investigated are not confirmed).
• Verifiers underperform on logical error detection and especially on contradiction identification.
• Contradiction-sensitive models are required to achieve robust verification on the most failure-prone cases.
• Pipeline verifiers leveraging stepwise granularity consistently outperform single-chain classifiers, underscoring the value of fine-grained supervision.
• Ambiguous and borderline items (REVEAL-Open) highlight the prevalence of world knowledge assumptions, hedging, and ambiguous references as sources of both LLM and annotator error.

6. Implications for Verification, NLI, and Model Development

REVEAL enables new research on verifier architectures (including NLI-based and pipeline models), explanation generation, and fine-grained rationality analysis. Its stepwise gold standard supports studies of when, how, and why LLMs’ CoTs fail (e.g., which error modes are due to logical jumps vs. unsupported facts). The resource offers a foundation for joint retrieval+verification pipelines and for training contradiction-aware verifiers. Free-text justifications can serve as supervision for developing interpretable models or training better annotators (Jacovi et al., 2024).

7. Limitations and Future Extensions

REVEAL is limited in domain (four QA benchmarks), language (English only), and annotation scale (≈ 4,200 reasoning steps). Inter-annotator agreement, while carefully measured, reveals intrinsic ambiguity in reasoning verification, especially on complex or domain-specific chains. A future direction includes enhancing evidence retrieval mechanisms, annotation guideline refinement focusing on edge cases (hedging, temporal consistency), and releasing new domains. Incorporating free-text explanations into both model and annotator guidance, as well as expanding coverage to more model architectures, will underpin further advances in reasoning verification research.

REVEAL thus sets a benchmark for rigorous, stepwise reasoning verification—enabling the interrogation not just of answer correctness but of the entire evidence-grounded, logically valid reasoning trajectory produced by modern LLMs (Jacovi et al., 2024).