HARP: Human Annotated Reasoning Problems

• HARP is a supervised reasoning framework that pairs questions with human-authored step-by-step rationales and answers, enabling transparent problem-solving.
• It is integrated into an SFT+RLVR pipeline where initial fine-tuning on rationale data guides subsequent reinforcement learning to refine answer accuracy.
• The benchmark spans competition math problems and patterned reasoning tasks, emphasizing scalable annotation, quality control, and robust evaluation.

Searching arXiv for the cited HARP-related papers to ground the article in current literature. arxiv_search query="(Pang et al., 14 Oct 2025) Reasoning Pattern Matters Learning to Reason without Human Rationales" max_results=5

arxiv_search query="(Yue et al., 2024) HARP A challenging human-annotated math reasoning benchmark" max_results=5

arxiv_search query="(Recchia et al., 29 Mar 2025) FindTheFlaws Annotated Errors for Detecting Flawed Reasoning and Scalable Oversight Research" max_results=5

arxiv_search query="(Clark et al., 2023) BaRDa A Belief and Reasoning Dataset that Separates Factual Accuracy and Reasoning Ability" max_results=5

Human Annotated Reasoning Problems (HARP) denote supervised reasoning benchmarks in which each training example is a triple $(q_i, r_i, a_i)$ consisting of a question, a human-written rationale, and an answer. In this formulation, expert annotators read a question $q_i$, write an explicit, step-by-step rationale $r_i$ that leads to the correct answer $a_i$, and thereby provide models with human-guided reasoning trajectories. The principal motivation is to encourage LLMs to learn to “think through” problems explicitly rather than rely on surface cues or spurious statistical patterns. The same acronym is also used for a specific benchmark, “HARP: A challenging human-annotated math reasoning benchmark,” which contains 5,409 problems from U.S. national math competitions and an average of 2.14 distinct human-written solutions per problem (Pang et al., 14 Oct 2025, Yue et al., 2024).

1. Conceptual scope and nomenclature

In the broad usage introduced in work on rationale-efficient reasoning, HARP refers to a class of supervised reasoning problems defined by the availability of human-authored rationales paired with answers. In this sense, HARP is a data regime: the model is exposed not only to correct outputs but also to explicit intermediate reasoning traces. This framing makes rationale collection a first-class design choice, because the rationale corpus determines the initial reasoning behaviors that can be imparted during supervised fine-tuning (Pang et al., 14 Oct 2025).

In a narrower and benchmark-specific usage, HARP denotes a math reasoning dataset scraped from the Art of Problem Solving Wiki and spanning A(J)HSME, AMC, AIME, and USA(J)MO problems from 1950–2024. Its final problem count is 5,409, with a default short-answer split of 4,780 automatically checkable problems and a multiple-choice split of 4,110 problems with human-written distractors. The benchmark was introduced partly because previously tough math evaluations such as MATH were reported as close to saturation by frontier models (Yue et al., 2024).

This dual usage is important. A common misconception is to treat HARP as only the competition-math benchmark. The literature instead supports two levels of reference: HARP as a general supervised reasoning format centered on $(q,r,a)$ triples, and HARP as a particular instantiation for mathematics. Related resources such as BaRDa and FindTheFlaws are not named HARP, but they are explicitly positioned as human-annotated reasoning resources or as blueprints for integrating into a larger HARP benchmark (Clark et al., 2023, Recchia et al., 29 Mar 2025).

2. Function within the SFT+RLVR paradigm

Modern LLM training often uses HARP-style data through a two-stage SFT+RLVR pipeline. In the supervised fine-tuning stage, the model $\pi_\theta$ is optimized on $(q,r,a)$ triples to maximize the likelihood of the concatenated rationale–answer sequence $f_e(r,a)$ given $q$. Equivalently, one minimizes

$$L_{\mathrm{SFT}}(\theta) = - \sum_i \log P_\theta\bigl(f_e(r_i,a_i)\mid q_i\bigr).$$

This stage establishes explicit reasoning behaviors by exposing the model to human-guided reasoning trajectories rather than only terminal answers (Pang et al., 14 Oct 2025).

The second stage is Reinforcement Learning with Verifiable Rewards (RLVR). Initialized from $q_i$0 learned by SFT, the model is further tuned using only question–answer pairs $q_i$1 and a reward that can be computed without consulting gold rationales:

$q_i$2

where

$q_i$3

Here, $q_i$4 is a verifiable reward, for example $q_i$5 if $q_i$6, else $q_i$7, and the KL penalty toward a static reference policy $q_i$8 prevents excessive deviation. Practical optimizers include PPO or GRPO (Pang et al., 14 Oct 2025).

Within this framework, HARP rationales are not merely explanatory metadata. They are the warm-start signal that shapes the policy before reward optimization. The cited work argues that by warm-starting RL with an SFT stage that teaches explicit reasoning trajectories, LLMs learn both formal reasoning patterns and how to refine them via self-generated rollouts (Pang et al., 14 Oct 2025).

3. Patterned reasoning tasks and the rationale bottleneck

A central development in the HARP literature is the distinction between general reasoning tasks and “patterned reasoning tasks.” A patterned reasoning task is one where every instance shares the same underlying solution procedure $q_i$9, while only the surface content varies. Formally, a stable reasoning pattern $r_i$0 applies to all instances, each instance $r_i$1 has content $r_i$2, and the output is $r_i$3. Examples given include classification with fixed decision criteria, verification routines, or extraction under a known schema (Pang et al., 14 Oct 2025).

For such tasks, the paper “Reasoning Pattern Matters: Learning to Reason without Human Rationales” reports two strands of evidence. First, in cause-side controlled experiments on numerical semantic matching, reducing the SFT rationale set from 10 k to 1 k examples while preserving the same pattern $r_i$4 causes only a $r_i$5 point F1 drop after full SFT+RLVR, and intentionally corrupting 25% of rationales while leaving the procedural pattern intact yields only a $r_i$6 point F1 loss. Second, effect-side analysis introduces Rollout-based Forking Token Detection (RFTD), which identifies high-entropy positions in a generated rationale, substitutes top-$r_i$7 alternative tokens at each position, generates $r_i$8 continuations, and marks a token as a forking point if the divergence rate exceeds a threshold $r_i$9. Models trained with SFT+RLVR show forking tokens directly tied to pattern steps, whereas purely RLVR or hint-based methods show generic connectors that do not reflect the task’s core reasoning structure (Pang et al., 14 Oct 2025).

These results motivate PARO, “Pattern-Aware LLMs as Rationale AnnOtators.” PARO encodes $a_i$0 as step-by-step instructions in the prompt, provides two human-annotated exemplars to illustrate format and depth, and does not feed in the gold answer in the prompt. For each $a_i$1 pair, a large reasoning LLM such as Qwen3-235B-Thinking generates a synthetic rationale $a_i$2, after which the backbone is fine-tuned on $a_i$3 triples using $a_i$4 and then trained with RLVR as usual. No new loss functions are introduced; the algorithmic novelty lies in prompt engineering and rationale collection (Pang et al., 14 Oct 2025).

On numerical semantic matching, full human SFT(10 k)+RLVR attains 92.3% accuracy and 83.2% F1, whereas PARO-generated 1 k rationales with SFT(1 k)+RLVR attain 92.2% accuracy and 83.6% F1. On transaction purpose classification with 62 categories, human 1 k SFT+RLVR yields 87.9% accuracy and 87.2% F1, whereas PARO 1 k SFT+RLVR yields 88.2% accuracy and 87.9% F1. The paper states that PARO achieves parity or slight gains over human-annotated rationales while using 10× fewer human examples, specifically only two pattern exemplars plus a small seed set (Pang et al., 14 Oct 2025).

A plausible implication is that, for patterned reasoning tasks, the scarce resource is not large-scale rationale volume as such, but accurate specification of the shared reasoning pattern $a_i$5. The paper itself frames this as a paradigm shift for future HARP design (Pang et al., 14 Oct 2025).

4. Major benchmark realizations and adjacent resources

Several datasets instantiate HARP-like principles with different annotation granularities and evaluation targets.

Resource	Core annotation unit	Distinctive property
HARP (math)	Problems with human-written solutions	5,409 competition problems; six difficulty levels
BaRDa	Single-step entailments	Separates factual accuracy and reasoning accuracy
FindTheFlaws	Long-form correct and flawed solutions	Error localization and critique-oriented annotations
MHPR	Human-centric vision-language items	Multi-level data design with SFT-D and RL-D

The competition-math HARP benchmark is the most direct named instantiation. It contains 5,409 distinct problems, of which 4,780 have answers that can be checked automatically via symbolic libraries such as SymPy. It also retains 4,110 multiple-choice problems and excludes 310 proof-based problems, 6 calculus problems, and 313 problems that require multiple choice from the default short-answer split. Human experts assign six difficulty levels, from 1 to 6, and the benchmark includes an average of 2.14 distinct human-written solutions per problem, with a maximum of 14 (Yue et al., 2024).

BaRDa is a human-annotated dataset designed to separate “truth” from “rationality.” It contains 3,000 entailments, including 1,787 valid and 1,213 invalid entailments, built from 6,681 true and 2,319 false statements. By mixing true and false facts and including counterfactual examples, it prevents models from using belief bias or the “content effect” as a shortcut for entailment validity. Its formulation is HARP-like in that the reasoning target is explicitly annotated, but its emphasis is on disentangling factual accuracy from reasoning accuracy rather than on long-form chain-of-thought imitation (Clark et al., 2023).

FindTheFlaws extends the human-annotated reasoning paradigm toward scalable oversight. It comprises five sub-datasets spanning medicine, mathematics, science, coding, and the Lojban language. Each dataset contains questions and long-form solutions with expert annotations validating their correctness or identifying specific errors in the reasoning. The resource is explicitly motivated by critique, debate, and prover-verifier research, where one needs not only correct solutions but also flawed ones with localized error annotations (Recchia et al., 29 Mar 2025).

MHPR is a vision-language benchmark rather than a text-only reasoning corpus, but it is relevant because it organizes data into Captioned Raw Data, Supervised Fine-Tuning Data, Reinforcement Learning Data, and Test Data, and its discussion explicitly points to lessons for HARP. MHPR evaluates joint perception and reasoning over human-centric scenes and introduces ACVG, an automated caption/VQA generation pipeline based on category-wise attribute decomposition, attribute-specific rewriting, and multi-model voting (Wang et al., 5 May 2026).

Taken together, these resources indicate that HARP is not a single annotation recipe. It includes step-by-step rationales, entailment judgments, flaw-localized critiques, and multimodal supervision stacks, provided that the benchmark preserves human-validated reasoning structure in a form usable for model training or evaluation.

5. Annotation, quality control, and answer verification

The high cost of rationale creation is a recurrent theme across HARP-style resources. In the general HARP formulation, generating high-quality rationales at scale is described as time-consuming and expensive, creating a bottleneck for SFT methods that depend on large rationale corpora (Pang et al., 14 Oct 2025).

BaRDa addresses annotation reliability through a four-phase construction process using EntailmentBank, a fine-tuned T5 “Entailer,” GPT-3 prompting, Mechanical Turk annotation, majority vote, and filtering for unanimous agreement. Crowdworkers judge each factual statement as “true” or “false” by real-world standards, and each entailment as “valid” if the hypothesis would reasonably follow assuming the premises were true, otherwise “invalid.” The prompt explicitly reminds annotators that some premises may be counterfactual and that entailment judgments must ignore real-world veracity and focus only on the logical relationship. To maximize label reliability, BaRDa samples only those crowd-labeled items on which annotators unanimously agreed (Clark et al., 2023).

FindTheFlaws uses domain-specific expert review but maintains common principles across all five datasets. Annotators are instructed to identify any step or sentence that is “illogical, untrue, misleading, or otherwise serious error.” Each flawed solution carries metadata specifying the location of the error and a short natural-language description. Correct solutions are verified end-to-end by experts, annotators are encouraged to consult external references, and disagreements are resolved by majority vote in CELS or by discarding unreliable questions in MedQA (Recchia et al., 29 Mar 2025).

The math HARP benchmark emphasizes answer verification infrastructure rather than rationale adjudication. Its checker extends Xwin-LM’s open-source checker with a pipeline that extracts the model’s “Answer:” line via regex, normalizes strings, attempts SymPy parsing of LaTeX expressions, compares exact string match and SymPy-evaluated equivalence, handles tuples, intervals, and mixed fractions specially, and imposes a timeout of 10 seconds per check. The authors also release code and human-annotated test cases for extension (Yue et al., 2024).

MHPR shows how HARP-like annotation can be scaled in multimodal settings through ACVG. In CAAP, captions from Qwen2.5-VL-72B, Intern3-VL-78B, and LLaMA-4 are parsed into a fixed attribute schema, disagreement is identified per attribute, and Qwen3-32B selects the majority or refers to visual cues. The confidence score is

$a_i$6

and the attribute is accepted if $a_i$7. Low-confidence attributes are flagged for manual review. In VAGP, exact three-way agreement yields “easy cases,” two-way agreement yields “weak-consistency,” and unresolved cases go to human review (Wang et al., 5 May 2026).

A plausible synthesis is that HARP curation has moved toward hybrid pipelines: direct human authoring where reasoning precision is paramount, and automatic generation or transformation with expert filtering where scale is required.

6. Evaluation metrics, empirical behavior, and future directions

Evaluation in HARP-style benchmarks is heterogeneous because the targets differ. BaRDa defines factual accuracy over all statements, reasoning accuracy over all entailments, and a self-consistency measure based on the conditional constraint violation $a_i$8:

$a_i$9

$(q,r,a)$0

$(q,r,a)$1

On four GPT-series models, the reported factual accuracy scores are 74.1, 80.6, 82.6, and 87.1, while the reasoning accuracy scores are 63.1, 78.0, 71.8, and 79.2 for GPT-3 (curie), GPT-3 (davinci), GPT-3.5 (turbo), and GPT-4 respectively (Clark et al., 2023).

FindTheFlaws uses precision, recall, and F1, with solution-level “Match” and “Error-grading” tasks, and sentence-level “Match-all” and “Grade-all” tasks for CELS. On Adversarial MedQA Match, the paper reports F1 values of 0.548 for clinicians, 0.592 for GPT-4o, 0.780 for o3-mini, and 0.761 for Claude 3.7, together with 95% confidence intervals. The paper states that on many sub-datasets, top LLMs already exceed individual expert baselines, while domains such as CELS Lojban remain far below human ceiling (Recchia et al., 29 Mar 2025).

The math HARP benchmark evaluates ten state-of-the-art models zero-shot with chain-of-thought prompts. On the 4,780-problem short-answer split, reported accuracies include 75.9% for o1-mini, 68.2% for o1-preview, 58.1% for Gemini 1.5 Pro, 51.3% for Llama 3.1 405B, 47.0% for GPT-4o, and 48.6% for Claude 3.5 Sonnet. On the hardest bracket of 197 problems, the corresponding accuracies are 41.1%, 21.8%, 9.6%, 6.1%, and 5.6% for o1-mini, o1-preview, Gemini 1.5 Pro, GPT-4o, and Claude 3.5 Sonnet. The paper also reports pass@1 of 52.4% and pass@64 of 80.8% for Gemini 1.5 Pro, with majority vote at $(q,r,a)$2 reaching only 66.3% (Yue et al., 2024).

Several forward-looking design lessons are stated directly across the literature. From rationale-efficient training, future HARP datasets should focus on clearly specifying the underlying reasoning pattern $(q,r,a)$3 and collecting minimal seed rationales rather than large rationale corpora (Pang et al., 14 Oct 2025). From BaRDa, clear separation of tasks, inclusion of counterfactuals, multiple annotators with high-agreement sampling, intrinsic self-consistency metrics, and reusable prompt templates are presented as best practices (Clark et al., 2023). From FindTheFlaws, pairing vetted correct solutions with correspondingly constructed flawed solutions and fine-grained error annotations supports critique models, debate, and prover-verifier games (Recchia et al., 29 Mar 2025). From MHPR, category-wise attribute schema, multi-model voting, and challenge-driven RL sets derived from bad-case analysis are described as reusable design components for future HARP extensions (Wang et al., 5 May 2026).

This suggests that HARP is evolving from a narrow notion of human-written chain-of-thought supervision toward a broader benchmark design philosophy: reasoning should be human-validated, structurally explicit, and accompanied by evaluation protocols that separate answer correctness from the quality, consistency, or inspectability of the reasoning process.