InAbHyD: Inductive & Abductive Reasoning

• InAbHyD is a synthetic dataset designed for evaluating large language models’ ability to perform inductive and abductive reasoning under parsimony constraints.
• It structures reasoning through incomplete world models, observations, and hidden axioms to test the models’ explanation compactness and coverage.
• The benchmark operationalizes Occam’s Razor by favoring minimal, high-coverage hypotheses that efficiently explain observed data.

Searching arXiv for the InAbHyD benchmark and closely related reasoning benchmarks to ground the article in current literature. InAbHyD is a programmable, synthetic dataset for evaluating whether LLMs can perform inductive reasoning, abductive reasoning, and combinations of the two while following Occam’s Razor. The name stands for Inductive and Abductive Hypothesis Discovery, it is pronounced “in-a-bid,” and it is introduced in “LLMs Do Not Follow Occam’s Razor: A Benchmark for Inductive and Abductive Reasoning” (Sun et al., 3 Sep 2025). InAbHyD frames reasoning as hypothesis generation from an incomplete world model and a set of observations: the system must propose missing axioms that, together with the incomplete theory, explain the observations. Its distinctive contribution is not only to test whether hypotheses explain the data, but also to quantify whether they do so parsimoniously (Sun et al., 3 Sep 2025).

1. Conceptual orientation

InAbHyD is designed around the claim that most reasoning benchmarks for LLMs emphasize deductive reasoning, whereas inductive and abductive reasoning are also essential in solving real-world problems and are less explored (Sun et al., 3 Sep 2025). Within the benchmark’s framing, induction is the inference of general rules from instances, and abduction is the inference of plausible explanations for observations. Both are treated as forms of inverse reasoning: the task begins from effects or observations and asks for causes, rules, or class memberships that would explain them.

A central design principle is the explicit incorporation of parsimony. The benchmark operationalizes Occam’s Razor as a preference for hypothesis sets that explain many observations with few, general hypotheses. This distinguishes InAbHyD from benchmarks that evaluate only whether an answer is logically correct. The benchmark therefore measures both explanatory adequacy and the compactness of the explanatory theory (Sun et al., 3 Sep 2025).

The paper’s title encodes its main empirical conclusion: LLMs can often produce hypotheses that technically fit the data, but they do not systematically choose the simplest high-coverage explanations. This suggests that current gains in deductive reasoning do not transfer straightforwardly to inverse theory construction under a parsimony constraint (Sun et al., 3 Sep 2025).

2. Formal task and logical representation

Each reasoning example consists of an incomplete world model, a set of observations, and a required output: a set of hypotheses that fill in missing axioms so that the world model plus hypotheses explain the observations (Sun et al., 3 Sep 2025). The world model is presented as a first-order logic ontology tree whose nodes are concepts and whose edges are subtype relations. Concepts can have properties and members, and some axioms are hidden and removed from the model.

Formally, the theory $\mathcal T$ contains three axiom types:

• property axioms, such as $\forall x(o(x) \rightarrow p(x))$,
• membership axioms, such as $o(A)$,
• subtype axioms, such as $\forall x(o_1(x) \rightarrow o_2(x))$.

A subset of these axioms is randomly marked as hidden. These hidden axioms constitute the ground-truth hypotheses $\mathcal H^*$. The observations $\mathcal O$ are ground FOL facts chosen so that the hidden axiom is necessary, or at least uniquely natural, to explain them (Sun et al., 3 Sep 2025).

A reasoning instance is therefore the pair

$(\mathcal T_{\text{incomplete}}, \mathcal O).$

The model receives a natural-language rendering of the incomplete theory and observations and is instructed to produce hypotheses to explain all observations. The intended logical condition is that $\mathcal T \cup \mathcal H$ entails all observations in $\mathcal O$, with proof trees $\mathcal P$ pre-computed by the data generator for evaluation (Sun et al., 3 Sep 2025).

This formulation unifies inductive and abductive reasoning under a single “hypothesize-to-explain” paradigm. General rules such as $\forall x(o(x) \rightarrow p(x))$0 or $\forall x(o(x) \rightarrow p(x))$1 instantiate induction, while ground facts such as $\forall x(o(x) \rightarrow p(x))$2 instantiate abduction. In more complex cases, models must compose both forms of reasoning within a single hypothesis set (Sun et al., 3 Sep 2025).

3. Dataset construction and controllable difficulty

The dataset is fully synthetic and programmable. Its generation process begins with the creation of an ontology tree of height $\forall x(o(x) \rightarrow p(x))$3, where each non-leaf node has 2 or 3 children. Height is a central difficulty parameter because world-model size grows roughly exponentially with $\forall x(o(x) \rightarrow p(x))$4 (Sun et al., 3 Sep 2025).

The nodes are then filled in level-order traversal. Each node receives a fictional concept name, three properties, and three members. The use of fictional concept names is intended to avoid contamination from real-world knowledge, while real person names and simple property words preserve naturalness in the language rendering. A second pass randomly selects some properties, membership statements, or subtype relations to hide; these hidden axioms become the ground-truth hypotheses $\forall x(o(x) \rightarrow p(x))$5 (Sun et al., 3 Sep 2025).

For each hidden axiom, the generator constructs observations that make the hidden statement needed to explain the data. Natural-language translation uses the grammar from PrOntoQA, with variants such as “Each ragdoll is a cat,” “Every ragdoll is a cat,” or “All ragdolls are cats” for subtype statements (Sun et al., 3 Sep 2025).

The benchmark supports both single-hypothesis and multi-hypothesis settings. In the multi-hypothesis case, the dataset statistics reported for ontology heights 1 through 4 are as follows (Sun et al., 3 Sep 2025):

Height	Avg. # world-model axioms	Avg. # observations	Avg. # ground-truth hypotheses
1	9.0	10.0	3.0
2	14.0	11.8	3.5
3	25.5	15.2	4.6
4	46.8	20.0	6.6

The benchmark emphasizes that hypotheses and observations grow roughly linearly with height, whereas ontology axioms grow exponentially. This suggests that rising difficulty is driven not only by the number of hidden hypotheses, but also by the expanding structural search space through which the model must navigate (Sun et al., 3 Sep 2025).

4. Core subtasks and reasoning regimes

InAbHyD defines three core subtasks, each corresponding to a type of hidden axiom and a characteristic reasoning mode (Sun et al., 3 Sep 2025).

Infer property (inductive) hides a universal property axiom such as

$\forall x(o(x) \rightarrow p(x))$6

The world model includes membership facts like $\forall x(o(x) \rightarrow p(x))$7, and the observations are $\forall x(o(x) \rightarrow p(x))$8. The intended hypothesis is a general law of the form “All $\forall x(o(x) \rightarrow p(x))$9s have property $o(A)$0.” This is induction from repeated instances to a shared rule.

Infer membership relation (abductive) hides a membership axiom such as $o(A)$1. The world model provides universal implications from $o(A)$2 to several properties, and the observations assert that $o(A)$3 has those properties. The intended hypothesis is “$o(A)$4 is an $o(A)$5.” This is abduction because a latent class membership is proposed to explain why one individual exhibits a collection of features.

Infer subtype relation (inductive + abductive) hides a subtype axiom

$o(A)$6

The world model contains membership facts $o(A)$7, and the observations are $o(A)$8. The intended hypothesis is “All $o(A)$9s are $\forall x(o_1(x) \rightarrow o_2(x))$0s.” The benchmark characterizes this as combining induction over multiple instances with abduction of a missing concept-to-concept relation (Sun et al., 3 Sep 2025).

The benchmark does not sharply separate induction from abduction. Instead, subtasks are labeled as inductive, abductive, or both, and multi-hypothesis settings require composition across them. A good hypothesis set must satisfy three conditions: coverage of all observations, syntactic validity and consistency with the theory, and parsimony in the number and generality of the hypotheses (Sun et al., 3 Sep 2025).

5. Occam’s Razor metric

The benchmark’s principal methodological innovation is its Occam’s Razor-based quality metric. Let $\forall x(o_1(x) \rightarrow o_2(x))$1 denote the incomplete world model, $\forall x(o_1(x) \rightarrow o_2(x))$2 the observations, $\forall x(o_1(x) \rightarrow o_2(x))$3 the proof trees for observations under (\mathcal T \cup \mathcal