CogARC: Human-Adapted Abstract Reasoning Corpus

• CogARC is a human-adapted subset of ARC that uses few-shot visual tasks to probe abstract rule learning in humans.
• It records high-temporal-resolution data such as example viewing, edit sequences, and multi-attempt submissions to trace the cognitive process.
• The corpus bridges cognitive science and AI by enabling analysis of human learning strategies, error patterns, and process efficiency under sparse supervision.

Searching arXiv for CogARC and related ARC papers to ground the article in current literature. The Cognitive Abstraction and Reasoning Corpus (CogARC) is a human-adapted subset of the Abstraction and Reasoning Corpus (ARC) designed to probe the cognitive processes involved in learning abstract rules from sparse examples and applying them to novel inputs. Whereas ARC was created to benchmark abstract visual reasoning in machines, CogARC extends that framework into a behavioral environment for humans by preserving ARC’s few-shot input–output structure while recording high-temporal-resolution traces of example viewing, edit sequences, and multi-attempt submissions. In the study introducing CogARC, the corpus was administered across two experiments to a total of 260 human participants on 75 abstract visual reasoning problems, with success defined by exact reconstruction of the correct test output from a small number of demonstrations (Ahn et al., 25 Feb 2026).

1. Origins in ARC and the rationale for a human-adapted corpus

CogARC is derived from ARC, a benchmark in which each problem presents a few input–output example pairs illustrating an underlying transformation, and the solver must infer the rule and produce the correct output for a new test input. ARC spans 400 training and 400 evaluation problems, each governed by a distinct rule grounded in core knowledge priors from developmental psychology, including objectness, geometry, number, and goal-directedness. Within this setting, ARC is intended to assess abstract reasoning and generalization from few examples rather than pattern memorization (Ahn et al., 25 Feb 2026).

The motivation for CogARC is methodological as well as substantive. Traditional paradigms such as Raven’s Progressive Matrices provide end-point accuracy but obscure the strategy and process dynamics of abstract reasoning. CogARC was therefore developed to fill this gap by selecting a diverse set of ARC problems, optimizing task and interface features to elicit fine-grained, time-resolved behavior, and collecting detailed logs of example viewing, edit sequences, and multi-attempt submissions. Unlike prior human-ARC studies in which participants solved only a small number of problems, CogARC required each participant to attempt the full problem set within a two-hour window, enabling within-person analyses of learning and strategy over time alongside across-person heterogeneity within individual problems (Ahn et al., 25 Feb 2026).

This design places CogARC at the intersection of psychometrics, cognitive science, and AI evaluation. A plausible implication is that CogARC is not merely a human benchmark layered onto ARC, but a process-level instrumentation of abstract rule inference under sparse supervision.

2. Problem selection, task constraints, and interface design

CogARC comprises 75 problems selected from the ARC training set after manual review for diversity of rule families and minimal overlap so that each problem is novel to the participant. To simplify human interaction, tasks were restricted to those with identical input and output grid sizes. Across the corpus, task properties ranged from 3×3 to 21×21 grids, 2–6 input–output examples per problem, and 1–9 colors excluding black (Ahn et al., 25 Feb 2026).

The problem set was organized along two descriptive axes: core knowledge prior and human-rated complexity. The core knowledge categories were objectness ($n=20$), geometry and pattern ($n=31$), number and counting ($n=15$), and goal-directedness ($n=9$). Complexity was rated independently by three raters on a 1–3 scale, where 1 denoted simpler rules and 2–3 denoted multiple interacting rule components; disagreements were resolved by discussion. The resulting counts were 1 ($n=21$), 2 ($n=38$), and 3 ($n=16$) (Ahn et al., 25 Feb 2026).

Representative tasks included drawing a blue outline around gray tiles, applying a conditional recoloring rule in which connected groups of fewer than three same-colored tiles are recolored to green, rotational pattern completion by adding a missing segment, connecting colored cells with a path, and replacing red cells with purple diagonal paths. These examples illustrate that CogARC preserves ARC’s emphasis on compositional transformations over objects, relations, and counts rather than low-level pixel reproduction (Ahn et al., 25 Feb 2026).

Interface design differed across the two experiments while preserving the open-ended ARC task structure. In both experiments, participants had access to ten colors and could copy the test input into the output editor and reset the grid. In Experiment 1, grid resizing was permitted, and examples, test input, and editable output were displayed simultaneously. In Experiment 2, the interface used separate “example view” and “edit view” screens with default focus on the editor and a button to toggle views. These constraints were chosen to maintain tractable and interpretable human behavior while preserving the inferential demands of ARC (Ahn et al., 25 Feb 2026).

3. Experimental protocol and recorded behavioral signals

Two experiments established the initial CogARC dataset. Experiment 1 was a supervised remote pilot with $n=40$ participants aged 18–35 recruited via a university job board and supervised via encrypted Zoom. All participants solved the same 75 problems in fixed order, with up to three attempts per problem and feedback. Two five-minute breaks occurred after problems 25 and 50, a progress indicator tracked completion, and all actions were logged with millisecond timestamps (Ahn et al., 25 Feb 2026).

Experiment 2 was a large-scale online study with 233 recruited participants, of whom 220 were analyzed. Participants aged 18–35 were recruited via Mechanical Turk and completed demographics. Problems were randomized per participant; up to three attempts with feedback were allowed. Compensation included a $5 base payment and problem-based performance bonuses of$0.10–$0.20 per correct solution. For data quality, participants who solved fewer than 10 problems correctly were excluded ($n=13$$), and trials in which participants submitted without viewing examples were excluded from analysis (<a href="/papers/2602.22408" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Ahn et al., 25 Feb 2026</a>).</p> <p>Behavioral recording focused on high-resolution process measures rather than terminal correctness alone. Logged signals included example-view toggles, edit sequences defined as ordered series of tile edits, timestamps at millisecond resolution, and multi-attempt submissions. A “solution trajectory” was defined as the evolving distance from the current grid state either to the participant’s eventual submitted output in the primary analyses or to the correct solution in supplementary analysis, computed across the ordered edit sequence (<a href="/papers/2602.22408" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Ahn et al., 25 Feb 2026</a>).</p> <p>The principal analytic constructs were accuracy, deliberation, difficulty, similarity, and efficiency. Deliberation time was defined as the latency from trial onset to the first edit action. Difficulty was operationalized behaviorally as a per-problem score from 1 to 4: 1 for success on the first attempt, 2 for success on the second attempt, 3 for success on the third attempt, and 4 for no success. Edit-sequence similarity was computed using the Jaccard index over unordered sets of edits,</p> <p>$$n=31$0

averaged across participant pairs within a problem. Edit trajectory efficiency, termed “normalized extra steps,” quantified redundancy relative to the minimum edits needed to reach the submitted output. Time trends were analyzed via within-participant linear regression of deliberation time and difficulty score on serial trial number,

$n=31$1

The study did not employ entropy of solution strategies, survival or hazard analyses, or logistic or mixed-effects models; the emphasis was on descriptive statistics, correlations, and simple linear regressions (Ahn et al., 25 Feb 2026).

4. Quantitative performance and problem hardness

Human performance in CogARC was generally high but heterogeneous across both problems and participants. In Experiment 1, overall accuracy was mean 89.5% (SD 10.2%) across problems and attempts. First-attempt success was unevenly distributed: 76.0% of tasks were solved on the first attempt by at least 50% of participants, 37.3% by at least 75%, and 8.0% by at least 90%, with median first-attempt success per problem of 69.4%. Mean deliberation time was 22.3 s (SD 13.5 s), and mean per-problem difficulty score was 1.50 (SD 0.55) (Ahn et al., 25 Feb 2026).

In Experiment 2, 55.5% of participants completed at least 70 of the 75 trials, and participants completed 58.2 problems on average, with median 70. Overall accuracy was mean 80.1% (SD 16.6%), median 83.6%, with a range of 13.7%–100%. The first-attempt success distribution remained heterogeneous: 78.7% of tasks were solved by at least 50% of participants on the first attempt, 46.7% by at least 75%, and 17.3% by at least 90%, with median first-attempt success per problem of 73.9%. Deliberation times were positively skewed with a long right tail; the mean was 52.74 s, the median 25.0 s, and the SD 1336.9 s. Mean per-problem difficulty score was 1.62 (SD 0.57) (Ahn et al., 25 Feb 2026).

The two experiments can be summarized as follows.

Experiment	Participants	Selected results
Experiment 1	40	Mean accuracy 89.5%; mean deliberation 22.3 s; mean difficulty 1.50
Experiment 2	220 analyzed	Mean accuracy 80.1%; mean deliberation 52.74 s; mean difficulty 1.62

Hardness was validated against both experimenter ratings and behavioral measures. In Experiment 1, deliberation correlated with difficulty across problems at $n=31$2, $n=31$3, and difficulty increased with experimenter-rated complexity, with significant differences between complexity levels 1 vs 2 and 1 vs 3 after FDR correction. No significant differences were observed across core knowledge categories. In Experiment 2, difficulty correlated strongly with deliberation time at $n=31$4, $n=31$5, and with complexity ratings at $n=31$6, $n=31$7; difficulty correlated negatively with edit-sequence similarity at $n=31$8, $n=31$9, indicating greater strategy divergence on harder problems (Ahn et al., 25 Feb 2026).

Several null results are equally important for interpreting CogARC. In Experiment 2, difficulty showed no significant relationships with normalized extra steps ($n=15$0, $n=15$1), grid size ($n=15$2, $n=15$3), number of colors ($n=15$4, $n=15$5), or the number of edits required to reach the test output ($n=15$6, $n=15$7). The study interprets this pattern as indicating that cognitive rule inference, rather than low-level perceptual or motor demands, was the principal driver of performance differences across problems (Ahn et al., 25 Feb 2026).

5. Solution trajectories, structured errors, and cognitive interpretation

A distinctive feature of CogARC is its process-level account of how solutions emerge. Despite the open-ended output space, errors were often structured and convergent across participants. Common incorrect solutions were defined as identical first-attempt outputs submitted by at least five participants. Such errors varied in prevalence across problems but frequently reflected plausible misgeneralizations supported by the examples rather than random guessing (Ahn et al., 25 Feb 2026).

Trajectory analyses revealed problem-dependent dynamics. For a counting-and-coloring rule in which groups of fewer than three tiles were recolored to green, trajectories were largely short and monotonic, converging efficiently either to the correct solution or to two frequent incorrect solutions. Normalized extra steps clustered near 1 for both successes and errors, although one error exhibited a secondary tail mode consistent with partial exploration followed by convergence on the same incorrect outcome. For a pattern-completion rule requiring addition of a rotationally consistent red segment, trajectories were more heterogeneous, with longer paths and plateaus prior to submission. Three distinct incorrect outputs collectively outnumbered the correct solution, indicating systematic misgeneralizations of the transformation. Path-connecting and path-extension problems showed near-minimal edit sequences for correct solutions with occasional restarts, while common errors were more variable and often longer, though still structured (Ahn et al., 25 Feb 2026).

These observations support several substantive conclusions stated in the CogARC study. Harder problems elicited longer deliberation times and greater divergence in solution strategies. Even when problem-solving trajectories differed in length and smoothness, incorrect solutions were often highly convergent. Some trajectories progressed directly and efficiently toward a stable outcome, whereas others involved extended exploration or partial restarts before converging. The structured nature of errors suggests that participants relied on coherent hypotheses about the transformation, revealing interpretable constraints on generalization and misgeneralization under uncertainty (Ahn et al., 25 Feb 2026).

Time-on-task analyses add a further distinction between task fluency and rule-learning ability. In Experiment 2, participants initiated solutions faster over time, with a significantly negative mean slope for deliberation time versus trial number of $n=15$8 and $n=15$9, $n=9$0, but became slightly less accurate, with a mean difficulty slope of $n=9$1 and $n=9$2, $n=9$3. The correlation between deliberation and difficulty slopes was weak and nonsignificant at $n=9$4, $n=9$5. This was interpreted as consistent with increased familiarity with the interface and task structure, possibly combined with mild fatigue, rather than improved underlying rule-learning capacity (Ahn et al., 25 Feb 2026).

6. Relation to ARC-centered AI research and implications for modeling

CogARC is closely aligned with ARC research in AI because both are built around few-shot abstract visual reasoning grounded in core knowledge priors. The human corpus differs in emphasis: rather than optimizing solver accuracy alone, it captures deliberation, trajectory shape, solution convergence, and the structure of errors. The CogARC study explicitly recommends complementing accuracy with process metrics such as Jaccard similarity of solution strategies, normalized extra steps, trajectory convergence, and per-problem hardness, and recommends models that replicate human error structure rather than success alone (Ahn et al., 25 Feb 2026).

This recommendation interacts directly with several strands of ARC methodology. One line of work frames ARC as a program-synthesis problem over domain-specific languages and attempts to learn or search over symbolic abstractions. Neural-guided bidirectional program search, for example, combines execution-guidance with inverse semantics and learns symbolic abstractions through DreamCoder-style library learning, with the stated aim of supporting compositional reuse and systematic generalization on ARC-style tasks (Alford et al., 2021). Another line translates ARC tasks from the vision domain into the language domain, using an encoder to describe objects, colors, positions, sizes, and symmetries in text, querying zero-shot LLMs for the missing output description, and decoding the resulting description back into a grid. That work argues that natural language can serve as a substrate for cognitive priors and highlights identity tracking and object-centric textual descriptions as useful for ARC and, by extension, for CogARC-like settings (Camposampiero et al., 2023).

More recent reasoning-oriented LLM work remains centered on ARC rather than CogARC as such, but it is relevant because it operationalizes the same core priors. Knowledge Augmentation for Abstract Reasoning (KAAR) organizes priors into hierarchical levels—objectness; geometry/topology and numbers/counting; then goal-directedness—and interleaves stage-wise prior augmentation with repeated-sampling planning-aided code generation. That framework reports consistent gains over a non-augmented repeated-sampling planning-aided code baseline while also emphasizing that ARC remains challenging even for recent reasoning-oriented LLMs (Lei et al., 23 May 2025).

Taken together, these adjacent literatures suggest that CogARC can function as a constraint source for AI models rather than merely as another benchmark. This suggests that the distinctive value of CogARC lies in exposing not only whether a system succeeds, but also whether it deliberates, converges, and misgeneralizes in ways that resemble human abstract reasoning. Within the original CogARC study, this point is explicit: the corpus is presented as a rich behavioral environment for studying how people generalize, misgeneralize, and adapt their strategies under uncertainty, and as a resource for cognitively grounded AI supported by publicly released tasks, action logs, common solutions, and task metadata (Ahn et al., 25 Feb 2026).