Harbor-Index: Agentic Benchmark Evaluation Set

Updated 8 September 2026

The Harbor-Index is a meticulously curated evaluation set comprising 82 difficult and audited tasks designed to assess the performance of agentic language models across seven capability areas, derived from 29 benchmarks.
Harbor-Index employs a multi-stage curation process involving difficulty filtering, AI review, and human audits, ensuring tasks reflect genuine challenges in software engineering, scientific research, and more, with significant emphasis on reproducibility and meaningful evaluation
The evaluation includes a protocol with isolated Docker sandboxes, separate verifier environments, and explicit resource contracts to provide standardized performance assessments, and then robust statistical analysis to identify weaknesses/information drift problems.

Harbor-Index is a curated evaluation set for agentic LLMs, comprising 82 difficult and audited tasks drawn from 29 benchmarks across seven capability areas. It is derived from the broader Harbor Adapters infrastructure, which standardizes benchmark interaction through instructions, executable environments, tests, and reference or oracle solutions. Harbor-Index is intended to retain the breadth and challenge of large-scale agent evaluations while reducing redundancy, cost, runtime, and the risk that low performance reflects defective tasks rather than limitations of the evaluated model or harness (Shi et al., 3 Sep 2026).

1. Origin and evaluation problem

Agentic benchmark evaluation differs from conventional question-answering evaluation because tasks require models to act within executable environments, use tools, manipulate files or services, interact over multiple steps, and satisfy programmatic or judge-based verifiers. A naïve evaluation involving $m$ benchmarks and $n$ agents requires approximately $\mathcal{O}(mn)$ benchmark–agent integrations. Harbor Adapters reduce this integration burden to approximately $\mathcal{O}(m+n)$ by adapting each benchmark once to a common task schema and integrating each agent once with Harbor.

The common representation consists of four elements:

Instruction: the task specification presented to the agent.
Environment: files, tools, services, containers, or other executable context.
Tests: executable or judge-based verification procedures.
Solution: a reference or oracle solution demonstrating feasibility.

The broader adapted suite nevertheless presents several evaluation problems. Tasks may be redundant, saturated, computationally expensive, or affected by heterogeneous scoring procedures. A low success rate may result from an unavailable environment event, an incorrect reference answer, an unstated formatting requirement, a verifier loophole, or a genuine capability limitation. Harbor-Index was created to distinguish difficult tasks from valid, informative difficulty: difficulty arising from reasoning, domain expertise, algorithmic work, long-horizon interaction, or multi-step execution rather than arbitrary implementation details.

The large-scale evaluation motivating Harbor-Index consumed approximately 226 billion input/output tokens and more than $300,000 of compute. Harbor-Index provides a smaller evaluation target suitable for repeated model development, regression testing, harness comparison, and agent research.

2. Composition and domain coverage

Harbor-Index 1.0 contains 82 tasks from 29 benchmarks distributed across seven capability or domain areas.

Capability area	Tasks	Benchmarks
Software Engineering	31	10
Scientific Research	16	7
Agents, Tools & Systems	14	5
Knowledge	9	2
Mathematics & Reasoning	7	2
Data Analytics	3	2
Safety & Security	2	1
Total	82	29

Software engineering contributes the largest share, including GSO, SWE-bench Verified, AlgoTune, FeatureBench, SWE-bench Pro, SWE-Lancer, BigCodeBench, USACO, SWE-smith, and SWT Bench. Scientific research includes BIX-Bench, LAB-Bench, SciCode, SLDBench, ReplicationBench, CodePDE, and QCircuitBench. Agents, tools, and systems include GAIA2, GAIA, Terminal-Bench 2, SkillsBench, and WideSearch.

The index also includes Humanity’s Last Exam and GPQA Diamond in the Knowledge category; ARC-AGI-2 and Omni-Math in Mathematics & Reasoning; Spider 2 and DA-Code in Data Analytics; and CyberGym in Safety & Security. The allocation is intentionally uneven rather than balanced by benchmark count. Selection emphasizes distinctive and informative signal, including specialized capabilities that may not be captured by broad model rankings.

Representative tasks measure repository-level software engineering, numerical and algorithmic optimization, computational biology, scientific programming, PDE solver generation, quantum circuit design, multi-step tool use, command-line interaction, web research, expert knowledge, abstract visual reasoning, mathematical problem solving, text-to-SQL transformation, data analysis, and vulnerability analysis in C/C++.

The index is not a statistically representative sample of all agentic work. It is concentrated in software engineering and scientific research, with only two safety and security tasks and three data-analytics tasks. Multimodal, web, healthcare, legal, and workplace domains are incompletely represented.

3. Construction and curation pipeline

Harbor-Index was produced through a multi-stage funnel involving difficulty filtering, AI review, human review, and an audit-and-fix loop.

Difficulty filtering

Candidate tasks were evaluated using three leading models—Claude Opus 4.6, GPT-5.4, and Gemini 3.1 Pro—under each model’s native harness and Terminus-2. Three trials were run for each model–harness combination, producing 18 trials per task:

$3\ \text{models}\times 2\ \text{harnesses}\times 3\ \text{trials}=18.$

A task passed the difficulty filter when its number of successful trials satisfied

$n_{\mathrm{succ}}\leq 6,$

corresponding to an observed success rate of at most $33\%$ . For most benchmarks, success was defined as reward greater than zero; benchmark-specific pass cutoffs were used for nonbinary scores. This stage reduced the initial pool to 1,311 candidate tasks.

AI quality audit

The 1,311 candidates were reviewed by a Gemini-3-Flash-based auditor. The auditor received the task instruction, configuration, Dockerfile, tests, reference solution, 18 trials, verifier outputs, and selected complete trajectories. It inspected between two and four trajectories per task, including successful, failed, and exceptional cases.

The audit evaluated two principal criteria:

Test–instruction alignment: every verifier assertion had to be traceable to a stated or implied requirement, and every instruction requirement needed corresponding test coverage.
Essential difficulty: task difficulty had to arise from substantive reasoning, algorithmic thinking, domain expertise, long-horizon interaction, or multi-step execution.

Tasks were rejected when difficulty depended primarily on unspecified whitespace, decimal precision, hidden tolerance bounds, magic function names, undisclosed JSON keys, arbitrary output ordering, or fragile formatting requirements.

The AI audit reduced the candidate pool from 1,311 to 307 tasks. A cross-judge comparison on 74 tasks between Gemini-3-Flash and Opus-4.7 produced three-way agreement on 66 tasks, or $89.2\%$ , Cohen’s $\kappa=0.747$ , and binary accept-versus-non-accept agreement on 69 tasks, or $93.2\%$ .

Human audit and senior selection

The 307 survivors were reviewed by 14 domain-experienced human reviewers. Candidates handled by senior reviewers received at least one review; candidates handled by junior reviewers received at least two. Reviewers could accept, reject, flag for repair, or escalate a task.

This stage left more than 110 candidates. A three-member senior panel then selected 100 tasks using difficulty, diversity, quality, and insight into model behavior. Diversity was defined across capability axes, environments, modalities, domains, and failure patterns rather than merely benchmark names.

Audit-and-fix loop

The 100-task intermediate set underwent a final trajectory-grounded audit. At least two senior reviewers examined every task for genuine failures, false positives, false negatives, environment failures, infrastructure defects, and reward-hacking opportunities. Tasks were repaired when possible, rerun with the filtering models, and audited again. Tasks that remained broken or became too easy after repair were removed, yielding the final 82-task Harbor-Index 1.0 release.

This process found that approximately one third of the hardest candidate tasks reaching human review were broken rather than genuinely difficult. Examples included GAIA2 tasks whose required simulation events were never triggered, SWE-bench Pro verifiers requiring undisclosed data-testid strings and log formats, CRUSTBench requiring an unstated Rust interface, an HLE reference answer that conflicted with the mathematically correct answer, and FeatureBench tasks depending on an undisclosed import symbol or module path.

4. Validation protocol and scoring

Harbor-Index 1.0 was evaluated with nine models under two harness conditions, producing

$n$ 0

model–harness–task evaluations.

The closed-weight models were GPT-5.5, Claude Opus 4.8, Gemini 3.1 Pro, and Qwen3.7 Max. The open-weight models were GLM 5.2, Kimi K2.6, MiniMax M3, DeepSeek V4 Pro, and MiMo V2.5 Pro. GPT-5.5, Claude Opus 4.8, and Gemini 3.1 Pro used first-party APIs and native harnesses. The other models were served through OpenRouter and used Claude Code as their native condition. Every model also ran with Terminus-2.

The protocol used isolated Docker sandboxes, separate verifier environments, explicit time and resource contracts, default model settings, tightened audit-stage timeouts, and Harbor version 0.6.4 pinned to commit 331dcba30efcc3fa8282a21de562a3834d6e6244.

Scoring was benchmark-dependent. It included programmatic unit tests, exact-match grading, thresholded continuous metrics, and LLM-as-a-judge evaluation for free-form tasks such as HLE, GAIA, and GPQA Diamond. For continuous tasks, raw rewards were converted into locked pass/fail outcomes. If $n$ 1 denotes the raw reward for trial $n$ 2 on task $n$ 3, the threshold was defined as

$n$ 4

where $n$ 5 denotes flooring to five decimal places over nonfailed observed trials, including oracle calibration runs where available. A trial passed when

$n$ 6

For bounded metrics with a known upper bound, reaching that bound also counted as passing. The aggregate pass rate was the number of passed tasks divided by 82.

Model	Native harness	Native pass	Native cost/run	Terminus-2 pass	Terminus-2 cost/run
GPT-5.5	Codex	28.0%	$n$7155</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> <tr> <td>Claude Opus 4.8</td> <td>Claude Code</td> <td style="text-align: right">20.7%</td> <td style="text-align: right">$n$8293</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> <tr> <td>Gemini 3.1 Pro</td> <td>Gemini CLI</td> <td style="text-align: right">13.4%</td> <td style="text-align: right">$n$989</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> <tr> <td>GLM 5.2</td> <td>Claude Code</td> <td style="text-align: right">8.5%</td> <td style="text-align: right">$\mathcal{O}(mn)$052</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> <tr> <td>Kimi K2.6</td> <td>Claude Code</td> <td style="text-align: right">6.1%</td> <td style="text-align: right">$\mathcal{O}(mn)$133</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> <tr> <td>MiniMax M3</td> <td>Claude Code</td> <td style="text-align: right">3.7%</td> <td style="text-align: right">$\mathcal{O}(mn)$218</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> <tr> <td>Qwen3.7 Max</td> <td>Claude Code</td> <td style="text-align: right">4.9%</td> <td style="text-align: right">$\mathcal{O}(mn)$336</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> <tr> <td>DeepSeek V4 Pro</td> <td>Claude Code</td> <td style="text-align: right"><strong>4.9%</strong></td> <td style="text-align: right">$\mathcal{O}(mn)$435</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> <tr> <td>MiMo V2.5 Pro</td> <td>Claude Code</td> <td style="text-align: right">2.4%</td> <td style="text-align: right">$\mathcal{O}(mn)$54</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> </tbody></table></div> <p>No model–harness configuration exceeded a 30% pass rate. GPT-5.5 with Codex achieved the highest result at 28.0%. The results indicate that native harnesses substantially benefit the strongest closed models, while some open-weight models obtain a more favorable cost–performance tradeoff under Terminus-2. Harness effects are consequential, although model capability remains the dominant factor.</p> <p>The evaluation recorded 453 timeouts with no answer, 361 near-miss solutions, and 445 fundamentally incorrect answers. Lower-scoring models timed out on approximately 36.7% of rollouts, compared with 18.7% for the three highest-scoring models. Native harnesses reduced timeout rates from 42% to 26% in task-matched comparisons. Terminus-2’s bash-only action space, without native image and web tools, contributed to some differences between harness conditions.</p> <p>Per-run costs ranged from approximately $\mathcal{O}(mn)$6293 for Claude Opus 4.8 with Terminus-2. These values depend on provider pricing, token usage, caching, model choice, and harness behavior. 5. Failure analysis and validity Harbor-Index distinguishes model and harness failures from benchmark defects. The broader trajectory analysis identified wrong factual answers, algorithmic bugs, hidden-test regressions, syntax or language errors, silent deliverables, and environment blocks. Across 200 human-audited trajectories, pooled human inter-rater agreement was $\mathcal{O}(mn)$7. A calibrated Gemini 3.1 Pro judge achieved $\mathcal{O}(mn)$8 against human gold labels on a larger calibration set. The task-level failure taxonomy includes: Broken environments: required events never occur or necessary inputs are missing. Instruction–verification mismatch: the verifier checks an unstated requirement. Gameable tests: success depends on guessing an undisclosed symbol, path, or implementation detail. Wrong gold answers: the reference answer is incorrect. Verifier loopholes: agents can exfiltrate answers, tamper with verification, or exploit shared filesystems. Model-level failure patterns included wrong factual answers, algorithmic defects, hidden-test regressions, syntax errors, and incomplete or silent deliverables. Harnesses also affected behavior. Claude Code and Codex enabled more open-ended iteration and self-correction, whereas Terminus-2’s more linear Plan–Execute–Complete structure was more vulnerable when verification and correction were required. Gemini agents sometimes exhibited information drift during external search, replacing a correct initial hypothesis with conflicting information. Harbor-Index therefore treats auditability as part of evaluation validity. A difficult task is not necessarily a useful task. A useful task must have a feasible oracle, aligned instructions and tests, reproducible execution, and difficulty attributable to a meaningful capability demand. 6. Uses, limitations, and maintenance Harbor-Index is designed for repeated evaluation rather than one-time leaderboard ranking. Its compact size makes it suitable for model development, harness comparisons, regression testing, agentic workflow research, and analysis of failure modes. The broader Harbor Adapter infrastructure remains necessary for comprehensive benchmark coverage; Harbor-Index is a curated subset rather than a replacement for domain-specific evaluation. The released artifacts include Harbor benchmark adapters, experiment tools and analyses, large-scale trajectories, Harbor-Index 1.0 source files, a Harbor Hub dataset, and interactive results and trajectories. The broader artifact release contains a manifest of 178,647 benchmark–task–model–agent cells, 793,698 trial records, and approximately 340 GB of trajectory data. A contributor extending Harbor-Index is expected to adapt a benchmark to the instruction/environment/tests/solution schema, provide a reproducible environment, validate the oracle, run original-versus-Harbor parity experiments, document adapter deviations, collect multi-model and multi-harness trajectories, apply difficulty filtering, conduct AI and human audits, execute the audit-and-fix loop, and reevaluate difficulty after repair. Several limitations constrain interpretation: Coverage limitation: 82 tasks across 29 benchmarks cannot represent all agentic capabilities. Selection bias: difficulty was initially measured using three frontier models and six model–harness configurations. Configuration dependence: observed difficulty depends on models, tools, timeouts, sandboxes, verifiers, and harness affordances. Judge dependence: some tasks rely on LLM-as-a-judge evaluation, which introduces variance and systematic bias. Small-sample uncertainty: one task corresponds to approximately $\mathcal{O}(mn)$ 9 percentage points of the aggregate score. Contamination and exploitation: public tasks may be memorized, overfit, or exploited through previously undiscovered verifier weaknesses. Infrastructure cost: although substantially cheaper than the full suite, frontier runs can still cost hundreds of dollars. Temporal instability: model providers, APIs, preview endpoints, harnesses, and dependencies may change. The index is intended to be maintained as a live evaluation resource. Tasks may need replacement when they saturate, become contaminated, lose environmental validity, or reveal new verifier exploits. Its central methodological contribution is not merely the selection of 82 hard tasks, but the combination of adapter standardization, trajectory-grounded auditing, human review, repair, and validity-focused curation. Harbor-Index consequently represents a compact measure of current agentic performance under specified model–harness conditions, rather than a universal or timeless measure of intelligence. Markdown Report Issue Upgrade to Chat References (1) 1. Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation (2026) Topic to Video (Beta) No one has generated a video about this topic yet. Sign Up to Generate All Videos Subscribe on YouTube Whiteboard No one has generated a whiteboard explanation for this topic yet. Sign Up to Generate Follow Topic Get notified by email when new papers are published related to Harbor-Index. Sign Up to Follow Topic by Email Continue Learning What specific criteria were used in the difficulty filtering stage of Harbor-Index? How does Harbor-Index differentiate between tasks that are genuinely difficult and those that are merely flawed or poorly designed? What are the potential impacts for open source and closed weighted models while using different harnesses like Terminus-2? How did Harbor-Index address the issue of verifier loopholes in its evaluation tasks? Find recent papers about curating evaluation tasks for agentic language models. Related Topics Terminal-Bench 2.0: AI Agent Benchmark Terminal-Bench v1.0 & v2.0 Benchmark General AgentBench: Multi-Domain LLM Evaluation Auto-ClawEval: Automated Agent Evaluation ClawGym-Bench: Evaluating Claw Agents AARRI-Bench: Benchmark for Intern Research Behavior TerminalBench (TB): CLI Agent Benchmark Long-Horizon Terminal Bench Inference-Time Harnesses for LLM Agents HarnessDev: Definition, Methods, and Applications Content Overview References Topic to Video Whiteboard Follow Topic Continue Learning Related Topics Stay informed about trending AI papers: About Whiteboards Videos Labs Email Digest Chrome Extension RSS Terms Privacy Contact Twitter Discord YouTube

Model

Native harness

Native pass

Native cost/run

Terminus-2 pass

Terminus-2 cost/run

GPT-5.5

Codex

28.0%

$n$7155</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> <tr> <td>Claude Opus 4.8</td> <td>Claude Code</td> <td style="text-align: right">20.7%</td> <td style="text-align: right">$n$8293</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> <tr> <td>Gemini 3.1 Pro</td> <td>Gemini CLI</td> <td style="text-align: right">13.4%</td> <td style="text-align: right">$n$989</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> <tr> <td>GLM 5.2</td> <td>Claude Code</td> <td style="text-align: right">8.5%</td> <td style="text-align: right">$\mathcal{O}(mn)$052</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> <tr> <td>Kimi K2.6</td> <td>Claude Code</td> <td style="text-align: right">6.1%</td> <td style="text-align: right">$\mathcal{O}(mn)$133</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> <tr> <td>MiniMax M3</td> <td>Claude Code</td> <td style="text-align: right">3.7%</td> <td style="text-align: right">$\mathcal{O}(mn)$218</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> <tr> <td>Qwen3.7 Max</td> <td>Claude Code</td> <td style="text-align: right">4.9%</td> <td style="text-align: right">$\mathcal{O}(mn)$336</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> <tr> <td>DeepSeek V4 Pro</td> <td>Claude Code</td> <td style="text-align: right"><strong>4.9%</strong></td> <td style="text-align: right">$\mathcal{O}(mn)$435</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> <tr> <td>MiMo V2.5 Pro</td> <td>Claude Code</td> <td style="text-align: right">2.4%</td> <td style="text-align: right">$\mathcal{O}(mn)$54</td> <td style="text-align: right"></td> <td style="text-align: right"></td> </tr> </tbody></table></div> <p>No model–harness configuration exceeded a 30% pass rate. GPT-5.5 with Codex achieved the highest result at 28.0%. The results indicate that native harnesses substantially benefit the strongest closed models, while some open-weight models obtain a more favorable cost–performance tradeoff under Terminus-2. Harness effects are consequential, although model capability remains the dominant factor.</p> <p>The evaluation recorded 453 timeouts with no answer, 361 near-miss solutions, and 445 fundamentally incorrect answers. Lower-scoring models timed out on approximately 36.7% of rollouts, compared with 18.7% for the three highest-scoring models. Native harnesses reduced timeout rates from 42% to 26% in task-matched comparisons. Terminus-2’s bash-only action space, without native image and web tools, contributed to some differences between harness conditions.</p> <p>Per-run costs ranged from approximately $\mathcal{O}(mn)$6293 for Claude Opus 4.8 with Terminus-2. These values depend on provider pricing, token usage, caching, model choice, and harness behavior.

5. Failure analysis and validity

Harbor-Index distinguishes model and harness failures from benchmark defects. The broader trajectory analysis identified wrong factual answers, algorithmic bugs, hidden-test regressions, syntax or language errors, silent deliverables, and environment blocks. Across 200 human-audited trajectories, pooled human inter-rater agreement was $\mathcal{O}(mn)$7. A calibrated Gemini 3.1 Pro judge achieved $\mathcal{O}(mn)$8 against human gold labels on a larger calibration set.

The task-level failure taxonomy includes:

Broken environments: required events never occur or necessary inputs are missing.
Instruction–verification mismatch: the verifier checks an unstated requirement.
Gameable tests: success depends on guessing an undisclosed symbol, path, or implementation detail.
Wrong gold answers: the reference answer is incorrect.
Verifier loopholes: agents can exfiltrate answers, tamper with verification, or exploit shared filesystems.

Model-level failure patterns included wrong factual answers, algorithmic defects, hidden-test regressions, syntax errors, and incomplete or silent deliverables. Harnesses also affected behavior. Claude Code and Codex enabled more open-ended iteration and self-correction, whereas Terminus-2’s more linear Plan–Execute–Complete structure was more vulnerable when verification and correction were required. Gemini agents sometimes exhibited information drift during external search, replacing a correct initial hypothesis with conflicting information.

Harbor-Index therefore treats auditability as part of evaluation validity. A difficult task is not necessarily a useful task. A useful task must have a feasible oracle, aligned instructions and tests, reproducible execution, and difficulty attributable to a meaningful capability demand.

6. Uses, limitations, and maintenance

Harbor-Index is designed for repeated evaluation rather than one-time leaderboard ranking. Its compact size makes it suitable for model development, harness comparisons, regression testing, agentic workflow research, and analysis of failure modes. The broader Harbor Adapter infrastructure remains necessary for comprehensive benchmark coverage; Harbor-Index is a curated subset rather than a replacement for domain-specific evaluation.

The released artifacts include Harbor benchmark adapters, experiment tools and analyses, large-scale trajectories, Harbor-Index 1.0 source files, a Harbor Hub dataset, and interactive results and trajectories. The broader artifact release contains a manifest of 178,647 benchmark–task–model–agent cells, 793,698 trial records, and approximately 340 GB of trajectory data.

A contributor extending Harbor-Index is expected to adapt a benchmark to the instruction/environment/tests/solution schema, provide a reproducible environment, validate the oracle, run original-versus-Harbor parity experiments, document adapter deviations, collect multi-model and multi-harness trajectories, apply difficulty filtering, conduct AI and human audits, execute the audit-and-fix loop, and reevaluate difficulty after repair.

Several limitations constrain interpretation:

Coverage limitation: 82 tasks across 29 benchmarks cannot represent all agentic capabilities.
Selection bias: difficulty was initially measured using three frontier models and six model–harness configurations.
Configuration dependence: observed difficulty depends on models, tools, timeouts, sandboxes, verifiers, and harness affordances.
Judge dependence: some tasks rely on LLM-as-a-judge evaluation, which introduces variance and systematic bias.
Small-sample uncertainty: one task corresponds to approximately $\mathcal{O}(mn)$ 9 percentage points of the aggregate score.
Contamination and exploitation: public tasks may be memorized, overfit, or exploited through previously undiscovered verifier weaknesses.
Infrastructure cost: although substantially cheaper than the full suite, frontier runs can still cost hundreds of dollars.
Temporal instability: model providers, APIs, preview endpoints, harnesses, and dependencies may change.

The index is intended to be maintained as a live evaluation resource. Tasks may need replacement when they saturate, become contaminated, lose environmental validity, or reveal new verifier exploits. Its central methodological contribution is not merely the selection of 82 hard tasks, but the combination of adapter standardization, trajectory-grounded auditing, human review, repair, and validity-focused curation. Harbor-Index consequently represents a compact measure of current agentic performance under specified model–harness conditions, rather than a universal or timeless measure of intelligence.

Markdown Report Issue Upgrade to Chat

References (1)

Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation (2026)

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Harbor-Index.