Knowledge Dissection Toolkit (KDT)

• KDT is an end-to-end system that converts undocumented process-based simulation workflows into structured KI packages using procedural, evaluative, and diagnostic components.
• The toolkit employs a nine-stage pipeline incorporating source acquisition, wrapper generation, diagnostic recovery, and progressive validation to ensure reproducibility and scalability.
• KDT elevates model operational knowledge by externalizing tacit expertise into machine-readable formats, enabling agent-driven simulation across diverse Earth science domains.

Knowledge Dissection Toolkit (KDT) is an end-to-end system for constructing agent-actionable Knowledge Infrastructure (KI) packages from raw model sources, documentation, and examples for process-based scientific simulation. In the formulation used for agentic Earth science, KI is defined as the triple $KI = (O, P, D),$ where $O$ is a library of validated modelling operators, $P$ is an ordered list of staged domain protocols, and $D$ is a set of diagnostic-recovery triplets mapping failure symptoms to diagnoses and remedies. KDT operationalizes this abstraction by externalizing procedural, evaluative, and diagnostic knowledge that is often tacit in Earth-science modelling workflows, thereby enabling compatible coding agents to execute validated end-to-end simulations through machine-readable scaffolding rather than undocumented specialist practice (Li et al., 18 May 2026).

1. Conceptual definition and motivation

KDT is motivated by a persistent asymmetry in process-based simulation: models in hydrology, agriculture, biogeochemistry, and related Earth-science domains encode decades of mechanistic understanding, but their operational use depends on specialist knowledge that is rarely fully documented. The paper identifies unit conversions, data-format quirks, parameter conventions, and failure diagnostics as examples of this tacit layer. Large-language-model coding agents can generate and debug code, but without embedded procedural, evaluative, and diagnostic knowledge they are reported to exhibit fabrication, silent errors, or infinite debugging loops (Li et al., 18 May 2026).

Within this framework, KI is the central abstraction. The operator set $O = \{O_1, \ldots, O_m\}$ captures procedural knowledge as validated modelling operators. The ordered protocol list $P = [P_1, \ldots, P_n]$ captures evaluative knowledge as staged checks applied during workflow execution. The diagnostic component $D = \{(s_k \rightarrow (\delta_k, r_k))\}$ captures diagnostic knowledge by linking failure symptoms $s_k$ to diagnoses $\delta_k$ and remedies $r_k$. This decomposition is intended to make operational expertise explicit, structured, and executable.

The stated goals of KDT are scalability, reproducibility, generalizability, and accessibility. Scalability refers to autonomous KI generation for hundreds of models across many domains. Reproducibility refers to externalizing operational expertise into machine-readable form so that any compatible agent can re-execute a validated workflow. Generalizability refers to the claim that the same three-layer structure $O$0 suffices across domains. Accessibility refers to lowering the barrier for non-specialists by packaging complex workflows into KI that agents can drive via natural language. A common misunderstanding is to treat KDT as only a code-generation system; in the architecture described, code generation is one component inside a broader apparatus of protocolization, validation, and recovery.

2. Nine-stage construction pipeline

The KDT pipeline comprises nine sequential stages, denoted $O$1 through $O$2, implemented internally with Python scripts, JSON/YAML schemas, and agent-orchestrated code generation. The sequence begins with source acquisition and ends with progressive validation through an ephemeral agent called Tester (Li et al., 18 May 2026).

At $O$3, Source Acquisition, KDT clones or downloads model source code, example inputs and outputs, and documentation files. At $O$4, I/O Pipeline Mapping, it traces the model executable or scripts to identify required input files, variable names, file formats, and expected outputs. At $O$5, Unit Discovery and Conversion, it scans code and documentation for unit annotations, builds a unit-conversion table $O$6, and embeds conversion operators into $O$7.

At $O$8, Tool Generation, KDT synthesizes a Python or shell wrapper $O$9 for each procedural step such as “generate forcing netCDF” or “run model exec.” Each wrapper includes a formal I/O schema, calls to lower-level libraries or binaries, and embedded assertions on output shape and range. At $P$0, SKILL.md Authoring, KDT populates a documentation template that lists every decision point, such as “What soil data source?” or “Which physics flag?”, and links those decisions to the corresponding operators.

At $P$1, Diagnostic-Triplet Construction, KDT defines $P$2 by examining common failure patterns such as unit mismatch, missing file, or implausible output. It authors syntax-independent symptom detectors, for example “no file ./forcing.nc,” and maps them to a diagnosis and a remedy. At $P$3, Package Assembly, KDT collates $P$4, $P$5, $P$6, and SKILL.md into a standard directory structure and serializes metadata in a package.yaml manifest. At $P$7, Binary Test, it executes a minimal run using example inputs to confirm that the operator wrappers invoke the model and produce expected example outputs. At $P$8, Progressive Validation, it runs the full workflow against real-world sites, if observations exist, or synthetics, logging pass/fail status for each $P$9.

This stage decomposition is significant because it separates acquisition, extraction, validation, and recovery into explicit artifacts. A plausible implication is that KDT treats model operationalization not as a single compilation step but as a progressive conversion of undocumented practice into reusable workflow structure.

3. Internal representations and formal workflow semantics

KDT uses explicit machine-readable representations for all three KI layers. Operators $D$0 are JSON objects with fields for name, inputs, code, outputs, and constraints. The inputs and outputs are schema definitions, with examples such as GeoJSON and NetCDF dimensions. The constraints field stores assertions such as $D$1 or “sum of flux = 100%.” Protocols $D$2 are ordered lists of checks, where each check is a Python function $D$3 with a diagnostic message. Diagnostic triplets are stored in a YAML file, diagnostic.yaml, in which each symptom pattern $D$4 is a regular-expression matcher on logs or file states, $D$5 is a plain-language diagnosis, and $D$6 is a call to one or more operators $D$7 to repair the state (Li et al., 18 May 2026).

The operator-extraction procedure begins from a model source directory $D$8, example inputs $D$9, and example outputs $O = \{O_1, \ldots, O_m\}$0. KDT identifies an executable entry point by scanning for “main” or “run” patterns, instruments the entry point with file-I/O hooks, and collects input and output paths together with variable names. For each step $O = \{O_1, \ldots, O_m\}$1 in the instrumented call graph, it derives the operator’s name, input schema, and output schema, then auto-generates wrapper code that validates input schemas, calls $O = \{O_1, \ldots, O_m\}$2 via subprocess or API, and validates output schemas and value ranges.

The staged protocol semantics are equally explicit. Given initial state $O = \{O_1, \ldots, O_m\}$3, the workflow proceeds through stages $O = \{O_1, \ldots, O_m\}$4 according to

$O = \{O_1, \ldots, O_m\}$5

followed by a protocol application at each stage. Each protocol is a conjunction of checks,

$O = \{O_1, \ldots, O_m\}$6

The example check given in the paper is a mass-balance criterion,

$O = \{O_1, \ldots, O_m\}$7

This formalization is important because it makes correctness conditions stage-local rather than deferring all evaluation to terminal outputs.

4. Diagnostic recovery and agent runtime behavior

Recovery is treated as a first-class component rather than an ad hoc debugging strategy. Algorithm 2 takes a failure state $O = \{O_1, \ldots, O_m\}$8 and a stage index $O = \{O_1, \ldots, O_m\}$9, extracts a symptom signature $P = [P_1, \ldots, P_n]$0, and matches that signature against the diagnostic set $P = [P_1, \ldots, P_n]$1. If a match is found, KDT logs the diagnosis, applies the remedy operators, retries the failed stage, and then re-runs the relevant protocol checks. If no match is found, execution aborts with a fatal error. If a retried stage still fails, recovery recurses until the retry count reaches $P = [P_1, \ldots, P_n]$2, after which execution aborts (Li et al., 18 May 2026).

In the agent interface described in the paper, a CLI coding agent receives a single prompt of the form: “Run MODEL_X on Basin Y from $P = [P_1, \ldots, P_n]$3 to $P = [P_1, \ldots, P_n]$4. KI package directory is /ki/MODEL_X.” The agent loads the operators, protocols, and diagnostics from the package directory and executes a loop over the pipeline stages. When checks fail, it invokes the recovery mechanism automatically. On success, it computes performance metrics and writes final outputs.

Runtime decision selection is mediated through SKILL.md. The document enumerates decision points $P = [P_1, \ldots, P_n]$5, and at each $P = [P_1, \ldots, P_n]$6 the agent consults the documentation to choose among options. The paper gives a concrete example: when the decision is “soil data source,” the prompt can direct the agent to pick the default HWSD v1.2 and, if multiple options exist, to choose the one matching the project’s spatial resolution. The chosen option is recorded in decisions.log for audit. Remedies in $P = [P_1, \ldots, P_n]$7 are themselves operators, such as $P = [P_1, \ldots, P_n]$8 or $P = [P_1, \ldots, P_n]$9, so recovery is part of the same operator language as ordinary workflow execution.

A common misconception is that a runnable workflow is equivalent to a validated workflow. In KDT, executability, stage-wise protocol compliance, and domain-level performance thresholds are distinct. Operators may run successfully while protocols fail; recovery may restore executability without restoring physical plausibility; and a complete run may still fall short of success criteria such as Nash–Sutcliffe efficiency.

5. Empirical evaluation across hydrology and multi-domain scaling

The depth test reported for KDT is a 3 000-Trial Coupled-Hydrology Benchmark using VIC 5.1.0 plus Lohmann routing across three Huai River subbasins: Xixian headwaters, Wangjiaba midregion, and Bengbu lower basin. The evaluation used 10 CLI variants from Anthropic, OpenAI, Google, Moonshot, and Alibaba, with 100 independent runs per agent per basin, for 3 000 total trials. Each run executed in a fresh Ubuntu container with HydroCraft data and a timeout of 60 min. The pipeline contained $D = \{(s_k \rightarrow (\delta_k, r_k))\}$0 steps from basin delineation to forcing preparation, VIC execution, routing, and evaluation. A run was defined as successful only if all 14 $D = \{(s_k \rightarrow (\delta_k, r_k))\}$1 passed and Nash–Sutcliffe efficiency satisfied $D = \{(s_k \rightarrow (\delta_k, r_k))\}$2, where

$D = \{(s_k \rightarrow (\delta_k, r_k))\}$3

The reported results partitioned agents into three tiers: Tier 1 $D = \{(s_k \rightarrow (\delta_k, r_k))\}$4 included Claude Sonnet 4.5 (84%), Claude Opus 4.5 (84%), and Kimi K2.5 Coding (80%); Tier 2 $D = \{(s_k \rightarrow (\delta_k, r_k))\}$5 included six agents; Tier 3 $D = \{(s_k \rightarrow (\delta_k, r_k))\}$6 included Qwen3 Coder+ (33%). Agents without KI plateaued below 40% (Li et al., 18 May 2026).

The scale test extended beyond hydrology to 117 additional models across 14 domains. In the expert-supervised transfer set, 25 models were evaluated at three sites each with three agent runs per site, and success was defined by domain-specific thresholds such as Pearson $D = \{(s_k \rightarrow (\delta_k, r_k))\}$7 and $D = \{(s_k \rightarrow (\delta_k, r_k))\}$8. In that setting, 80% (60/75) of model-site combinations passed. In the fully autonomous scaling set, 92 models were dissected end-to-end by KDT with no human intervention; 59 were validated against observations, and 33 were verified runnable on synthetic or example inputs. All 92 packages produced a runnable workflow.

The paper also reports structural convergence across the KI corpus. Across all 119 KIs, modelling decisions and failure remedies converged despite different underlying physics. Specifically, 2 406 diagnostic triplets showed that I/O format and unit errors accounted for 55% of failures in every domain, with median Spearman $D = \{(s_k \rightarrow (\delta_k, r_k))\}$9. Across 3 478 decision points, three universal categories—parameter selection, physics options, and unit system—covered 55% of all decisions. This suggests that operational expertise is structured and extractable rather than ad hoc.

6. Scientific significance, scope, and future directions

KDT is presented as a means of converting process-based modelling workflows into standardized, machine-readable interfaces. In this formulation, externalizing operational expertise into $s_k$0 enables coding agents to operate complex simulation pipelines more reliably, raising success rates from below 40% to above 80% in the reported hydrology benchmark. The framework also scales from a single hand-built workflow to 117 additional models with decreasing human supervision, while the observed convergence of failure modes and decision structures across 14 domains is used to support the domain-invariant character of the KI abstraction (Li et al., 18 May 2026).

Its significance is twofold. First, it lowers the access barrier between non-specialist users and process-based simulation by packaging complex workflows into KI that agents can drive via natural language. Second, it lowers the integration barrier between modelling communities by making model operation legible as interoperable operators, protocols, and diagnostics rather than as idiosyncratic lab practice. The abstract further characterizes this as a path toward a “living scientific commons,” in which process-based science becomes answerable to whoever needs to know and extendable by whoever can contribute.

The future directions named in the paper are community-driven extension of KI repositories, deeper calibration routines, and application beyond Earth sciences. These directions do not alter the existing definition of KDT, but they indicate that the toolkit is intended not only for one-off model packaging but also for ongoing maintenance and broadening of the KI paradigm. A plausible implication is that KDT reframes model usability as an infrastructure problem: not merely whether a model exists, but whether its operational knowledge has been sufficiently dissected, validated, and serialized for agent execution.