KAT-Coder-V2.5: Executable Coding Agent

• KAT-Coder-V2.5 is a coding-focused agentic model built to operate within executable repositories, emphasizing verifiable, reproducible tasks over mere pass rates.
• It integrates the AutoBuilder system to reconstruct multilingual repositories into sandboxed, verifiable environments, boosting success rates from 16.5% to 57.2%.
• The model employs reinforcement learning with sandbox stabilization and Multi-Teacher On-Policy Distillation to unify expert domains and optimize tool-use synthesis.

KAT-Coder-V2.5 is a coding-focused agentic model designed to operate inside executable software repositories and interactive tool environments rather than as a single-turn code generator. Its technical report defines the central bottleneck not primarily as model scale, but as the scarcity of reproducible environments, verifiable rewards, and high-value trajectories; accordingly, the system is built around an end-to-end agentic post-training stack comprising AutoBuilder for repository reconstruction, KwaiClawEnv for large-scale tool-use synthesis, reinforcement learning in reliability-hardened sandboxes, and Multi-Teacher On-Policy Distillation for expert unification (Huang et al., 6 Jul 2026).

1. Lineage and conceptual framing

KAT-Coder-V2.5 belongs to a sequence of KAT-Coder reports that progressively move from text-centric coding assistance toward executable, environment-grounded agents. The earlier "KAT-Coder Technical Report" describes a four-stage curriculum of Mid-Term Training, Supervised Fine-Tuning, Reinforcement Fine-Tuning, and Reinforcement-to-Deployment Adaptation, with explicit emphasis on IDE traces, Error-Masked SFT, and Tree-Structured Trajectory Training (Zhan et al., 21 Oct 2025). The subsequent "KAT-Coder-V2 Technical Report" shifts to a "Specialize-then-Unify" paradigm across five expert domains—SWE, WebCoding, Terminal, WebSearch, and General—combined through on-policy distillation and supported by KwaiEnv, MCLA, and Tree Training (Li et al., 29 Mar 2026).

Within that progression, V2.5 is distinguished by its insistence that software-engineering samples should be executable and verifiable. The report treats each SWE sample as a task triplet consisting of a clear task description, an executable repository environment, and a validation suite with fail-to-pass and pass-to-pass tests. It further states that training should emphasize process-aware behavior—exploration, localization, specification fidelity, repository conventions, minimal edits, and robust verification—rather than only final pass rates. This suggests a deliberate reorientation from static issue-text supervision toward repository-grounded agent execution.

The report does not specify model scale or compute budget. That omission is itself informative: the paper’s emphasis falls on infrastructure, data construction, reward design, and post-training methodology rather than on parameter-count disclosures.

2. AutoBuilder and verifiable repository tasks

AutoBuilder is the repository-side engine of KAT-Coder-V2.5. Its function is to reconstruct multilingual repositories into sandboxed environments and to turn repository states into verifiable software-engineering tasks. The core workflow is a build–verification loop with two agents: a build agent that generates installation and test scripts, and a verification agent that executes them in an isolated sandbox. Acceptance requires structured test outputs showing that more than 90% of expected tests are collected and that pass/fail outcomes are reproducible across runs. To make this scale practical, the system uses preconfigured base environments, language and build-system templates, and a retrievable library of successful build recipes. The reported effect is an increase in environment success rate from 16.5% to 57.2%, yielding more than 100,000 verifiable environments across 12 languages (Huang et al., 6 Jul 2026).

The same pipeline enforces hygiene constraints intended to preserve task realism while preventing leakage. Dependency updates and environment edits from the reference change are applied when they are not part of the programming challenge, and git history and commit metadata are removed to prevent solution leakage. From golden and test patches, the system regenerates self-contained task specifications containing a problem statement, requirements, and interface constraints; clarity checks remove ambiguous or inconsistent samples.

A second component is trajectory recovery. The report describes a two-stage hint-in-the-loop procedure in which process-level hints are injected to improve previously zero-pass tasks to approximately 20% pass rate. After the verified patch is fixed, a hint-free trajectory is regenerated from the original context, and only trajectories that pass verification and contain no hint leakage are retained. This converts failed-but-informative rollouts into usable supervision.

A further distinguishing element is process-aware filtering. Rule-based gates remove invalid, unstable, or exploitative trajectories, including hard-coding, test tampering, and mechanism bypassing. Heuristic process scoring then evaluates exploration, localization, pre-edit reasoning, specification fidelity, repository conventions, patch minimality, verification quality, recovery behavior, and honesty. A common misconception in repository-level coding is that a passing patch is sufficient evidence of competence; the V2.5 pipeline explicitly rejects that assumption by down-weighting or removing shortcut-based successes.

Harness rewriting extends this robustness goal. Tool names, argument conventions, output formats, and prompt templates are randomized, while perturbations such as missing dependencies, transient failures, truncated outputs, and noisy logs are injected. Because verification is anchored to structured test outcomes, the objective is to force harness-invariant strategies rather than harness-specific prompt adaptation.

3. KwaiClawEnv and large-scale tool-use synthesis

KwaiClawEnv is the general agentic side of the training stack. It is organized as a three-layer Service–Task–Eval architecture intended to synthesize large-scale tool-use trajectories from executable services and real task seeds. The Service layer constructs capabilities, the Task layer derives tasks and samples trajectories, and the Eval layer performs processing and quality control; the report describes this as a closed-loop system in which feedback improves later generations (Huang et al., 6 Jul 2026).

Service construction comes from two sources. One is human-authored Skill definitions, exemplified by OpenClaw, which are converted into deployable services with OpenAPI specifications, container configurations, and fixture data; the reported success rate for this conversion is greater than 90%. The other is LLM-generated services guided by categories and service chaining, which are used to build composite environments. The result is a unified modeling interface over heterogeneous environments, tools, and workflows.

Task construction begins from real business seeds and applies parameter expansion, constraint augmentation, and tool-chain orchestration. The scale figures are explicit: millions of candidate tasks are generated, and more than 100,000 high-quality instances are retained after multi-stage verification. The associated trajectories average approximately 15 tool calls, and the longest exceed 100 steps. Coverage is reported across coding, writing, and data-analysis scenarios.

Quality control is multi-layered. End-to-end consistency validation checks endpoint reachability, OpenAPI completeness, dependency connectivity, schema legality, tool-reference correctness, parameter consistency, machine-verifiable success criteria, service startup, fixture loading, interaction, trajectory integrity, and scoring correctness. Samples are labeled as repairable failure, rejected defect, or production-ready. A second filtering stage combines hard-rule checks for unsafe, invalid, or hallucinated behavior with LLM-as-Judge assessments of semantic correctness, execution efficiency, and interaction naturalness. Final trajectories are converted into a unified SFT-ready format.

In encyclopedia terms, KwaiClawEnv is not merely a data generator. It is a synthetic environment-construction system that attempts to make heterogeneous, service-mediated workflows trainable under standardized interfaces and verifiers.

4. Reinforcement learning, sandbox reliability, and reward design

The reinforcement-learning stack of KAT-Coder-V2.5 is built around harness diversity, infrastructure hardening, and explicit long-horizon credit assignment. Harness randomization is applied along three axes: tool-invocation protocol, context management, and control flow. The reported harnesses range from a white-box mini-swe-agent with simple control flow and low-noise signals to black-box harnesses such as ClaudeCode, Codex, OpenClaw, and OpenHands, which are closer to deployment distributions and may compress or reorganize context (Huang et al., 6 Jul 2026).

Reliability engineering is treated as a prerequisite for valid RL. The Gateway Server decouples rollout from training, mediates all traffic, writes trajectories to an Experience Buffer, and enforces token consistency by using the /generate endpoint rather than chat endpoints. The report states that this eliminates retokenization drift observed in approximately 40% of samples at approximately 200-turn scale. Sandbox stabilization further redesigns image management and early-release policy, reducing peak disk usage from approximately 95% to approximately 60% and timeout-induced invalid rollouts from approximately 6–7% to less than 1%. Fixes to environment-variable overrides reduce corrupted verifier outputs from approximately 6–7% to less than 1%. Overall sandbox feedback error rate falls from approximately 16% to less than 2%, and training collapses decrease by approximately 10x.

The policy objective is a clipped PPO formulation with per-token importance ratios:

$$\mathcal{J}_{\mathrm{PPO}}(\theta) = \mathbb{E}_{q\sim P(Q),\ o\sim\pi'(\cdot\mid q)} \left[\frac{1}{|o|}\sum_{t=1}^{|o|} \min\!\left( r_t\,\hat{A}_t,\, \mathrm{clip}\!\left(r_t,1-\epsilon,1+\epsilon\right)\hat{A}_t \right)\right], \quad r_t=\frac{\pi_\theta(a_t\mid s_t)}{\pi'(a_t\mid s_t)}.$$

Advantage estimation uses GAE:

$$\hat{A}_t = \sum_{l=0}^{T-t-1}(\gamma\lambda)^l\delta_{t+l}, \quad \delta_t = r_t + \gamma V'(s_{t+1}) - V'(s_t).$$

The critic is asymmetric and hindsight-augmented. Its privileged context $c_t$ may include final pass/fail reward, unit-test outcomes, coverage signals, patch diffs, task metadata, trajectory statistics, and subsequent turns:

$$\mathcal{L}_{\mathrm{critic}^{\mathrm{asym}}}(\psi) = \mathbb{E}_{(s_t,c_t,R_t)\sim\mathcal{D}} \left[\bigl(V(s_t,c_t;\psi)-R_t\bigr)^2\right], \quad R_t=\hat{A}_t+V'(s_t,c_t).$$

The actor observes only normal harness state during rollout, while the critic and hindsight context are discarded at inference.

Rewarding is split between rule-based and model-based components. Rule-based rewards require passing all fail_to_pass and pass_to_pass tests for full core-task score, and they add penalties for repetition, garbled content, tool invocation errors, invalid tool-call placement, redundant intra-turn tool calls, excessive parallel tool calls, and debug artifact cleanup. Dense incentives are also provided for file-search accuracy and partial unit-test pass rates. The model-based component is a rubric-guided GRM judge trained with:

$$r = \begin{cases} \displaystyle \frac{|GT \cap Pred|}{|GT|} - |Pred \setminus GT| \cdot \lambda, & GT \neq \emptyset \ \displaystyle 1 - |Pred \setminus GT| \cdot \lambda, & GT = \emptyset \end{cases}$$

This reward prioritizes recall while penalizing false predictions.

5. Multi-expert organization and Multi-Teacher On-Policy Distillation

KAT-Coder-V2.5 is not presented as a monolithic policy learned from a single mixed corpus. Instead, the report describes five domain experts—agentic SWE, agentic tool use or Claw, terminal use, web coding, and general knowledge—which are unified through Multi-Teacher On-Policy Distillation (MOPD). In this respect, V2.5 remains close to the multi-expert logic of KAT-Coder-V2, although the V2.5 report makes the function-space fusion objective explicit and centers it on reverse KL rather than on the earlier OPD description (Li et al., 29 Mar 2026).

The MOPD objective is:

$$\mathcal{L}_{\mathrm{MOPD}}(\theta) = \mathbb{E}_{(x,d)\sim \mathcal{D}} \mathbb{E}_{y\sim\pi_\theta(\cdot\mid x)} \Bigg[ \sum_{t=1}^{|y|} w_t\, \mathrm{KL}\!\left( \pi_\theta(\cdot\mid x,y_{<t}) \,\middle\|\, \pi_{T_d}(\cdot\mid x,y_{<t}) \right) \Bigg].$$

The teacher $\pi_{T_d}$ is domain-specific, and the report characterizes reverse KL as mode-seeking and as reducing cross-domain interference (Huang et al., 6 Jul 2026).

Two stabilization devices are added for long-context distillation. The first is an off-policy cold start:

$$\mathcal{L}_{\mathrm{cold}}(\theta) = \mathbb{E}_{(x,d)\sim \mathcal{D}} \mathbb{E}_{y\sim\pi_{T_d}(\cdot\mid x)} \left[ -\sum_{t=1}^{|y|} \log \pi_\theta(y_t\mid x,y_{<t}) \right].$$

The second is drift-aware dynamic truncation based on top-$k$ overlap:

$$\rho_t = \frac{|\mathcal{T}_t^k \cap \mathcal{S}_t^k|}{k}, \qquad \rho_t\in[0,1].$$

The weight $$\hat{A}_t = \sum_{l=0}^{T-t-1}(\gamma\lambda)^l\delta_{t+l}, \quad \delta_t = r_t + \gamma V'(s_{t+1}) - V'(s_t).$$0 is a monotonic function of $$\hat{A}_t = \sum_{l=0}^{T-t-1}(\gamma\lambda)^l\delta_{t+l}, \quad \delta_t = r_t + \gamma V'(s_{t+1}) - V'(s_t).$$1, and if $$\hat{A}_t = \sum_{l=0}^{T-t-1}(\gamma\lambda)^l\delta_{t+l}, \quad \delta_t = r_t + \gamma V'(s_{t+1}) - V'(s_t).$$2 remains below a threshold for $$\hat{A}_t = \sum_{l=0}^{T-t-1}(\gamma\lambda)^l\delta_{t+l}, \quad \delta_t = r_t + \gamma V'(s_{t+1}) - V'(s_t).$$3 consecutive tokens, backpropagation is truncated on subsequent tokens. Length-stratified batching is used to preserve long-context proportions.

A plausible implication is that V2.5 treats multi-domain fusion as a stability problem as much as a capability problem. The report explicitly argues that MOPD avoids the see-saw degradation associated with weight-space merging.

6. Benchmarks, limitations, and research significance

Evaluation is conducted under a unified Claude Code harness with fixed tool set, context budget, execution environment, and decoding configuration. The benchmark suite spans repository-level software engineering, agentic tool use, and the AA Coding Index. Reported results are as follows (Huang et al., 6 Jul 2026).

Benchmark	KAT-Coder-V2.5	Position
SWE-Bench Pro	65.2	Second
KAT Code Bench	53.1	Second
PinchBench (Avg)	94.9	Top
KAT Claw Bench	85.5	Below Opus 4.8 and GLM-5.2
Terminal-Bench 2.1	60.7	Below GLM-5.2 and Opus 4.8
SciCode	50.3	Near GLM-5.2

These scores support the paper’s summary claim that KAT-Coder-V2.5 delivers the best agentic tool-use result on PinchBench and ranks second only to Opus 4.8 on repository-level software engineering. The result profile is uneven in a diagnostically useful way. Repository-level SWE and tool-use performance are the strongest areas; terminal and scientific coding remain weaker relative to larger general-purpose models. This pattern is consistent with the report’s own account of V2.5 as an infrastructure- and environment-centered system rather than as a scale-maximized generalist.

The ablations and analyses attribute concrete gains to specific pipeline components. AutoBuilder raises environment-construction success rate from 16.5% to 57.2%; near-miss recovery raises previously zero-pass tasks to approximately 20% before hint-free regeneration; KwaiClawEnv contributes more than 100,000 high-quality tool-use trajectories; hindsight-augmented value estimation reduces variance and improves sample efficiency; the combined reward framework improves pass rates, reduces token consumption, and standardizes behavior; harness randomization improves cross-harness generalization; and MOPD avoids see-saw degradation typical of weight-space merging.

The report is also explicit about remaining constraints. Environment coverage and tool availability are bounded, some terminal and scientific tasks favor larger general-purpose models, and long-horizon credit assignment remains an open challenge. Earlier environment fragility caused reward corruption, and black-box trajectory compression can make supervision noisy. Planned directions are broader verifiable environment construction, improved RL credit assignment and stability, and expanded tool and ecosystem coverage.

As a research artifact, KAT-Coder-V2.5 is significant less for disclosed backbone architecture than for its claim that strong coding agents depend on executable environments, verifier quality, and trajectory curation. That emphasis places it within a broader shift from benchmark-oriented code generation toward agentic post-training over real repositories and tool-mediated workflows. The public service is listed at https://streamlake.com/product/kat-coder.