CodeAct: Executable Code as Action
- CodeAct is a paradigm that treats code as the action language, merging code generation with execution feedback for dynamic reasoning.
- It integrates memory retrieval, code-driven planning, and persistent interpreter feedback to support flexible multi-turn interactions.
- Empirical studies show its strength in complex reasoning tasks while highlighting task-dependent performance and the need for domain-specific adaptations.
CodeAct denotes a code-as-action paradigm for language agents in which actions are expressed as executable code, typically Python, and the agent continues reasoning over execution outputs and errors rather than over text alone. In later work, the term also refers to concrete implementations such as OpenHands CodeAct, and it frequently appears as a host framework, evaluation condition, or baseline rather than as the newly proposed method in a paper. A parallel early use of the name in ad hoc teamwork emphasized enhanced memory plus code-driven reasoning for teammate inference, but the most widely reused formulation is the executable-action agent interface introduced for multi-turn interaction with environments (Shi et al., 2023, Wang et al., 2024).
1. Origins and core conception
An early application of the name appeared in the Avalon ad hoc teamwork study, where CodeAct was introduced as “a general agent that equips LLM with enhanced memory and code-driven reasoning, enabling the repurposing of partial information for rapid adaptation to new teammates.” Its architecture combined memory retrieval, code-driven reasoning with action, and code execution with self-debugging, and it was evaluated through team-selection quality in a social-deduction environment (Shi et al., 2023).
The more general formulation was established in “Executable Code Actions Elicit Better LLM Agents,” which defined CodeAct as the use of executable Python code to consolidate agent-environment actions into a unified action space. In that formulation, an agent receives an observation from a user or environment, may optionally plan, emits Python code as the action, and then treats the execution result or error as the next observation. The paper contrasted this with JSON and plain-text action formats, arguing that code natively supports control flow, data flow, tool composition, existing software packages, and automated feedback through interpreter errors (Wang et al., 2024).
This yields a distinctive view of code. Code is not merely an output format, nor only a final artifact; it is the action language itself. Later papers repeatedly preserve that interpretation, even when the concrete agent is OpenHands CodeAct, a planning baseline, or a domain-specific execution loop. Across these uses, the recurring idea is that code functions simultaneously as an action representation, a reasoning substrate, and a structured interface to tools and intermediate state (Wang et al., 2024).
2. Execution model and runtime semantics
CodeAct systems are typically multi-turn. A particularly explicit instantiation appears in “Code-enabled LLMs can outperform reasoning models on diverse tasks,” where CodeAct consists of a language module and a code module: the model emits messages containing natural-language reasoning plus Python code, a persistent Python workspace executes the code, and the workspace returns outputs, errors, and remaining-budget information. The interface uses <turn>, <code>, and <return> tags, variables persist across code cells, and stopping is governed by budgets such as 16k output tokens, 4 minutes total compute time, and 10 reasoning turns, plus an additional final-guess turn if no answer has yet been returned (Zhang et al., 23 Oct 2025).
In benchmarked software-agent settings, CodeAct often appears through OpenHands. ScienceAgentBench evaluated OpenHands CodeActAgent v1.9 as an interactive agent framework equipped with a Python interpreter, bash shell, and web browser, with actions unified into a large space of Python API calls. The agent is expected to write and run a self-contained Python program file in a workspace, inspect failures, and revise accordingly. This presentation makes clear that CodeAct is not equivalent to one-shot code generation; it is an interactive execution regime with file editing, environment interaction, and iterative repair (Chen et al., 2024).
A further technical refinement comes from “Agents Learn Their Runtime: Interpreter Persistence as Training-Time Semantics,” which treats the execution contract itself as learnable. That paper formalizes execution as
and isolates whether interpreter state persists across turns. Its 2×2 train/runtime study finds that solution quality is statistically indistinguishable across persistence conditions, but behavior differs sharply: a persistent-trained model in a stateless runtime triggers unresolved-reference failures in roughly 80% of episodes, while a stateless-trained model in a persistent runtime redundantly re-derives retained state and uses roughly 3.5× more tokens. This establishes interpreter persistence as part of the semantics of CodeAct traces rather than a mere implementation detail (May et al., 1 Mar 2026).
3. CodeAct as framework, host, and baseline
In subsequent literature, CodeAct is commonly reused in one of three roles: as an off-the-shelf framework, as a baseline to be surpassed, or as a host scaffold into which another method is inserted.
| Paper | Role of CodeAct | Characterization |
|---|---|---|
| CHANCERY (Irwin et al., 5 Jun 2025) | Agentic evaluation condition | Framework that “transforms text into code which the model can more easily interpret” |
| SAGE (Hayashi et al., 8 Nov 2025) | Host framework and baseline | OpenHands CodeAct described as “a ReAct-style agent framework with an extensive tool set” |
| ScienceAgentBench (Chen et al., 2024) | Interactive agent framework | OpenHands CodeActAgent v1.9 with Python interpreter, bash shell, and web browser |
| FaultLine (Nitin et al., 21 Jul 2025) | Main open-source baseline | CodeAct 2.1 in OpenHands as a “popular state-of-the-art open-source agentic framework” |
| SafeRun (Chen et al., 8 Jun 2026) | Planning baseline | CodeAct given a Python tool without a deterministic solver |
This role shift is important. CHANCERY does not introduce a new CodeAct algorithm; it evaluates a “CodeAct agent” alongside direct LLMs and a ReAct agent on binary corporate-governance reasoning. SAGE does not propose CodeAct either; it wraps OpenHands CodeAct with an explore → abstract plan → re-execute loop and reformats trajectories so that the abstraction module can consume them. FaultLine uses CodeAct 2.1 in OpenHands as the main general-purpose software-engineering baseline for proof-of-vulnerability generation. SafeRun treats CodeAct as the representative unconstrained code-execution planner against which a deterministic solver-backed architecture is contrasted (Irwin et al., 5 Jun 2025, Hayashi et al., 8 Nov 2025, Nitin et al., 21 Jul 2025, Chen et al., 8 Jun 2026).
This broad reuse suggests that CodeAct has become a reference design pattern for executable-action agents. Its presence in legal reasoning, software engineering, scientific coding, planning, prompt optimization, and healthcare graph traversal indicates that the literature increasingly treats it as a reusable execution substrate rather than a narrowly scoped prompting recipe.
4. Empirical performance across domains
Representative results show that CodeAct can be strong, but highly task-dependent.
| Context | CodeAct result | Comparator |
|---|---|---|
| CHANCERY corporate-governance reasoning (Irwin et al., 5 Jun 2025) | 78.1% accuracy | ReAct 76.1%, GPT-4o 75.2% |
| SWE-Bench Verified with OpenHands CodeAct + SAGE (Hayashi et al., 8 Nov 2025) | 74.0% Pass@1 resolve rate | Baseline CodeAct 71.8% |
| Proof-of-vulnerability generation (Nitin et al., 21 Jul 2025) | 9/100 successful PoV tests | FaultLine 16/100 |
| SafeRun running-planning benchmark (Chen et al., 8 Jun 2026) | 97.6 average safety score | Prompt engineering 79.1, SafeRun 100.0 |
| FHIR-AgentBench after RL post-training (Knorr et al., 13 May 2026) | 76.7 ± 0.9 correctness | Base Qwen3-8B 19.7 ± 0.3, o4-mini 46.9 |
On CHANCERY, the CodeAct agent produced the highest reported score on a benchmark intentionally designed to require multi-step reasoning over charter provisions and, in some cases, general corporate law. The paper interprets this as evidence that agentic decomposition and tool-mediated reasoning help on legal tasks structured as rule-grounded Yes/No adjudication (Irwin et al., 5 Jun 2025).
On SWE-Bench Verified, OpenHands CodeAct serves both as baseline and host framework for SAGE. The best reported result in that paper is 74.0% Pass@1 with OpenHands CodeAct + SAGE + GPT-5 (high), compared with 71.8% for baseline CodeAct under the same backbone, showing that plan abstraction can improve an already strong CodeAct loop without changing model weights (Hayashi et al., 8 Nov 2025).
The picture is less favorable in security. FaultLine reports that CodeAct 2.1 solves 9 out of 100 proof-of-vulnerability tasks, whereas FaultLine solves 16, which the paper interprets as a 77% relative improvement. The failure analysis attributes much of the gap to the absence in vanilla CodeAct of explicit source-to-sink tracing and branch-condition reasoning (Nitin et al., 21 Jul 2025).
In scientific coding, ScienceAgentBench presents a mixed outcome. OpenHands CodeAct often improves valid execution relative to direct prompting, but self-debug outperforms CodeAct on 4 of 5 model families. Without expert-provided knowledge, the best CodeAct success rate is 21.6 for Claude-3.5-Sonnet; with knowledge, the best is 27.5 for GPT-4o. The paper highlights a striking cost-performance contrast for Claude-3.5-Sonnet without knowledge: CodeAct achieves 21.6 success rate at a cost of \$0.958, whereas self-debug reaches 32.4 at \$0.057 (Chen et al., 2024).
In tabular fact verification, CodeAct with Python execution can also be competitive, but the paper’s verdict is guarded. Qwen3-32B CodeAct with SIMBA reaches 69.2 accuracy and 69.3 macro-F1 on SciTab, and GPT-4o + MiPROv2 improves CodeAct on PubHealth, SciTab, and TabFact-mini, yet the overall conclusion is that CoT remains a strong default and CodeAct is less stable than CoT or optimized ReAct across settings (Du et al., 20 Feb 2026).
5. Extensions, hierarchies, and training on top of CodeAct
A substantial line of work keeps the code-as-action premise but modifies the control policy around it. Two Tree-of-Code papers are the clearest examples. One hybridizes Tree-of-Thought and CodeAct by treating each final code execution result as a node, exploring nodes with breadth-first search, and selecting the final answer by majority voting over successful executions; on MToolEval it reports 81.60% correctness for ToC versus 74.40% for CodeAct, while reducing average turns from 5.5 to 2.3 (Ni et al., 2024). A closely related version introduces “CodeProgram” as an end-to-end program-granularity unit and reports that Tree-of-Code reaches 67.1% versus 49.4% for CodeAct on M3ToolEval, and 38.0% versus 19.2% on API-Bank level-3, again with much fewer turns (Ni et al., 2024).
Another direction uses CodeAct as an elicitation scaffold rather than a standalone agent. CodeAdapt combines CodeAct with Generalization-guided Few-shot Learning, using as few as five training problems and selecting two demonstrations. Averaged over four matched model families, 0-shot CodeAct scores 48.3, whereas CodeAdapt with GFL reaches 66.5 and the corresponding reasoning models average 61.9; depending on model pair, CodeAdapt is also 10–81% more token efficient than the reasoning-model baseline (Zhang et al., 23 Oct 2025).
Other work makes CodeAct itself trainable. ToolBrain wraps smolagents.CodeAgent in a trace-based RL framework with GRPO, DPO, and judge-based reward generation, and its email-search case study shows a 7B CodeAct agent improving from 13.3% to 43.3% validation correctness after 60 GRPO steps (Le et al., 24 Sep 2025). In healthcare, a multi-turn CodeAct agent over FHIR retrieval and Python post-processing is post-trained with GRPO and a judge-based binary reward; the resulting Qwen3-8B + RL system reaches 76.7 ± 0.9 correctness on FHIR-AgentBench, far above the untrained Qwen3-8B baseline (Knorr et al., 13 May 2026).
The most explicit architectural generalization is S1-NexusAgent’s hierarchical Plan-and-CodeAct design. There, CodeAct is the inner-loop execution protocol, while the outer loop performs global scientific planning and replanning. On Biomni-Eval-1, the paper reports 37.23 for Only Outer-loop, 38.99 for Only Inner-loop, and 42.42 for the full Dual-loop system; it also shows that SFT + RL improves the CodeAct executor from 38.40 to 42.42. The system complements CodeAct with MCP-native tool integration, intention-aware dynamic tool retrieval, object-reference-based sparse context management, and critic-based trajectory distillation into reusable Scientific Skills (Team, 2 Feb 2026).
A further extrapolation is SPEAR, which ports the code-as-action pattern into automatic prompt engineering. Its optimizer is itself a CodeAct-style agent with evaluate, python, set_prompt, and finish tools, and the paper argues that the Python tool is the largest single lever on complex judge tasks because it enables structural error analysis over the evaluation DataFrame rather than textual critique alone (Lu et al., 25 May 2026).
6. Limitations, critiques, and nomenclature
The literature is equally clear that CodeAct is not a universal solution. Some papers are under-specified: CHANCERY reports a “CodeAct agent” at 78.1% accuracy but does not document its base model, exact tool list, prompts, step limits, or CodeAct-specific ablations, leaving the source of the gain unresolved (Irwin et al., 5 Jun 2025). ScienceAgentBench shows that a larger action space can be a burden rather than a benefit, especially when models struggle with specialized file-editing or bash-style commands, and that CodeAct can be dramatically more expensive than simpler execution-feedback loops (Chen et al., 2024).
Other critiques are domain-structural. FaultLine argues that general-purpose CodeAct lacks explicit source-to-sink tracing and branch-condition reasoning for vulnerability testing, which leaves it behind a workflow inspired by static and dynamic program analysis (Nitin et al., 21 Jul 2025). SafeRun argues that CodeAct’s ability to generate and execute general-purpose Python is still insufficient for determinism-critical planning: its average safety score of 97.6 is strong relative to prompt engineering, but below the 100.0 delivered by a decoupled architecture that restricts the LLM to invoking a deterministic solver (Chen et al., 8 Jun 2026). In tabular fact verification, the main recurring failure modes are table parsing, syntax errors, and numerical aggregation or comparison mistakes, and optimizer gains are inconsistent enough that the paper avoids endorsing a single best optimization strategy for CodeAct (Du et al., 20 Feb 2026).
A cautious synthesis is that CodeAct’s flexibility is strongest when exact computation, tool composition, or local trial-and-error are central, and weaker when the task requires hard deterministic guarantees, highly specialized intermediate representations, or domain-specific traversal constraints that are not made explicit. That interpretation is consistent with the mixed evidence across planning, security, scientific coding, and structured reasoning benchmarks.
The name also has a nomenclature problem. In most of this literature, “CodeAct” refers to the executable-code action paradigm or to descendants such as OpenHands CodeAct. By contrast, “CodeACT” in “Code Adaptive Compute-efficient Tuning” is a training framework for code LLM finetuning built around Complexity and Diversity Aware Sampling and Dynamic Pack; it is unrelated to code-as-action agents (Lv et al., 2024). Likewise, “ACT” in “Auto-Train for Code Translation” denotes an automated pipeline for code translation finetuning, not a CodeAct agent architecture (Saxena et al., 22 Jul 2025). Distinguishing these homographs is necessary because they share surface form while referring to different technical objects.