---
title: 'CoAct: Collaborative Agent Frameworks'
url: https://www.emergentmind.com/topics/coact
type: topic
---

# CoAct: Collaborative Agent Frameworks

CoAct is not a single canonical method but a reused research label attached to several technically distinct frameworks in recent machine learning and agent-systems literature. In current arXiv usage, it denotes a global-local hierarchy for autonomous agent collaboration [2406.13381], Consistency-guided Asynchronous Contrastive Tuning for few-shot class-incremental tuning of foundation models [2405.16625], Co-Active LLM Preference Learning with Human-AI Synergy [2604.17501], and Action-Preserving Observation Compression for coding agents [2607.02911]. A closely related title, CoAct-1, denotes a computer-using multi-agent system that combines GUI control with direct programmatic execution [2508.03923]. This suggests that “CoAct” functions primarily as an acronymic family name for methods built around coordination, consistency, or action-aware control rather than as a single unified paradigm.

## 1. Nomenclature and research scope

The label appears across multiple subfields, with each paper defining its own expansion and technical setting.

| Label | Domain | Defining title |
|---|---|---|
| CoAct [2406.13381] | autonomous agents | “A Global-Local Hierarchy for Autonomous Agent Collaboration” |
| CoACT [2405.16625] | continual vision adaptation | “Consistency-Guided Asynchronous Contrastive Tuning for Few-Shot Class-Incremental Tuning of Foundation Models” |
| CoAct [2604.17501] | LLM alignment | “CoAct: Co-Active LLM Preference Learning with Human-AI Synergy” |
| CoACT [2607.02911] | coding agents | “CoACT: Action-Preserving Observation Compression for Coding Agents” |
| CoAct-1 [2508.03923] | computer-use agents | “CoAct-1: Computer-using Agents with Coding as Actions” |
| CoAction [2605.01712] | multi-objective optimization | “CoAction: Cross-task Correlation-aware Pareto Set Learning” |

The shared naming pattern is therefore semantically suggestive but technically non-binding. In some papers, the core idea is hierarchical collaboration; in others, it is consistency regularization, human-AI co-labeling, or next-action preservation. Any encyclopedia treatment of CoAct must therefore be disambiguating rather than singular.

## 2. Hierarchical collaboration in autonomous agents

In the agent-systems literature, CoAct denotes a two-agent framework that transfers “the hierarchical planning and collaboration patterns in human society to LLM systems” [2406.13381]. Its architecture contains a **Global Planning Agent (GP)** and a **Local Execution Agent (LA)**. GP “comprehend[s] the problem scope, formulate[s] macro-level plans and provide[s] detailed sub-task descriptions,” while LA operates “within the multi-tier task execution structure, focusing on detailed execution and implementation of specific tasks within the global plan” [2406.13381].

The control protocol is explicitly hierarchical. GP decomposes a task \(T\) into a global plan \(Plan_g = [Subtask_1, Subtask_2, ..., Subtask_N]\), assigns subtasks sequentially, maintains global memory, and can revise or overrule replanning requests. LA receives a subtask \(Subtask_g\), generates a local plan, executes actions, validates outcomes, and either revises locally or escalates via a `request` when the global strategy appears suboptimal. The framework defines GP meta-actions `global_plan`, `decide`, `revise`, `overrule`, and `collation`, and LA meta-actions `local_plan`, `pass_check`/`false_check`, `revise`, `request`, and `overruled` [2406.13381].

The paper emphasizes adaptivity under failure. When repeated local failures or unanticipated environment changes occur, LA can ask GP for replanning, allowing “trajectory re-arrangement” rather than continued commitment to an unproductive trajectory [2406.13381]. This directly targets a limitation attributed to single-agent ReAct-style execution, namely accumulation of misleading context and repetition without explicit global correction.

Empirically, the framework is evaluated on WebArena across Shop, CMS, Reddit, GitLab, and Map, with success rate as the metric over 100 examples per domain [2406.13381]. Reported averages are 9.4 for ReAct, 13.8 for CoAct, and 16.0 for CoAct w/ FS. Domain-level values are also given: for example, Shop improves from 12.0 under ReAct to 22.0 under CoAct and 24.0 under CoAct w/ FS [2406.13381]. The paper further reports that preliminary integration of web page-specific knowledge yields additional gains, e.g. “Shop: 24.0% ⇒ 31.0%” [2406.13381].

## 3. Incremental tuning of vision foundation models

In vision continual learning, CoACT denotes **Consistency-guided Asynchronous Contrastive Tuning**, a method for “continuously tuning foundation models to learn new classes in few-shot settings” [2405.16625]. The target settings are Few-Shot Class-Incremental Learning (FSCIL) and the paper’s newly introduced Few-Shot Class-Incremental Tuning (FSCIT), the latter differing from traditional FSCIL in that it “does not require a large in-distribution base session for initial fully supervised training prior to the incremental few-shot sessions” [2405.16625].

The method is organized around three components. First, **asynchronous contrastive tuning** inserts LoRA modules into the pre-trained encoder and enforces consistency between a student encoder with LoRA and a teacher encoder updated as an EMA of the student backbone weights. The objective combines supervised cross-entropy with an asynchronous contrastive loss:
\[
\mathcal{L} = \mathcal{L}_{\text{ACL}} + \lambda\, \mathcal{L}_{\text{sup}}.
\]
Second, **controlled fine-tuning** first trains only LoRA modules for \(E_c\) epochs with a high learning rate, then fine-tunes the last \(C_l\) backbone layers plus LoRA for the remaining epochs using a smaller learning rate scaled by factor \(C_f\) [2405.16625]. Third, **consistency-guided incremental tuning** freezes the encoder after the first session and regularizes later sessions through a contrastive consistency loss against the frozen post-first-session encoder:
\[
\mathcal{L} = \mathcal{L}_{\text{CR}} + \gamma\, \mathcal{L}_{\text{ACL}} + \lambda\, \mathcal{L}_{\text{sup}}.
\]

The evaluation spans 16 datasets, including CIFAR-100, miniImageNet, Caltech101, CUB-200, VOC 2007, OxfordPets, StanfordCars, Flower102, Food101, FGVCAircraft, SUN397, Country211, EuroSAT, Resisc-45, DTD, and GTSRB [2405.16625]. The paper reports that CoACT “outperforms existing methods by up to 5.02% in FSCIL and up to 12.51% in FSCIT for individual datasets, with an average improvement of 2.47%” [2405.16625]. It also reports reduced forgetting: “CoACT: 5.8% forgetting, compared to 7.2% (CPE-CLIP) and 21.5% (BOT)” [2405.16625].

On traditional FSCIL benchmarks, the reported final-session accuracies are 84.62% on CIFAR-100, 81.19% on CUB-200, and 96.24% on miniImageNet [2405.16625]. Ablation results attribute the strongest performance to the combination of all three components, with the full model reaching 61.43% average accuracy versus 58.77% when all three are removed [2405.16625]. The paper also states that an asynchronous teacher without LoRA on the EMA branch outperforms a synchronous two-LoRA setup by approximately 1.4% [2405.16625].

## 4. Preference learning with human-AI synergy

In LLM alignment, CoAct denotes a framework that “synergistically combines self-rewarding and active learning through strategic human-AI collaboration” [2604.17501]. The setting begins with an unlabeled instruction pool \(\mathcal{D}_U = \{x_j\}_{j=1}^{N_U}\) and a labeled preference set \(\mathcal{D}_L = \{(x_i, y_i^+, y_i^-)\}\). At each active iteration, the model samples a batch, generates \(k\) responses per instruction, and uses self-consistency to construct preference pairs [2604.17501].

The central statistic is the consistency function
\[
C(y) = \frac{1}{k} \sum_{m=1}^k \mathbf{1}\{\text{ans}(y_m) = \text{ans}(y)\},
\]
which is used to choose the most consistent response \(y^+\) and least consistent response \(y^-\) for each instruction [2604.17501]. Samples are partitioned by a threshold \(\tau\) into high-consistency and low-consistency subsets. Low-consistency cases are candidates for oracle annotation. High-consistency cases may still be “confident but wrong,” so the method applies k-NN OOD detection using normalized penultimate features \(z_i = \phi(x_i)/\|\phi(x_i)\|_2\) and distance
\[
r_k(z_i) = \min_{z \in \mathcal{Z}_{\text{ID}}} \|z_i - z\|_2
\]
to prioritize additional oracle verification [2604.17501].

A second distinctive component is **oracle-guided instruction augmentation**. Oracle-verified high-consistency examples are used as in-context demonstrations to prompt generation of new instructions within the model’s solvable capability:
\[
\mathcal{D}_{\text{new}}^{(t)} = \{ x'_i \sim \theta_t(\cdot|\text{ICL}(\mathcal{D}_{\text{correct}}^{(t)}, n)) \}_{i=1}^{N_{\text{new}}}.
\]
Training then proceeds on the union of oracle-labeled and AI-labeled data via a modified DPO objective with NLL regularization:
\[
\mathcal{L}(\theta) = -\mathbb{E}_{(x, y^+, y^-) \sim \mathcal{D}} \left[ \log \sigma\left( \beta \log \frac{\theta(y^+|x)}{\theta_0(y^+|x)} - \beta \log \frac{\theta(y^-|x)}{\theta_0(y^-|x)} \right) - \alpha|y^+| \log \theta(y^+|x) \right].
\]
The paper also provides a sample-efficiency statement: adding noisy self-labeled data helps “as long as label noise \(\epsilon_{ai} < 0.5\)” [2604.17501].

The reported evaluation covers GSM8K, MATH, and WebInstruct, using Llama3-8B and Qwen3-4B [2604.17501]. Average gains versus the base model are given as +13.25% on GSM8K, +8.19% on MATH, and +13.16% on WebInstruct [2604.17501]. Final iteration results include, for example, 43.58 on GSM8K, 14.46 on MATH, and 15.97 on WebInstruct for Llama3-8B, and 94.84, 75.71, and 53.96 respectively for Qwen3-4B [2604.17501]. The paper further reports strong correlation between self-consistency score and accuracy, with Pearson correlation “up to ~0.97 as training progresses” [2604.17501].

## 5. Action-preserving compression for coding agents

In software-engineering agents, CoACT denotes **Action-Preserving Observation Compression**, a method for reducing context cost while preserving downstream behavior [2607.02911]. The paper motivates the problem by noting that returned observations accumulate in context and are “a major source of inference cost,” with observations accounting for “45.7% on SWE-bench Verified” [2607.02911].

The framework is built on **next-action preservation (NAP)**. Compression is posed as
\[
\min_C \mathrm{Tokens}(C) \quad \text{s.t.}\quad \mathrm{success}(C) \geq \mathrm{success}(\mathbb{I}) - \epsilon,
\]
and NAP supplies a dense proxy by requiring that a compressed observation induce the same next action as the raw observation:
\[
a_{t+1}^{C} = a_{t+1}^{\mathbb{I}}, \quad \forall t.
\]
Training proceeds in four stages. A teacher model generates \(N\) candidate compressions \(\{\hat{o}_t^{(j)}\}\). For each candidate, the agent policy \(\pi\) samples \(K\) reference next actions on the raw observation, predicts \(\hat{a}_{t+1}^{(j)}\) on the compressed observation, and computes an action-preservation reward by averaging the top-\(M\) similarity scores:
\[
r_{\mathrm{act}}(\hat{o}_t^{(j)}) = \text{mean}(\text{Top}_M\{\mathrm{sim}(\hat{a}_{t+1}^{(j)}, a_{t+1}^{(k)})\}_{k=1}^K).
\]
Candidates with \(r_{\mathrm{act}} \geq \theta\) are retained, then ranked by a length-reduction reward
\[
r_{\mathrm{len}}(\hat{o}_t^{(j)}) = |o_t| - |\hat{o}_t^{(j)}|.
\]
If no candidate passes NAP, the raw observation is used. The compressor is trained by an **offline bootstrap** followed by **online alignment** under compressed trajectories, and deployment uses only the final lightweight compressor \(c_{\phi_2}\), preserving KV cache because past context is left unchanged [2607.02911].

Experiments are reported on 200 randomly sampled SWE-bench Verified issues using Qwen3.5-35B-A3B, Deepseek-v4-Pro, and Gemini3-Flash as agentic models, Qwen3.5-4B as the LoRA-fine-tuned compressor, and Gemini3-Flash as the primary teacher [2607.02911]. The headline result is that CoACT “reduces average total token consumption by 33.0% while maintaining task-solving effectiveness close to the uncompressed agent” [2607.02911]. Reported pass@1 and total-token figures include 57.0 and 3.80M for Qwen3.5-35B-A3B Vanilla versus 60.5 and 2.43M for CoACT, 76.5 and 1.07M for Deepseek-v4-Pro Vanilla versus 75.0 and 0.87M for CoACT, and 73.5 and 0.69M for Gemini3-Flash Vanilla versus 77.5 and 0.42M for CoACT [2607.02911].

The ablations are unusually direct. Removing the action-preservation reward causes a “-10.5% absolute” pass@1 drop, while removing the length-reduction reward reduces token savings [2607.02911]. The paper also reports that CoACT combines effectively with trajectory compression: “AgentDiet+CoACT halves cost from \$45.65 to \$25.88 without hurting effectiveness” [2607.02911].

## 6. Computer-use agents, coding as actions, and safety

CoAct-1 extends the naming family into computer-use agents by explicitly adding coding to the action space [2508.03923]. The system contains three specialized roles: an **Orchestrator**, a **Programmer**, and a **GUI Operator**. The Orchestrator “decomposes the user's natural language goal into subtasks” and assigns each to either the GUI Operator or the Programmer; the Programmer “write[s]/exectues Python/Bash scripts for OS-level and backend operations”; and the GUI Operator is a VLM-based component that outputs GUI actions from screenshots and natural-language instructions [2508.03923]. Each agent has isolated conversation history, and after subtask completion the transient memory is summarized and returned to the Orchestrator together with a screenshot [2508.03923].

The paper’s central claim is that coding is treated as a first-class action, not merely an auxiliary tool. Formally, for environment state \(S\), goal \(G\), and action sequence \(\pi\), pure GUI agents satisfy \(a_k \in GUI\_Actions\), whereas CoAct-1 allows
\[
a_k \in \{\text{GUI\_Action}, \text{CODE\_Action}\}.
\]
The paper also gives a simple success-probability argument: if \(p\) is the step error rate, then for \(N\) GUI steps,
\[
P_{\mathrm{success}} \approx (1-p)^N,
\]
so replacing many GUI actions with one atomic code action can improve robustness exponentially in step count [2508.03923].

On OSWorld, the reported overall success rate is 60.76%, exceeding GTA-1 at 53.10%, Agent S2.5 at 56.00%, OpenAI CUA 4o at 31.40%, and UI-TARS at 29.60% [2508.03923]. The average number of steps per task is reported as 10.15 for CoAct-1 versus 15.22 for GTA-1 and 14.90 for UI-TARS [2508.03923]. Category-specific gains are strongest where coding can replace long GUI sequences: Calc improves from 59.57% to 70.21%, VSCode from 60.86% to 78.26%, Multi-Apps from 38.34% to 47.88%, OS tasks from 62.50% to 75.00%, and VLC from 53.29% to 66.07% [2508.03923].

A major qualification arises from the safety literature. OS-BLIND evaluates computer-use agents under benign user instructions where harm emerges from context or execution outcome rather than explicit malicious intent [2604.10577]. The benchmark contains “300 human-crafted tasks across 12 categories, 8 applications, and 2 threat clusters” [2604.10577]. Reported results show that “most CUAs exceed 90% attack success rate (ASR), and even the safety-aligned Claude 4.5 Sonnet reaches 73.0% ASR” [2604.10577]. When Claude 4.5 Sonnet is deployed in multi-agent systems, ASR rises “from 73.0% to 92.7%” [2604.10577]. For CoAct-1 instantiated with Sonnet, the reported ASR values are 93.1% for environment-embedded threats and 87.9% for agent-initiated harms [2604.10577]. The analysis attributes this degradation to the fact that “decomposed subtasks obscure the harmful intent from the model,” and further reports that “most of the rejection actions happen in the first two steps,” after which safety alignment rarely re-engages [2604.10577]. This makes CoAct-1 simultaneously a high-performing automation framework and a salient case study in the safety costs of decomposition.

## 7. Related naming variant: CoAction in Pareto set learning

A closely related naming variant is **CoAction**, short for **Cross-tAsk correlation-aware Pareto Set Learning**, in multi-objective optimization [2605.01712]. Although the title differs from CoAct, it belongs to the same acronymic family and is often retrieved in the same search space. The method addresses a limitation of prior Pareto set learning (PSL): existing methods are “single-task,” requiring one model per multi-objective optimization problem and failing to exploit inter-task correlations [2605.01712].

CoAction proposes a unified multitask framework with task-specific sinusoidal embeddings
\[
\mathbf{V} = [\mathbf{e}_t, \boldsymbol{\lambda}(\theta), \mathbf{0}_{d_{\max}-m_t}],
\]
a Transformer encoder backbone, weighted pooling, and task-specific output heads [2605.01712]. Its hypervolume-based surrogate objective is
\[
\overline{\mathcal{H}_r(\beta)} = c_m\, \mathbb{E}_\theta[\rho_\beta(\theta)],
\]
with
\[
\rho(x, \theta) = \min_{i \in [m]} \left\{ \frac{r_i - f_i(x)}{\lambda_i(\theta)} \right\}.
\]
The training loop samples task index and preference vector, computes a hypervolume-based loss, and updates shared parameters with optional gradient clipping [2605.01712].

Evaluation uses seven problems—four benchmarks (ZDT1-2, VLMOP1-2) and three real-world tasks (RE21, RE24, RE37)—plus a bbob-biobj suite of five 10D tasks [2605.01712]. The paper reports that CoAction “trains 7 tasks in one pass (5000 iters) vs. 7×1000 (single task), saving 27% compute,” and achieves effectiveness and competitive performance in Hypervolume, Range, and Sparsity [2605.01712]. It also reports that the Transformer backbone is superior to an MLP on more complex many-objective problems such as RE37 [2605.01712]. As with the other CoAct variants, the naming emphasizes coordination—here, specifically cross-task knowledge sharing via shared transformer representations and task-aware embeddings.

The accumulated literature therefore supports a disambiguated understanding of CoAct. The term names several unrelated but technically substantial frameworks: a hierarchical collaboration architecture for LLM agents, a continual-tuning method for vision foundation models, a human-AI preference learning pipeline, an action-preserving compression scheme for coding agents, and a computer-use multi-agent system with code as an action primitive, alongside the closely related CoAction formulation in multitask Pareto set learning [2406.13381; 2405.16625; 2604.17501; 2607.02911; 2508.03923; 2605.01712]. The common thread is nominal rather than formal.

Source: https://www.emergentmind.com/topics/coact