Cooperative Prompt Optimization (COPRO)

• COPRO is a methodology that decomposes prompt optimization into multi-agent tasks, using gradient fusion, RL, and combinatorial search to improve LLM performance.
• It employs specialized frameworks like MAPGD, MultiPrompter, and DSPy to address high-dimensional prompt space and gradient conflicts with coordinated agent strategies.
• Empirical results demonstrate that COPRO achieves higher accuracy and interpretability in applications such as classification, text-to-image generation, and hallucination detection.

Cooperative Prompt Optimization (COPRO) is a class of methodologies for automated search and refinement of prompts for LLMs and related generative systems, leveraging multi-agent collaboration, cooperative game structures, and structured search strategies. COPRO frameworks have been developed within diverse contexts, including multi-agent reinforcement learning for compositional prompt generation (Kim et al., 2023), interpretable bandit-based and gradient-inspired optimization (Han et al., 14 Sep 2025), and breadth-limited combinatorial search within modular frameworks such as DSPy (Sarmah et al., 2024). These methods aim to surmount the computational and conceptual limitations of single-agent prompt optimization—including high-dimensionality, gradient conflicts, limited interpretability, and restrictive search trajectories—by decomposing the task among specialized agents or policy modules.

1. Foundational Definitions and Objectives

Cooperative Prompt Optimization addresses the challenge of finding a prompt $p \in \mathcal{P}$ that minimizes a task-specific loss $F(p)$ for downstream LLM or generative model performance. The general form of the optimization problem, as articulated in MAPGD (Han et al., 14 Sep 2025), is

$$F(p) = \mathbb{E}_{(x, y) \sim \mathcal{D}} [\ell(M(x; p), y)],$$

where $M(x; p)$ is the model output under prompt $p$, and $\ell$ is a supervised or self-supervised loss (e.g., cross-entropy, negative ROUGE). Similarly, COPRO approaches within DSPy (Sarmah et al., 2024) and MultiPrompter (Kim et al., 2023) define the objective as maximizing (or minimizing) evaluation utilities—such as exact-match accuracy, cross-entropy alignment, or multimodal rewards—over a space of template and example selections parameterized by $\theta$.

COPRO methods seek to overcome the combinatorial explosion of prompt space and improve sample efficiency, adaptability, and robustness, by distributing the search or construction process among multiple agents, each specializing or acting cooperatively, with coordination mechanisms designed for joint optimization.

2. Algorithmic Frameworks for COPRO

(a) Multi-Agent Gradient-Based COPRO (MAPGD)

MAPGD (Han et al., 14 Sep 2025) models prompt engineering as a multi-agent optimization problem, employing $K$ specialized agents $\{A_k\}_{k=1}^K$:

• $A_1$: Task-instruction clarity
• $F(p)$0: Few-shot example selection
• $F(p)$1: Output-format enforcement
• $F(p)$2: Stylistic/refinement

At iteration $F(p)$3, each agent proposes an update $F(p)$4—treated as a “pseudo-gradient” in embedding space—based on model errors observed in a minibatch. These updates are fused using a weighted aggregation in semantic space, with weights $F(p)$5 proportional to recent validation gains, and postprocessing (e.g., via LLM) for syntactic correctness. Candidate prompts are generated through application of the fused update, paraphrasing/diversity sampling, followed by bandit-based selection (UCB1) over the dev set for efficient evaluation.

(b) Cooperative Game-Theoretic COPRO (MultiPrompter)

In MultiPrompter (Kim et al., 2023), COPRO is formalized as a fully cooperative Markov decision process (MDP)/stochastic game $F(p)$6 involving $F(p)$7 prompter agents constructing a prompt by sequentially emitting tokens. States encode both the initial prompt and the constructed prefix; agents take turns choosing tokens until an end-of-sequence condition or token budget is met. The reward, shared by all agents, is computed only at episode termination, and can incorporate CLIP relevance, aesthetic scores, and a cooperation entropy bonus for subprompt balance. Each agent’s policy $F(p)$8 is parameterized by a transformer and learned using actor-critic methods (notably PPO with a centralized critic that accesses cross-agent information during training).

(c) Structured Tree Search COPRO (DSPy Teleprompters)

Within the DSPy framework (Sarmah et al., 2024), COPRO executes a breadth-limited, depth-limited cooperative search over the prompt parameter space. At each refinement level $F(p)$9, up to $$F(p) = \mathbb{E}_{(x, y) \sim \mathcal{D}} [\ell(M(x; p), y)],$$0 neighbors are generated from each surviving candidate by discrete alterations: example insertion, instruction rephrasing, or demonstration reordering. The best $$F(p) = \mathbb{E}_{(x, y) \sim \mathcal{D}} [\ell(M(x; p), y)],$$1 candidates are retained, and the process runs for $$F(p) = \mathbb{E}_{(x, y) \sim \mathcal{D}} [\ell(M(x; p), y)],$$2 iterations. The objective is explicit alignment between LLM as judge and ground-truth human annotations, e.g., maximizing exact-match utility on hallucination detection tasks.

Framework	Agent Coordination	Optimization Type	Example Setting
MAPGD	Specialized agent fusion	Gradient-inspired	Classification, reasoning tasks
MultiPrompter	Turn-based cooperative	Actor-Critic RL	Text-to-image prompt design
DSPy COPRO	Branching tree search	Greedy BFS/Annealed	Prompt quality for LLM judging

3. Coordination, Fusion, and Credit Assignment

A defining aspect of COPRO is coordinated optimization across multiple agents or modules. MAPGD (Han et al., 14 Sep 2025) resolves possibly conflicting update directions by projecting each agent’s pseudo-gradient into a shared semantic space using encoders like Sentence-BERT, then clusters and fuses the gradients based on similarity. Bandit-based candidate selection (e.g., UCB1) ensures effective exploration-exploitation under limited evaluation budgets.

MultiPrompter (Kim et al., 2023) employs a centralized critic for policy-gradient credit assignment: each agent’s value function conditions on the next agent’s planned subprompt, reducing variance and enhancing coordination via future-aware bootstrapping in the advantage computations.

In DSPy-style COPRO (Sarmah et al., 2024), branching search and prompt variant tracking in the candidate tree enforce cooperative template construction, but there is no underlying gradient or actor-critic signal; coordination is instead architectural (tree-structured prompt search space) and utility driven.

4. Comparative Empirical Results

Evaluations of COPRO methods span text, multimodal, and meta-evaluation benchmarks:

• MAPGD (Han et al., 14 Sep 2025) demonstrates superior F1 on LIAR, Jailbreak, and Ethos, as well as on the reasoning-generation benchmark DEREK: for LIAR (F1: MAPGD 0.71 vs. MC 0.62 vs. ProTeGi 0.64); for Jailbreak, MAPGD reaches 0.88, outperforming single-agent and random search baselines.
• MultiPrompter (Kim et al., 2023) for text-to-image prompt composition (CLIP/aesthetic reward) attains a test reward of 0.76 ± 0.10, exceeding manual (−0.68 ± 0.06), single-agent RL (0.28 ± 0.11), and competitive self-play (0.36 ± 0.12). The optimal number of cooperative prompters is $$F(p) = \mathbb{E}_{(x, y) \sim \mathcal{D}} [\ell(M(x; p), y)],$$3, above which coordination becomes bottlenecked.
• DSPy COPRO (Sarmah et al., 2024) achieves 82.13% accuracy and micro-F1 0.7920 on HaluBench hallucination detection, with computational cost characterized as medium ($$F(p) = \mathbb{E}_{(x, y) \sim \mathcal{D}} [\ell(M(x; p), y)],$$4150 prompt evaluations).

Method/Setting	Accuracy / Reward	Micro-F1 / F1	Prompt Length	Task
MAPGD (LIAR)	F1 = 0.71	—	—	Fact-checking classification
MultiPrompter (COCO)	0.76 ± 0.10	—	69.5 tokens	Text-to-image
DSPy COPRO (HaluBench)	82.13%	0.7920	—	Hallucination detection

A pattern across studies is that cooperative frameworks, particularly those with agent specialization and semantic fusion, consistently outperform single-agent or random-structure baselines both in accuracy and interpretability.

5. Limitations, Failure Modes, and Recommendations

COPRO methods vary in generality, efficiency, and robustness:

• Greedy breadth-limited search in DSPy COPRO (Sarmah et al., 2024) efficiently yields high-accuracy prompt templates but is prone to overfitting majority classes and fails to improve minority-class F1 (e.g., Macro-F1 = 0.2267 for COPRO vs. 0.8019 for GPT-4o baseline).
• Multi-agent RL approaches, such as MultiPrompter (Kim et al., 2023), become increasingly difficult to coordinate as $$F(p) = \mathbb{E}_{(x, y) \sim \mathcal{D}} [\ell(M(x; p), y)],$$5 grows. Performance drops beyond $$F(p) = \mathbb{E}_{(x, y) \sim \mathcal{D}} [\ell(M(x; p), y)],$$6 due to the growing complexity of agent interaction.
• MAPGD (Han et al., 14 Sep 2025) requires careful semantic coordination to resolve conflicting updates, and the bandit-based selection must balance exploration and evaluation budget.

Recommendations include using COPRO when the dataset is balanced and computational resources are sufficient for moderate prompt evaluations. For tasks with severe class imbalance or requiring continuous prompt parameter tuning, annealed or Bayesian optimization-based alternatives (such as MIPRO or BootstrapFewShot+Optuna in DSPy) may be more effective. Scalability remains a challenge, as search complexities scale as $$F(p) = \mathbb{E}_{(x, y) \sim \mathcal{D}} [\ell(M(x; p), y)],$$7 (branching factor and depth) for tree-search methods.

6. Relationships to Alternative and Hybrid Approaches

COPRO, as operationalized in MAPGD, MultiPrompter, and DSPy, contrasts with both random search and purely single-agent optimization (e.g., ProTeGi, Promptist), as well as with non-cooperative or competitive multi-agent variants. Structured COPRO methods (MAPGD) integrate gradient-based feedback with agent specialization, while MultiPrompter leverages multi-agent RL with a centralized critic for joint prompt composition. DSPy COPRO emphasizes combinatorial tree search, with no underlying LM finetuning. Comparative benchmarks (Table 1 in (Sarmah et al., 2024)) demonstrate that hybrid approaches, such as MIPRO (annealed multi-stage search with hyperparameter tuning) or BootstrapFewShot+Optuna (Bayesian hyperparameter optimization), may outperform COPRO in settings requiring class balance or more extensive hyperparameter tuning.

7. Future Directions and Open Challenges

Key challenges for COPRO include scalability to larger agent numbers or deeper prompt spaces, improved handling of class imbalance and credit assignment, and integration with more advanced multi-agent learning techniques such as MAPPO or mean-field RL (Kim et al., 2023). A plausible implication is that hybrid frameworks, combining cooperative prompt composition with bandit-based or Bayesian optimization, may address both search-space tractability and metric alignment. The support for agent specialization, semantic coordination, bandit-based candidate selection, and provable convergence (e.g., $$F(p) = \mathbb{E}_{(x, y) \sim \mathcal{D}} [\ell(M(x; p), y)],$$8 rate in MAPGD), positions COPRO as a robust, interpretable, and extensible methodology for prompt optimization, but further empirical and algorithmic refinements are required for widespread adoption across application domains.