EvoCurr: Self-Evolving Curriculum Framework

• EvoCurr is a self-evolving curriculum framework that incrementally challenges LLM solvers by generating tasks with escalating difficulty to enhance decision-making policies.
• It employs a dual LLM approach—using a curriculum designer and behavior coder—to dynamically generate and refine executable code without model retraining.
• The framework leverages real-time performance feedback and iterative corrections, proven effective in complex domains such as real-time strategy game AI.

EvoCurr is a self-evolving curriculum framework designed for inference-time improvement of LLM solvers on complex, long-horizon decision-making tasks. By orchestrating a dynamic interplay between a curriculum-generating LLM and a code-generating LLM, EvoCurr adaptively constructs a sequence of problem instances with ascending difficulty, enabling the solver to progressively acquire the required reasoning and control policies. This framework transparently integrates curriculum learning principles into LLM-driven code generation without requiring model retraining, manual curriculum engineering, or prior knowledge of domain decomposition (Cheng et al., 13 Aug 2025).

1. Framework Motivation and Conceptual Architecture

Modern LLMs demonstrate proficiency in reasoning and code synthesis, yet exhibit degraded performance in high-complexity domains characterized by long-horizon dependencies and compounding errors when given a one-shot, unstructured task prompt. EvoCurr addresses this bottleneck by instantiating a "self-evolve" loop that emulates human curriculum learning: an adaptive, staged introduction of progressively harder tasks governed by solver performance. The core pipeline comprises the following components:

1. Curriculum Designer LLM: Proposes the next curriculum instance $C_{i+1}$ by analyzing solver outcomes on $C_{i}$ and the properties of the final task $T_f$.
2. Behavior Coder LLM: Given the new curriculum $C_{i+1}$ and the current decision tree policy $B_{i}$, generates executable Python code representing the improved behavior tree $B_{i+1}$.
3. Environment/Simulator: Executes $B_{i+1}$ on $C_{i+1}$, returning performance metrics $P_{i+1}$ (e.g., win rates, error traces) to close the curriculum feedback loop.
4. Iteration: This cycle continues until the current curriculum matches the final task and the solver attains a target win rate.

This approach eliminates the need for model retraining and handcrafted curricula, operating entirely at inference-time with no access to gradient or model weights.

2. Curriculum Generation: Difficulty Quantification and Adaptation

EvoCurr endows the Curriculum Designer with an explicit performance-driven adaptation protocol. The curriculum's difficulty level, a scalar $d_i \in \mathbb{R}$ associated with $C_{i}$0, is adjusted using the solver’s observed win rate $C_{i}$1 and auxiliary metrics $C_{i}$2:

$C_{i}$3

where $C_{i}$4 in the experiments, and $C_{i}$5 are step sizes inferred by LLM prompt context. The LLM leverages recent history and task specification to prompt new curricula, typically adjusting unit counts, enemy compositions, or scenario geometry. Difficulty increases if the win rate exceeds $C_{i}$6 and is relaxed if the solver repeatedly fails.

Curriculum adaptation is operationalized through a pseudocode-driven prompting protocol:

$T_f$6

3. Solver LLM: Code Generation via Planner–Coder–Critic Loop

Each curriculum is converted into a decision-tree policy by a three-stage LLM loop:

• Planner: Synthesizes a high-level strategy based on current and previous policy, feedback from simulation outcomes, and environment parameters.
• Coder: Converts the strategic outline into Python decision-tree code (e.g., for StarCraft II’s python-sc2 API), embedding explicit conditional logic for actions, thresholds, and abilities.
• Critic: Executes the code in the environment, parses runtime tracebacks, and provides corrective feedback if execution fails (due to syntax errors, deprecated API calls, or underperformance).

The process iterates until the code passes environment validation and achieves the curriculum win rate threshold. This loop allows efficient error recovery and policy refinement within each curriculum stage.

Example (early-stage Marine micro-management):

$T_f$7

4. Self-Evolving Training Loop: Hyperparameters and Control Flow

The overall EvoCurr iteration (Algorithm 1) can be summarized as follows:

$T_f$8

Key hyperparameters include:

• $C_{i}$7: win rate threshold per curriculum (67%)
• $C_{i}$8: maximum curriculum stages (typically $C_{i}$9)
• $T_f$0: code generation attempts per curriculum ($T_f$1)
• $T_f$2: difficulty adjustment increments

Decision criteria:

• Advance curriculum if $T_f$3
• Relax difficulty if $T_f$4 after $T_f$5 attempts
• Any code error triggers Critic feedback

5. Experimental Results and Empirical Insights

EvoCurr was evaluated on the StarCraft II micro-management benchmark, where the final curriculum included complex multi-unit Terran vs Protoss engagements with extensive action dependencies, as detailed in the task specification table (cf. Table 1 in the source):

Unit Type (Terran)	Quantity	Tech/Abilities
Marine	20	Stimpack
Marauder	12	Stimpack
Ghost	3	Personal Cloak
Medivac	3	—
Siege Tank	2	Siege Mode
Viking Fighter	4	—
Liberator	2	—

Enemy units comprised Zealot (Charge), Stalker (Blink), High Templar (PsiStorm), Colossus (ExtLance), and Disruptor variants.

Benchmark outcomes:

• EvoCurr succeeded in 1/5 (20%) runs at mastering the final task (reaching a 100% win rate after 6 curricula). The average curriculum depth was approximately 5.6 (range: 4–7).
• The one-shot direct-generation baseline failed in all runs, typically producing code that never surpassed the 67% threshold or suffered from runtime errors.
• Ablations reveal that omitting the Curriculum Designer eliminates success, and removing the Critic loop increases code failures and decreases win rates.

6. Strengths, Limitations, and Applications

Strengths:

• Adaptive difficulty maintains the agent's performance within the "zone of proximal development," precluding trivialization or overwhelming failures.
• Code and decision-branch reuse from prior curricula enhances transfer to more complex challenges.
• Critic-driven feedback robustly prevents code hallucinations and API incompatibilities.

Limitations:

• Tendency for the Behavior Coder to focus on advanced micro-management in a subset of units, with coarse handling of others, attributed to token budget and input context constraints.
• Occasional misestimation of appropriate curriculum scaling. This suggests further optimization of prompt engineering and threshold selection.
• Absence of a formal, quantitative notion of "difficulty"; the system relies on implicitly learned LLM semantic priors.

Applications:

• Real-time strategy game AI for multi-agent coordination (e.g., Dota 2, fighting games)
• Robotics, where hierarchical curricula scaffold skill acquisition under physical and environmental uncertainty
• Structured decision tree synthesis in logistics, scheduling, and autonomous driving domains, especially where verifiable logic is preferable to end-to-end models

EvoCurr demonstrates that a coupled LLM system, synthesizing both task curricula and executable control code, can achieve significantly higher rates of success than non-curricular one-shot baselines in complex domains. The architecture provides a modular template for LLM-driven curriculum learning in automated reasoning and policy induction, with generalization potential beyond the tested domain (Cheng et al., 13 Aug 2025).