FAMOU: LLM-Driven Program Evolution

• FAMOU is a framework for LLM-driven evolution where evaluator feedback governs executable program mutation and candidate selection.
• It leverages structured, contract-based evaluation to assess multi-objective performance and enforce critical constraints in application-specific domains.
• FAMOU’s design, demonstrated in aerodynamic drag prediction and adversarial games, integrates methods like co-evolution and adaptive sampling to drive robust optimization.

Searching arXiv for FAMOU and closely related papers to ground the article. Searching for papers with “FAMOU” in title/abstract. FAMOU is a designation used in recent arXiv literature for LLM-driven program-evolution systems in which the search object is an executable program and evaluation is itself a structured, programmable component of the optimization loop. In "A Blueprint for Self-Evolving Coding Agents in Vehicle Aerodynamic Drag Prediction," FAMOU denotes the Famou-Agent-style evaluator feedback mechanism that governs candidate admission, failure analysis, and mutation guidance in surrogate discovery for vehicle drag prediction (Ren et al., 23 Mar 2026). In "Beyond Static Evaluation: Co-Evolutionary Mechanisms for LLM-Driven Strategy Evolution in Adversarial Games," FAMOU is expanded as the "Framework for Automated Mutation and Optimization of Utilities" and instantiated as a co-evolutionary framework for adversarial multi-agent games (Li et al., 9 Jun 2026). Across these uses, FAMOU is characterized by executable phenotypes, evaluator-mediated selection pressure, and feedback loops that couple code mutation to explicit engineering or gameplay objectives.

1. Terminology and conceptual scope

In the aerodynamic surrogate-discovery setting, a candidate is "not just a set of weights but a fully executable training-and-evaluation program (the phenotype) derived from a genome (pipeline configuration + code scripts)." FAMOU occupies the "Evaluate" step of each generation and serves four functions: enforcing an evaluation contract, computing multi-objective performance signals, extracting failure modes into a Constraint Memory, and returning a compact performance report to guide the LLM-driven mutation engine (Ren et al., 23 Mar 2026). This makes the evaluator an active control component rather than a passive scorer.

In the adversarial-game setting, FAMOU is defined as a framework that "maintains a population of executable strategy programs and alternates between evaluation, selection, and LLM-driven mutation, while continually adapting its evaluator via co-evolution" (Li et al., 9 Jun 2026). Here the evaluator is not fixed: the opponent pool itself changes as new champions are discovered, and this changing evaluator becomes part of the search dynamics.

A useful synthesis is that FAMOU shifts optimization from static model fitting to constrained search over programs. This is explicit in the aerodynamic paper, which states that surrogate discovery is formulated as constrained optimization over programs, not static model instances (Ren et al., 23 Mar 2026). A plausible implication is that FAMOU is best understood as a research pattern for auditable code evolution under domain-specific contracts.

2. Program representation and optimization logic

Both FAMOU instantiations operate on executable artifacts. In the aerodynamic system, the phenotype is a training-and-evaluation pipeline; in the adversarial system, each candidate is a "500-1700 line Python file implementing $\mathrm{compute\_action}(obs,info)$" (Li et al., 9 Jun 2026). This program-level representation allows mutations to alter data processing, model class, loss function, split policy, or tactical logic rather than merely retuning numeric parameters.

The aerodynamic formulation makes this explicit through a contract function

$$\mathcal{C}(c)=\mathbf{1}\bigl[\neg L(c)\wedge R(c)\wedge D(c)\wedge B(c)\bigr],$$

where $L(c)$ denotes leakage detected, $R(c)$ deterministic replay, $D(c)$ multi-seed robustness, and $B(c)$ budget compliance. If the contract passes, the continuous fitness is

$$F(c)=\omega_1\,\bar{\rho}+\omega_2\,(1-\sigma_\rho)-\omega_3\,\mathrm{cplx}(c),$$

and if any gate fails, $F(c)=-\infty$ and the failure reason is recorded in memory (Ren et al., 23 Mar 2026). The formulation is therefore explicitly discontinuous: contract violations do not merely reduce a score but invalidate a candidate.

The adversarial-game formulation uses a weighted evaluator

$F(c)=\sum_{i=1}^{n}w_i\,\mathrm{metric}(c,o_i),$

where the opponent pool $O=\{o_1,\dots,o_n\}$ and the weight vector $$\mathcal{C}(c)=\mathbf{1}\bigl[\neg L(c)\wedge R(c)\wedge D(c)\wedge B(c)\bigr],$$0 are themselves mutable components of the evaluator (Li et al., 9 Jun 2026). Parent selection is performed via UCB, after which the backbone LLM rewrites the entire Python file under one of five rewrite-style templates while preserving EVOLVE-BLOCK markers and API compatibility.

In both settings, the evaluator is coupled to population methods. The aerodynamic system uses an NSGA-II style selector with non-dominated sorting and a QD archive implemented as MAP-Elites, while the game framework uses fast evaluation for ranking and hierarchical deep evaluation for confirmation (Ren et al., 23 Mar 2026, Li et al., 9 Jun 2026). The shared methodological point is that FAMOU does not treat evaluation as a scalar endpoint; it treats it as a structured signal source for subsequent search.

3. Contract-centric feedback in aerodynamic drag prediction

Within the vehicle-drag pipeline, FAMOU is the core evaluation harness. The paper states that it "treats the surrogate pipeline as a system under test, runs it through a battery of programmable 'gates,' and then hands back structured feedback that influences which variation operators (data/model/loss/split) the agent invokes next" (Ren et al., 23 Mar 2026). The relevant gates are leakage prevention, deterministic replay, bounded resources, and multi-seed robustness.

The pseudocode centers on repeated execution over folds and seeds. For each fold $$\mathcal{C}(c)=\mathbf{1}\bigl[\neg L(c)\wedge R(c)\wedge D(c)\wedge B(c)\bigr],$$1 and seed $$\mathcal{C}(c)=\mathbf{1}\bigl[\neg L(c)\wedge R(c)\wedge D(c)\wedge B(c)\bigr],$$2, the harness runs the candidate pipeline twice under the same conditions and flags "non-determinism" if the predictions differ. It computes $$\mathcal{C}(c)=\mathbf{1}\bigl[\neg L(c)\wedge R(c)\wedge D(c)\wedge B(c)\bigr],$$3, $$\mathcal{C}(c)=\mathbf{1}\bigl[\neg L(c)\wedge R(c)\wedge D(c)\wedge B(c)\bigr],$$4, and $$\mathcal{C}(c)=\mathbf{1}\bigl[\neg L(c)\wedge R(c)\wedge D(c)\wedge B(c)\bigr],$$5, checks leakage, monitors time and memory budgets, computes $$\mathcal{C}(c)=\mathbf{1}\bigl[\neg L(c)\wedge R(c)\wedge D(c)\wedge B(c)\bigr],$$6 and $$\mathcal{C}(c)=\mathbf{1}\bigl[\neg L(c)\wedge R(c)\wedge D(c)\wedge B(c)\bigr],$$7, and records "unstable-across-seeds" if $$\mathcal{C}(c)=\mathbf{1}\bigl[\neg L(c)\wedge R(c)\wedge D(c)\wedge B(c)\bigr],$$8 exceeds a robustness threshold (Ren et al., 23 Mar 2026). This turns reproducibility and reliability into first-class optimization constraints.

The evaluator returns both hard gates and soft signals. The hard outputs are pass/fail and explicit failure reasons such as leakage, nondeterminism, resource overrun, and seed-variance violation. The soft outputs are $$\mathcal{C}(c)=\mathbf{1}\bigl[\neg L(c)\wedge R(c)\wedge D(c)\wedge B(c)\bigr],$$9, $L(c)$0, MAE, RMSE, and complexity. These signals feed two downstream mechanisms: Constraint Memory and operator scoring. Constraint Memory stores failure causes so that future mutations known to induce those failures can be pruned; operator scoring uses fitness and failure patterns to favor productive mutation families and down-weight harmful ones (Ren et al., 23 Mar 2026).

The aerodynamic paper also specifies the metrics used by FAMOU. These include Spearman sign-ranking accuracy, seed-wise reliability defined as $L(c)$1, pointwise error via MAE and RMSE, a reported Combined Score

$L(c)$2

and the selection fitness

$L(c)$3

The paper states that the weights are tuned in pilot studies to reflect that ranking reliability is more valuable than raw error (Ren et al., 23 Mar 2026). This makes FAMOU explicitly aligned with ranking-centric design exploration rather than pure regression minimization.

4. Co-evolutionary evaluation in adversarial games

The adversarial-game version of FAMOU addresses a different pathology: evaluator drift in nonstationary competitive environments. The paper argues that "the evaluation landscape shifts as strategies improve, causing fixed evaluators to become unreliable and evolution to stagnate," and proposes three mechanisms: evaluator co-evolution, hierarchical deep evaluation, and weakness pressure (Li et al., 9 Jun 2026).

Evaluator co-evolution augments the opponent pool whenever a statistically confirmed new champion emerges. Formally, the evaluator is $L(c)$4, where $L(c)$5 is the opponent pool, $L(c)$6 the weight vector, and $L(c)$7 the weighted fitness. When a new champion $L(c)$8 is discovered, the pool is updated by $L(c)$9, and the new champion receives a high "gatekeeper" weight (Li et al., 9 Jun 2026). This forces future candidates to defeat not only initial seed bots but also the current frontier of discovered policies.

Weakness pressure changes the evaluator weights to emphasize the hardest opponent for the current champion. Let

$R(c)$0

The framework doubles the weight of this opponent and renormalizes the remaining weights so that $R(c)$1 (Li et al., 9 Jun 2026). The intended effect is to prevent progress from being measured mainly against easy baselines.

Hierarchical deep evaluation separates noisy ranking from champion confirmation. The fast phase uses 3 games per opponent for population ranking, while the deep phase uses 20 games per opponent for champion confirmation and benchmark checkpoints. The paper reports that early tests showed Spearman $R(c)$2 with $R(c)$3 against a 20-40 game baseline, motivating the shift away from shallow evaluation (Li et al., 9 Jun 2026). Deep-phase evaluation computes win rate with nonparametric bootstrap 95% confidence intervals via 10,000 resamples, average margin with Gaussian confidence intervals, Wilcoxon signed-rank tests on per-opponent win-rate differences, and paired $R(c)$4-tests on margin differences.

The overall loop starts from a seed strategy $R(c)$5 and an initial evaluator $R(c)$6, performs fast evaluation, parent selection via UCB, full-file LLM mutation, syntactic and API validation, and then champion detection followed by co-evolution and deep evaluation (Li et al., 9 Jun 2026). In this formulation, the evaluator is a dynamic participant in search rather than an immutable benchmark.

5. Feedback-guided mutation, islands, and adaptive sampling

In the aerodynamic setting, feedback is integrated with a larger population-based island model and adaptive sample selection. The paper describes multiple sub-populations evolving in parallel under different modeling theses, such as GNN versus MLP, with periodic elite migration across islands (Ren et al., 23 Mar 2026). FAMOU ensures that each island remains contract-compliant while allowing successful architectural motifs to propagate. Trajectory and ablation analyses identify adaptive sampling and island migration as primary drivers of convergence quality, and the best system reaches a Combined Score of 0.9335 with sign-accuracy 0.9180 (Ren et al., 23 Mar 2026).

Adaptive sampling is tied directly to evaluator feedback. Beyond fixed cross-validation folds, FAMOU tracks instance-level ranking errors and signals the agent to re-sample "hard" pairs where predictions flip sign. The stated purpose is to encourage data-pipeline mutations, including outlier handling and canonicalization tweaks, that reduce label noise (Ren et al., 23 Mar 2026). This implies that evaluation is not only selective but also curriculum-forming.

In the game framework, the island model is specified as 4 islands with 50 programs each, ring migration every 15 iterations, 2 migrants, and reset every 60 iterations (Li et al., 9 Jun 2026). The backbone LLMs are Gemini-2.5-Flash and DeepSeek-V4-Flash, with temperature $R(c)$7, max tokens $R(c)$8, and timeout 300 s. Mutation prompts ask the model to rewrite the entire file under styles such as "Structural redesign" and "Different algorithm," preserving EVOLVE-BLOCK markers and API compatibility (Li et al., 9 Jun 2026).

The game paper reports several emergent tactical modules that were absent from the seed strategies. These include H-DWA, which enumerates $R(c)$9 headings and simulates each for 1.5 s; A-Lock, which freezes role assignment under specified attack or pursuit conditions; and K-Filter, which estimates enemy acceleration using an EWMA update and predicts future positions for adaptive intercept (Li et al., 9 Jun 2026). The paper also notes SQRT-VDS repulsion scaling, T-DPH progress inertia, and A-DRL adaptive defense line. These examples are important because they show that FAMOU-style code evolution can synthesize algorithmic structures rather than only local heuristic adjustments.

6. Empirical findings, limitations, and nomenclature boundaries

The aerodynamic paper provides several illustrative cases that clarify how FAMOU behaves under failure and objective mismatch. In a leakage-detection case, a candidate adding an edge-attention block is assigned $D(c)$0 because the leakage detector finds a hidden dependency on solver metadata, after which that branch of code mutations is avoided in future generations (Ren et al., 23 Mar 2026). In a pairwise-ranking case, early generations stagnate under pure MSE regression; at iteration 34, FAMOU reports poor Spearman $D(c)$1 but good error, the agent infers "Objective mismatch," switches to a pairwise ranking loss, and the Combined Score jumps from 0 to 0.698. The ablation labeled "w/o Feedback" converges to 0.7782 average, compared with 0.8437 for the Full Method (Ren et al., 23 Mar 2026).

The adversarial-game paper reports that FAMOU "consistently outperforms both baselines under two backbone LLMs," achieving the highest combined score of 0.526 and the best generalization to unseen opponents, measured as a 61.7% win rate (Li et al., 9 Jun 2026). In the single-run ablations with the Gemini-2.5-Flash backbone over 400 iterations, the Full configuration attains 0.521, while variants without co-evolution, deep evaluation, weakness pressure, and all three mechanisms score 0.450, 0.385, 0.424, and 0.432, respectively. The paper states that removing hierarchical deep evaluation hurts most, with a $D(c)$2 drop (Li et al., 9 Jun 2026). In real-world validation, the DeepSeek-V4-Flash FAMOU champion places 1st in the hardware round-robin and 3rd in the simulation league at the AAMAS 2026 MCTF Competition.

A recurring limitation in both uses is that FAMOU depends on carefully engineered evaluation structure. In the aerodynamic setting, this takes the form of a hard evaluation contract with leakage prevention, deterministic replay, multi-seed robustness, and resource budgets; in the game setting, it takes the form of statistically reliable deep evaluation and co-evolving opponent pools (Ren et al., 23 Mar 2026, Li et al., 9 Jun 2026). This suggests that FAMOU is not a generic substitute for evaluation design; rather, it relocates design effort from manual model specification toward evaluator specification, failure taxonomy, and mutation governance.

The term should also be distinguished from "FAMOUS," the unrelated system for "High-Fidelity Monocular 3D Human Digitization Using View Synthesis," which addresses monocular 3D human digitization via view synthesis and 2D fashion priors (Hema et al., 2024). The similarity in spelling does not indicate methodological identity. Within current arXiv usage, FAMOU refers specifically to evaluator-centered code evolution in the two settings described above.