Digital Red Queen: Adversarial Program Evolution in Core War with LLMs

Abstract: LLMs are increasingly being used to evolve solutions to problems in many domains, in a process inspired by biological evolution. However, unlike biological evolution, most LLM-evolution frameworks are formulated as static optimization problems, overlooking the open-ended adversarial dynamics that characterize real-world evolutionary processes. Here, we study Digital Red Queen (DRQ), a simple self-play algorithm that embraces these so-called "Red Queen" dynamics via continual adaptation to a changing objective. DRQ uses an LLM to evolve assembly-like programs, called warriors, which compete against each other for control of a virtual machine in the game of Core War, a Turing-complete environment studied in artificial life and connected to cybersecurity. In each round of DRQ, the model evolves a new warrior to defeat all previous ones, producing a sequence of adapted warriors. Over many rounds, we observe that warriors become increasingly general (relative to a set of held-out human warriors). Interestingly, warriors also become less behaviorally diverse across independent runs, indicating a convergence pressure toward a general-purpose behavioral strategy, much like convergent evolution in nature. This result highlights a potential value of shifting from static objectives to dynamic Red Queen objectives. Our work positions Core War as a rich, controllable sandbox for studying adversarial adaptation in artificial systems and for evaluating LLM-based evolution methods. More broadly, the simplicity and effectiveness of DRQ suggest that similarly minimal self-play approaches could prove useful in other more practical multi-agent adversarial domains, like real-world cybersecurity or combating drug resistance.

• The paper introduces the Digital Red Queen algorithm that employs LLM-guided self-play to evolve assembly-like warriors in the Core War testbed.
• It demonstrates that iterative self-play with MAP-Elites diversity preservation drives convergence toward robust, generalist behavioral strategies while keeping genotype diversity high.
• The study reveals that historical self-play suppresses cyclic dynamics and enhances strategy robustness, offering insights applicable to adversarial multi-agent systems.

Digital Red Queen: LLM-Guided Adversarial Program Evolution in Core War

Introduction

"Digital Red Queen: Adversarial Program Evolution in Core War with LLMs" (2601.03335) investigates Red Queen evolutionary dynamics—iterative adaptation in adversarial settings—using LLMs to synthesize assembly-like programs, "warriors," that compete in the Turing-complete Core War environment. Unlike prevalent LLM-evolution frameworks relying on static fitness landscapes, this work positions itself in the context of continual coevolution. The primary contributions are (i) introducing the Digital Red Queen (DRQ) algorithm as a minimal self-play evolutionary loop powered by LLM-driven program synthesis, (ii) measuring statistical convergence toward generalist behavioral strategies under Red Queen pressure, and (iii) analyzing mechanisms of robustness, emergent convergence, and diversity persistence in an open-ended, adversarial artificial life testbed.

Algorithmic Framework

The DRQ algorithm is instantiated as a historical self-play loop: starting from an initial warrior $w_0$, each successive round $t$ uses an LLM-driven evolutionary process with MAP-Elites diversity preservation to generate a new warrior $w_t$ optimized to defeat all prior champions $\{w_0,\ldots,w_{t-1}\}$. Warrior fitness is determined via multi-agent Core War battles, and evolutionary optimization proceeds within each round using LLMs as mutation/generation operators. The choice of MAP-Elites ensures representation of diverse behavioral strategies and avoids premature convergence in genotype space.

This approach contrasts with policy-space response oracles (PSRO) and fictitious self-play (FSP) from deep RL, as it does not explicitly solve full meta-games or construct equilibrium mixtures, reflecting the practical constraints of program synthesis and the lack of an explicit action space. LLMs only function as black-box proposal operators, grounded by simulation-based selection.

Evaluation on Core War

Static Optimization Baseline

The study first benchmarks the performance of LLM-driven evolutionary search under a static objective (single round), where a specialist warrior is trained against a fixed human-designed opponent. As shown, such a strategy achieves high specialist performance—collectively defeating or matching $96.3\%$ of 294 human warriors, far exceeding zero-shot baselines:

Figure 1: Static optimization produces highly specialized warriors that collectively outperform human designs but remain brittle as individuals.

However, generalist performance collapses: any individual evolved warrior defeats or ties only $27.9\%$ of human warriors on average, demonstrating susceptibility to overfitting and lack of robustness.

Red Queen Dynamics and Convergence

Moving to full DRQ with iterated adversarial adaptation, the empirical dynamics reveal a monotonic increase in the generality of warrior behavior across rounds. Generality, defined as the fraction of unseen human warriors defeated or tied (zero-shot generalization), rises steadily, indicating selection for behavioral strategies robust to diverse threats.

Figure 2: DRQ warriors exhibit rising generality and statistical convergence in phenotype, while genotypic diversity is preserved.

Notably, phenotypic convergence emerges across independent runs; variance in behavioral performance (phenotype) decreases over time, indicating that the Red Queen loop steers populations toward a universal high-performance attractor in functional space. However, this is not mirrored at the genotypic level: the variance in code embeddings (genotypes) remains high, evidencing multiple genotypes mapping to similar behavioral optima—a pattern akin to convergent evolution in natural systems.

Figure 3: Phenotypic convergence is statistical and robust; genotypic convergence is absent, underscoring the existence of multiple codewise solutions for similar behaviors.

The convergence effect is detectable only statistically, averaging over independent DRQ runs. Full phenotypic fixation would require exponentially many rounds, implying that convergence acts as a weak but significant attractor under continual adversarial evolution.

Cyclic Dynamics Suppression

Core War possesses rich cyclic strategy interactions (e.g., rock-paper-scissors among bomber, replicator, and scanner archetypes). Incorporating a history of champions in the fitness function ($K>1$) reduces the emergence of simple cycles, quantified as triplets where $a$ beats $b$, $b$ beats $c$, and $c$ beats $a$. Increasing $K$ to 10 cuts cyclic dynamics by $77\%$, indicating that historical self-play stabilizes the evolutionary process and ensures progression toward generalist strategies.

Figure 4: History-aware DRQ significantly suppresses cyclic rock-paper-scissors dynamics compared to minimal history ($K=1$).

Mechanisms of Robust Behavior

Analysis of MAP-Elites archives uncovers that warriors with high numbers of spawned threads and low memory coverage attain the highest average fitness. This result aligns with known Core War heuristics—parallelism (through SPL opcode) confers robustness, as all threads must be terminated to eliminate a warrior.

Figure 5: High-performing behavioral niches involve low memory coverage and aggressive thread spawning via SPL instructions.

Additionally, warriors capable of fusing distinct strategies (replication and bombing) emerge, further supporting the algorithm's capacity to synthesize novel and effective behaviors.

Figure 6: Generated warriors demonstrate the ability to combine multiple high-level strategies and achieve superior win rates against human-designed baselines.

Role of Quality Diversity

Ablation of the diversity-preservation mechanism by collapsing MAP-Elites to a single cell leads to marked decreases in optimization performance, particularly in later rounds, affirming the necessity of maintaining behavioral diversity during search.

Figure 7: MAP-Elites diversity preservation yields substantial optimization gains versus single-cell evolutionary dynamics.

Predictability of Warrior Fitness

The mapping from warrior source code (genotype) to battle performance (phenotype) is highly non-linear and sensitive. Nonetheless, training a linear probe over text embeddings of warrior code yields moderate generality prediction ($R^2 \sim 0.46$), with little improvement from larger embedding models. The non-trivial predictability suggests possible avenues for accelerating evaluation cycles via surrogate modeling and for improved mechanistic interpretability of evolved programs.

Figure 8: The generality of warriors is moderately predictable from source code embeddings, though substantial variance remains unexplained.

Diversity and Expressivity

Despite behavioral convergence, visualizations confirm that DRQ discovers a broad range of qualitatively distinct strategies across the lineage.

Figure 9: A visualization of warrior-vs-warrior competition demonstrates persistent diversity of strategic behaviors among DRQ-generated programs.

Fitness curves across all runs show steady improvement, both in aggregate and individually.

Figure 10: Aggregate fitness consistently climbs across DRQ iterations, even as the rate of improvement diminishes in later stages (see also Figure 11 for per-run curves).

MAP-Elites archives sampled across rounds reveal diversity in discovered elites.

Figure 12: Selected MAP-Elites archives from DRQ runs highlight diversity in behavioral niches and associated performance.

Practical and Theoretical Implications

The DRQ findings indicate that extremely simple self-play, when paired with LLM-guided mutation and quality-diversity search, can produce robust, generalist strategies in complex, adversarial multi-agent environments. The use of the Turing-complete, yet fully sandboxed, Core War domain enables detailed study of evolutionary arms races relevant to real-world domains (e.g., cybersecurity, defense-evasion, agent-agent competition) without safety risks. The observed dissociation between phenotypic convergence (robust behaviors) and genotypic persistence (code base diversity) implies that LLM-evolved systems may independently converge toward similar solution patterns, even when grounded in distinct code bases—an important consideration for both red-teaming and robustness analysis.

Prospective extensions include parallel population coevolution, richer fitness landscapes, and application to more mission-critical domains where robust adversarial adaptation is requisite. Model-based surrogate fitness prediction and interpretability (as evidenced in the code-level generality regression experiments) offer practical means to enhance evolutionary search in high-cost environments.

Conclusion

This work demonstrates that historical self-play, LLM-guided mutation, and diversity maintenance are sufficient to instantiate Red Queen coevolutionary dynamics yielding robust generalist agents in the Core War testbed. The convergence to common behavioral strategies, amidst persistent genotypic diversity, mirrors fundamental principles of open-ended evolution. The methodologies and findings are immediately transferable to broader adversarial, multi-agent systems, laying groundwork for systematic investigation of evolutionary arms races, robustness, and high-level behavioral convergence in artificial life and beyond.