Scale-Dependent Collective Adaptation in Self-Amending LLM Societies: A Cross-Family Study of Emergent Governance

Abstract: We study group decision-making in artificial societies where the rules of play are themselves subject to collective amendment. Using the self-amending game Nomic, we compare multiple scales across two LLM families and find that collective adaptation does not improve monotonically with model size. Instead, both families exhibit a narrow mid-scale regime that supports sustained rule adoption, diverse amendments, and balanced consensus. Smaller models tend to remain rule-inert, whereas larger models often converge on restrictive voting patterns, and heterogeneous mixed-size groups collapse into veto-driven gridlock. These cross-scale contrasts persist under temperature perturbations and under a shift from unanimity to majority voting, although latent-state structure varies by family and scale. Hidden-state divergence alone does not explain collective performance: high representational divergence can coincide with poor behavioural outcomes. Linear probes reveal regime-selective coupling between latent vote-predictive signals and collective behaviour, but decodability is necessary rather than sufficient for adaptive play. Overall, the recurring regularity is non-monotonicity, not the particular scale at which the optimum appears. Self-amending games therefore provide a controlled testbed for studying collective adaptation in artificial societies beyond raw model scale.

• The paper demonstrates that collective adaptation in self-amending LLM societies emerges only at an intermediate scale, with distinct failure modes at smaller and larger scales.
• It employs structured simulations of rule-rewriting games across two LLM families to quantitatively measure proposal adoption, voting dynamics, and institutional change.
• Robust analysis shows that mid-scale regimes uniquely leverage internal vote-predictive decodability to catalyze effective, balanced governance.

Scale-Dependent Non-Monotonicity in Collective Adaptation of Self-Amending LLM Societies

Introduction

The paper "Scale-Dependent Collective Adaptation in Self-Amending LLM Societies: A Cross-Family Study of Emergent Governance" (2605.17510) provides a systematic, quantitative, and mechanistic analysis of how collective adaptation emerges in artificial agent societies powered by LLMs when institutional rules are objects of endogenous amendment. The study leverages the structured rule-rewriting game Nomic, parameter sweeps across two diverse LLM families (Qwen3.5 and Gemma3), and a suite of process-level, dynamical, and representation-focused measurements. The results challenge monotonic expectations for scaling, compellingly demonstrating that adaptive, varied institutional change manifests only in a regime-local "sweet spot" at intermediate model scales, with both smaller and larger models failing in distinct and predictable modes.

Experimental Paradigm: Self-Amending Societies with LLM Agents

The core methodology involves simulating agentic societies where each agent is an identical copy of an LLM checkpoint of a given scale. Five such agents play a structured variant of Nomic, in which at each turn, the acting agent proposes a rule change (ADD/AMEND/REPEAL/TRANSMUTE), other agents vote, and successful proposals directly amend the institution. The institutional rules are expressed in natural language and a formal patchable engine field. Dedicated validation separates executable and constitutional invalidity from socially-rejected proposals.

The experimental protocol controls for family, model scale (five per family: 0.8B → 27B for Qwen3.5; 270M → 27B for Gemma3), input prompts, and sampling stochasticity (primary: temperature 0.7, extended sweeps 0.1–1.3), and investigates both homogeneous and mixed-scale populations. Endpoints tracked include proposal adoption rates, rule set evolution, change-type diversity, game resolution, voting modes, and internal representational probes.

Core Findings: Non-Monotonic Adaptation with Scale

A key contribution is the robust identification and cross-family replication of a non-monotonic scaling regime in collective adaptation.

Phase Portrait & Sweet Spot: Both Qwen3.5 and Gemma3 display a single intermediate scale (Qwen3.5 4B, Gemma3 12B) that supports sustained, diverse, and coordinated institutional change—quantified by adoption rates (33.8%/53.9%), dynamic active-rule expansion, broad move-type entropy, and high winner rates (8/15 and 3/15 trials, respectively). Smaller models are almost entirely rule-inert, generating proposals that are technically valid but almost never adopted. Larger models, although capable of technically precise proposals, converge to veto-heavy/narrow-AMEND behaviors, resulting in institutional stasis or oscillation, and minimal resolution via point accumulation.

Figure 1: Cross-family phase portraits summarizing the emergence and scale-shift of the sweet-spot regime across Qwen3.5 and Gemma3.

Robustness Across Sampling and Institutional Perturbations

To exclude confounds, the study varies (i) output sampling temperature and (ii) the social institution (unanimity-to-majority voting versus always-majority). The regime distinctions—freezing/failure at small scale, exploratory sweet spot at mid-scale, veto-dominated stasis at large scale—are invariant to these manipulations.

• For Qwen3.5: the 4B regime remains the unique adaptive zone under all perturbations, and response curves retain their qualitative scale ordering.
• For Gemma3: the 12B regime both persists and strengthens with temperature increase (winner rate increases from 0/6 to 4/6 between $T \leq 0.5$ and $T \geq 1.1$).

  Figure 2: Adoption and winner rates across temperatures, showing sweet-spot persistence and strengthening under temperature increases, especially in Gemma3.

Similarly, shifting to majority voting from the start does not eliminate the regime distinctions or unlock adaptation at frozen scales; all critical patterns survive the intervention.

Figure 3: Rule adoption and acceptance rates under baseline and majority voting, demonstrating robustness of regime taxonomy across institutional changes.

Collective Trajectories: Dynamic and Mechanistic Characterization

Dynamic analysis reveals that not only endpoints but trajectories diverge by scale-regime:

• Sweet spots exhibit directed exploration then stabilization of key institutional parameters (e.g., penalty settings in Rule 206).
• Largest models exhibit oscillatory regime—cycling repeatedly through prior parameter values with high reversal rates ($\sim$79%), indicating a failure to stabilize or coordinate preferences.
• At the rule-structure level, sweet-spot models expand the set of mutable rules in a controlled manner, whereas both frozen and large-scale regimes either leave the constitution inert or concentrate narrowly on parameter retuning.

  Figure 4: Parameter-trajectory comparison between sweet-spot (4B) and large-scale (27B) Qwen3.5 models, highlighting exploratory versus oscillatory control.

  

  Figure 5: Evolution of mutable/immutable rule counts per scale, underlining distinct legislative growth at the sweet spot.

Mechanistic Analysis: Internal Representations and Voting Information Flow

Hidden-state probing is performed on model activations captured during matched baseline and institutional-intervention runs. Two main findings emerge:

• Decodability of Institutional Perturbation: Condition (baseline vs. intervention) is linearly decodable at almost all layers and scales, indicating that LLMs almost always encode institutional change.
• Vote-Prediction Decodability: Only in sweet-spot regimes (Qwen3.5 4B, Gemma3 12B) does vote-predictive information become linearly decodable at mid/intermediate layers ($+17.1$/$+18.3$ percentage points over majority baseline), and coincide with balanced (YES/NO) downstream voting. At veto-heavy scales, similar or larger probe margins are found only in shallow layers, and do not translate into adaptive, diverse collective behavior. Decodability is necessary but not sufficient for adaptation.

  Figure 6: Layerwise condition and vote-probe accuracy, with pronounced sweet-spot selectivity at intermediate layers for vote prediction.

• Cosine Divergence Dissociation: High mean internal divergence (e.g., Gemma3 1B) does not predict adaptive behavior—large representational change can coincide with stagnation or gridlock, providing a negative result for magnitude-based adaptation hypotheses.

  Figure 7: Mean cosine distance between matched hidden states under intervention, showing magnitude–behavior dissociation.

Failure Modes and Heterogeneous Society Dynamics

The study details qualitatively distinct failure signatures:

• Frozen regime: Small models generate self-defeating or contextually illogical proposals, and display voting inconsistency—even proposers routinely vote against their own proposed improvements.
• Oscillatory regime: Large models alternate amendments, often contradicting even their own recently adopted changes, revealing a lack of institutional memory or coordination.
• Heterogeneous societies: Mixed-scale populations collapse into gridlock, dominated by veto behavior from the largest model—sweet-spot members are unable to sustain adaptive regimes in the presence of high-scale vetoes, and this effect is stronger in Gemma3.

Implications and Theoretical Synthesis

This work provides definitive evidence that collective adaptation in self-amending LLM societies is scale-dependent and non-monotonic, contrasting with the monotonic trends predicted by classical scaling laws for token-level objectives [kaplan2020scaling, hoffmann2022training]. The phenomenon is not an artefact of sampling stochasticity or particular voting rules, and is robust across major transformer families. There is no scalar metric of “capabilities” that aligns with the onset of varied institutional change; rather, adaptation emerges as a regime phenomenon associated with specific coupling of internal vote-predictive information and balanced, deliberative social behavior.

These results add to the growing evidence for inverse- or U-shaped scaling behavior in complex tasks [mckenzie2024inverse, wu2025ushaped], challenging naive assumptions that model improvements uniformly translate into more sophisticated or cooperative collectives. The locus and nature of sweet-spot regimes likely depend on architectural, tokenizer, and data factors, but the non-monotonic shape generalizes.

From a practical standpoint, these findings have strong implications for institutional design in LLM-mediated systems, suggesting that naive scaling may harm, not help, rule-amending or open-ended governance. Theoretically, they point toward new directions in mechanistic and sociotechnical analysis, including the investigation of probe-coupling patterns, cross-agent heterogeneity, and the effects of non-self-play social settings.

Conclusion

This study establishes that large-scale LLM societies engaging in self-amendment do not monotonicly improve with scale: collective adaptation emerges only in a regime-local sweet spot, with nontrivial, cross-validated failure modes present at both extremes. The mechanistic signature of adaptation is the co-occurrence of mid-layer vote information decodability and balanced downstream voting. These insights reinforce the necessity for targeted, regime-aware model selection and institutional engineering in the design of AI-based governance and agent societies. Self-amending games constitute an effective, controlled environment for fundamental inquiry into the emergence of complex social and adaptational phenomena in artificial collectives.