Self-Organization to the Edge of Ergodicity Breaking in a Complex Adaptive System

Abstract: Self-organized criticality (SOC) is widely proposed as a fundamental mechanism for collective behavior, yet its role in objective-driven, heterogeneous adaptive systems underpinning real complex systems remains less understood. We introduce EvoSK, a minimal evolutionary model in which agents perform memory dependent reinforcement learning on a rugged Sherrington-Kirkpatrick landscape while the population evolves through extremal replacement of the least fit agents. We demonstrate that this coupled dynamics drives the system to a critical state residing on the transition boundary between ergodic and non-ergodic phases. At this boundary, the system exhibits scale-free evolutionary avalanches with a mean-field exponent $τ\approx -1.5$, while simultaneously achieving collective rewards that surpass those of any manually finetuned, non-evolutionary regime. Our results provide a mechanistic link between the statistical physics of ergodicity breaking and the functional optimality of complex adaptive systems, suggesting that the edge of ergodicity breaking acts as a robust attractor for systems adapting on rugged, high-dimensional landscapes.

• The paper introduces EvoSK, a novel model coupling reinforcement learning with Darwinian dynamics to achieve collective optimality.
• It empirically demonstrates that intermediate memory parameters yield maximum rewards and scale-free avalanche size distributions.
• Findings reveal that endogenous adaptation drives the system to the edge of ergodicity breaking, linking critical phenomena to performance.

Self-Organization to the Edge of Ergodicity Breaking in Complex Adaptive Systems

Introduction and Theoretical Framework

This paper introduces and analyzes the EvoSK model—a minimal yet highly nontrivial evolutionary framework capturing the interplay between heterogeneous reinforcement learning, many-body random interactions, and extremal replacement-driven Darwinian dynamics. Unlike classical SOC models, which tend to neglect explicit optimization, adaptive learning, or heterogeneous objectives, EvoSK establishes a setting where individual agents execute memory-dependent learning on a mean-field spin glass, while population-level evolutionary selection drives the dynamical regime.

The model extensions to the Bak-Sneppen paradigm are substantive: agents interact via Sherrington-Kirkpatrick (SK) couplings with nontrivial energy landscapes and independently adjust their choice randomness (temperature parameter) via evolution. The key control parameter is the reinforcement learning memory $\alpha$, which mediates the timescale and type of adaptivity each agent exhibits. The resulting population maintains a heterogeneous distribution over exploration parameter, which itself is endogenously shaped by extremal replacement (removal of the least-fit agent).

Model Specification

The EvoSK model instantiates $N$ agents, each selecting binary actions $\sigma_i \in \{-1, +1\}$ at every discrete time step. Interactions are fully random, modeled as a symmetric SK coupling $J_{ij}$ with mean-field normalization. Agents update their $q$-value estimates via a temporal-difference RL update with memory parameter $\alpha$ and select actions according to softmax probabilities parameterized by temperature $T_i = \beta_i^{-1}$.

At each evolutionary epoch, the agent with minimal fitness—operationalized as the $q$-value of its currently selected action—is removed and replaced by a new agent instantiated with a uniform random initial temperature and zeroed $q$-values. This procedure drives endogenous adaptation of both behavioral and evolutionary timescales, allowing for dynamic shaping of exploration-exploitation tradeoffs across the population.

This design fundamentally links reinforcement learning on rugged landscapes (incorporating local adaptation and slow plasticity) with a global selection mechanism that continuously enforces population-level adaptation and diversity in behavioral parameters.

Emergent Optimality and Population-Level Performance

A central claim substantiated in this work is that the EvoSK system self-organizes to a regime of collective reward maximization, and crucially, that this regime is neither attainable by fixed-parameter baselines nor by non-evolutionary RL with post-hoc temperature assignments. Empirically, the collective reward $\langle R \rangle$ achieves a broad optimal plateau for memory strengths $N$0. This plateau indicates robust insensitivity to fine-tuning, with the system outperforming both homogeneous and fixed heterogeneous baselines.

Figure 1: Steady-state rewards for EvoSK as a function of memory $N$1, benchmarking against non-evolutionary SK with uniform or matched temperature distributions.

Notably, even when the standard SK model is initialized using the empirically observed stationary temperature distribution $N$2 from EvoSK, it fails to replicate the performance gains seen under coupled evolution and selection. This demonstrates that the evolutionary rewiring of agent-level temperatures and the induced macrodynamics jointly produce an optimality unattainable by static parameter assignment.

Spontaneous Organization to the Edge of Ergodicity

A primary theoretical advance of the paper is the direct measurement and identification of the ergodicity breaking transition in the collective macrodynamics. The key diagnostic used is the total variation-based ergodicity coefficient $N$3, evaluated across rescalings of the stationary agent temperature ensemble. The approach is to infer a phase diagram by perturbing temperatures and measuring resilience or divergence of reward distributions over independent runs.

For intermediate values of $N$4—corresponding to the reward-maximizing regime—the system places itself precisely at the edge of ergodicity breaking: that is, $N$5 values at the critical point between ergodic ($N$6) and non-ergodic ($N$7) macrostates. The evolutionary dynamics robustly drive the population to this self-tuning edge without explicit tuning, and this is not an artifact of initialization.

Figure 2: Ergodicity measure $N$8 as a function of temperature rescaling for multiple memory strengths, showing self-organization to the ergodicity-breaking boundary for optimal $N$9.

This spontaneous criticality aligns the system with the phase transition in spin-glass theory, giving a mechanistic link between adaptive evolutionary dynamics, reinforcement learning, and nonequilibrium statistical physics.

Avalanche Dynamics and Scale-Free Cascades

The second canonical feature of SOC, demonstrably present here, is the emergence of scale-free avalanches. Avalanches are defined through cascades of agents with fitness below threshold during evolutionary epochs, and their size distributions are collected for various values of $\sigma_i \in \{-1, +1\}$0 and $\sigma_i \in \{-1, +1\}$1.

In the critical regime ($\sigma_i \in \{-1, +1\}$2), the distribution $\sigma_i \in \{-1, +1\}$3 exhibits robust power-law scaling over nearly four decades with exponent $\sigma_i \in \{-1, +1\}$4—the mean-field critical value observed in SOC and biological data such as neural avalanches. Outside this regime, the scaling breaks down, and distributions curve or terminate early, indicating the absence of scale-free behavior.

Figure 3: Avalanche size distributions in the steady state for $\sigma_i \in \{-1, +1\}$5, illustrating robust power law scaling and exponent $\sigma_i \in \{-1, +1\}$6 over broad size ranges.

The coincidence of the ergodicity edge and avalanche criticality is highly nontrivial, providing strong evidence that the evolutionary-adaptive dynamics induce the system toward universality classes associated with SOC, while at the same time producing functional macroscopic optimality.

Robustness and Phase Diagram

Supplementary analyses illuminate that—for extreme memory parameters ($\sigma_i \in \{-1, +1\}$7 or $\sigma_i \in \{-1, +1\}$8)—the system does not exhibit both collective optimality and critical scaling. At low memory (short-range adaptation), agents are trapped and only small avalanches occur; at high memory (hyper-exploration), criticality is lost and reward sharply deteriorates.

Figure 4: Avalanche statistics for $\sigma_i \in \{-1, +1\}$9 and $J_{ij}$0, illustrating breakdown of power-law scaling away from the ergodicity edge.

Implications and Prospects

The synthesis achieved in this work demonstrates a concrete mechanism by which complex adaptive systems with local learning, heterogeneity, and global selection pressures robustly self-organize to criticality, specifically to the edge of ergodicity breaking. These results yield several key implications:

• Functional-Statistical Link: The findings provide a mechanistic connection between functional optimality in adaptive, competitive environments and macroscopic criticality/ergodicity breaking as conceptualized in statistical physics.
• Self-Tuning Criticality: The approach does not require external fine-tuning of control parameters—criticality emerges as an attractor of endogenous evolutionary dynamics.
• Transferability: The minimal model and analytical methodology may generalize to a range of scenarios, including large-scale neural adaptation, multi-agent learning, and evolutionary computation where landscape ruggedness and agent heterogeneity dominate.
• Benchmarks and Simplicity: The model provides a bridge between the analytically tractable worlds of spin glasses and the empirical complexity of adaptive biological and AI systems, potentially serving as an experimental proxy for future investigations of criticality and optimization tradeoffs.

Investigation of related regimes (asymmetric couplings, alternate replacement protocols, or hierarchical selection) offers promising theoretical avenues to test the universality of these principles and their domain of applicability.

Conclusion

The EvoSK model establishes that evolutionary adaptation, when combined with reinforcement learning in highly heterogeneous environments, can autonomously drive a system to the edge of ergodicity breaking—the locus of both critical macroscopic fluctuations and collective optimality. The approach provides a powerful mechanism for linking microscopic adaptive behavior to emergent criticality, with broad applicability to biological, socio-economic, and large-scale machine learning systems. These results both unify and extend existing theories of SOC, ergodicity, and adaptive computation, offering foundational insights for future research in complex adaptive systems.