Evolving higher-order synergies reveals a trade-off between stability and information integration capacity in complex systems (2401.14347v2)

Abstract: There has recently been an explosion of interest in how "higher-order" structures emerge in complex systems. This "emergent" organization has been found in a variety of natural and artificial systems, although at present the field lacks a unified understanding of what the consequences of higher-order synergies and redundancies are for systems. Typical research treat the presence (or absence) of synergistic information as a dependent variable and report changes in the level of synergy in response to some change in the system. Here, we attempt to flip the script: rather than treating higher-order information as a dependent variable, we use evolutionary optimization to evolve boolean networks with significant higher-order redundancies, synergies, or statistical complexity. We then analyse these evolved populations of networks using established tools for characterizing discrete dynamics: the number of attractors, average transient length, and Derrida coefficient. We also assess the capacity of the systems to integrate information. We find that high-synergy systems are unstable and chaotic, but with a high capacity to integrate information. In contrast, evolved redundant systems are extremely stable, but have negligible capacity to integrate information. Finally, the complex systems that balance integration and segregation (known as Tononi-Sporns-Edelman complexity) show features of both chaosticity and stability, with a greater capacity to integrate information than the redundant systems while being more stable than the random and synergistic systems. We conclude that there may be a fundamental trade-off between the robustness of a systems dynamics and its capacity to integrate information (which inherently requires flexibility and sensitivity), and that certain kinds of complexity naturally balance this trade-off.

• The paper reveals that highly synergistic networks evolved via Boolean network optimization achieve significant integration capacity at the cost of chaotic dynamics and sensitivity to perturbations.
• The study shows that redundant networks offer enhanced stability through low entropy and fewer unique attractors but limit overall information-processing capacity.
• It demonstrates that networks evolved for high TSE complexity strike a balance between sub-critical stability and improved computational integration.

Stability and Information Integration Trade-offs in Complex Systems

The paper "Evolving higher-order synergies reveals a trade-off between stability and information integration capacity in complex systems" presents a nuanced exploration of the interplay between stability and information processing in complex systems, particularly through the lens of higher-order information structures. This paper uses evolutionary optimization techniques applied to boolean networks to scrutinize how varying degrees of synergies and redundancies impact system dynamics and information integration capabilities.

The authors focus on three distinct information-sharing patterns: redundancy, synergy, and complexity, leveraging advanced information-theoretic measures like the O-information and Tononi-Sporns-Edelman (TSE) complexity. The O-information offers insights into whether a system's information structure is dominated by redundancy or synergy, while TSE complexity quantifies how the system balances independence and integration.

By employing evolutionary optimization on boolean networks, three classes of networks were evolved, each favoring a different information structure: high redundancy, high synergy, or high complexity. The paper revealed that highly synergistic networks displayed chaotic dynamics with marked sensitivity to perturbations and long transient times, akin to random boolean networks. These systems exhibited a high capacity for information integration, as indicated by significant values of the integrated information measure, $\Phi^{R}$.

Conversely, networks dominated by redundancy were notably stable, exhibiting fewer unique attractors, lower entropy, and robustness to perturbations. These systems displayed sub-critical dynamics and limited information-processing capacity. Meanwhile, networks evolved for high TSE complexity exhibited a middle ground between these extremes, balancing stability with a degree of flexibility conducive to non-trivial information integration.

The findings of this paper imply a fundamental trade-off between dynamical stability and the ability to integrate information, echoing similar trade-offs observed in supply chain logistics. Redundant systems, while stable, lack computational capacities seen in more synergistic, chaotic systems. On the other hand, maximizing information integration appears contingent on accepting a degree of chaotic dynamics.

This research offers significant implications for the design and understanding of complex systems, such as neural architectures. The ability of high TSE-complex networks to blend stability with computational integration suggests parallels with the organization of biological brains. Furthermore, the identified link between randomness and synergy in this context could shed light on previously observed complexities in natural systems and present avenues for designing artificial systems with desired properties.

Future research could expand upon these findings by investigating larger and more intricate systems, potentially exploring implications in machine learning models and synthetic biology. Understanding the trade-offs highlighted in this paper could lead to the development of more adaptable and efficient complex systems, capable of maintaining a balance between stability and computational efficacy.