A Sensitivity Analysis of Cellular Automata and Heterogeneous Topology Networks: Partially-Local Cellular Automata and Homogeneous Homogeneous Random Boolean Networks (2407.18017v1)

Abstract: Elementary Cellular Automata (ECA) are a well-studied computational universe that is, despite its simple configurations, capable of impressive computational variety. Harvesting this computation in a useful way has historically shown itself to be difficult, but if combined with reservoir computing (RC), this becomes much more feasible. Furthermore, RC and ECA enable energy-efficient AI, making the combination a promising concept for Edge AI. In this work, we contrast ECA to substrates of Partially-Local CA (PLCA) and Homogeneous Homogeneous Random Boolean Networks (HHRBN). They are, in comparison, the topological heterogeneous counterparts of ECA. This represents a step from ECA towards more biological-plausible substrates. We analyse these substrates by testing on an RC benchmark (5-bit memory), using Temporal Derrida plots to estimate the sensitivity and assess the defect collapse rate. We find that, counterintuitively, disordered topology does not necessarily mean disordered computation. There are countering computational "forces" of topology imperfections leading to a higher collapse rate (order) and yet, if accounted for, an increased sensitivity to the initial condition. These observations together suggest a shrinking critical range.

• The paper demonstrates that ECA outperforms PLCA and HHRBN with superior memory retention and lower collapse rates on the 5-bit memory benchmark.
• Temporal Derrida plots and cycle analysis reveal that increased sensitivity in PLCA and HHRBN correlates with more intense attractor dynamics and reduced cycle lengths.
• The findings suggest that optimizing reservoir computing models can benefit from leveraging ECA’s robust computational stability over more sensitive heterogeneous networks.

Sensitivity Analysis of Cellular Automata and Heterogeneous Topology Networks

Summary

This paper explores the sensitivity analysis of Cellular Automata (CA) and Heterogeneous Topology Networks, posited as Partially-Local Cellular Automata (PLCA) and Homogeneous Homogeneous Random Boolean Networks (HHRBN). It compares these models' computational behavior and performance within the framework of Reservoir Computing (RC), specifically targeting the 5-bit memory benchmark.

The paper evaluates Elementary Cellular Automata (ECA), PLCA, and HHRBN across a variety of benchmarks and analyses to derive insights on their computational efficacy and sensitivity. The findings underscore an intriguing contrast: while ECA exhibits strong memory retention properties with reduced sensitivity to perturbations, PLCA and HHRBN show increased sensitivity but higher collapse rates, indicating a nuanced relationship between topology and computation.

Key Findings

1. 5-bit Memory Benchmark:
   • ECA outperforms PLCA and HHRBN, displaying superior memory retention and overall performance.
   • The collapse rate, defined as the propensity of a network to revert to an attractor state post perturbation, increases significantly from 27.8% in ECA to 53.7% and 59.7% in PLCA and HHRBN, respectively. This signifies that despite increased topological disorder, ECA maintains more stable computational behavior.
2. Temporal Derrida Plots (TDP):
   • CA systems, like rule 30, maintain a solid "chaotic" behavior profile, while more complex rules (54, 110) show extended sensitivity dynamics.
   • In PLCA and HHRBN, although sensitivity increases, the collapsing defect collapses suggest intense attractor dynamics leading to ordered behavior counterintuitively.
   • The distances post perturbation reveal critical insights into the erosion of computational integrity under increased heterogeneity.
3. Longest Simple Cycle Analysis:
   • PLCA and HHRBN networks exhibit significantly shorter longest simple cycles compared to ECA, directly impacting the system's theoretical memory capacity and practical information processing ability.
   • For instance, the average longest simple cycle for a 3-in-degree random network is below 150 steps for an $N$ of 160, starkly shorter than the 160 steps (equal to $N$) expected in ECA.
4. Sensitivity and "Chaos":
   • The paper explores fully discrete systems’ sensitivity and defines a working model for chaotic behavior suited to CA, PLCA, and HHRBN. It emphasizes that dense attractors and periodic data cycling serve as key markers of "chaotic" analogs in discrete systems.

Theoretical and Practical Implications

1. Algorithmic Design and Computational Models:
   • The recognition of nuanced behavior in PLCA and HHRBN can guide the development of more energy-efficient algorithms for Edge AI, particularly when reservoir computing is leveraged.
   • ECA's robustness underscores its suitability over PLCA and HHRBN in implementing hardware and FPGA-based reservoir computing, promising both reliability and computational stability.
2. Network Topology in Biological Systems:
   • This research lays the groundwork for understanding complex biological neural networks (BNN) as intermediate substrates between CA and complete BNNs. It posits that PLCA and HHRBN, with their increased sensitivity, mimic certain aspects of biological computation more closely due to their heterogeneity.
3. Future Research Directions:
   • Exploration of mixed-rule CA or other forms of heterogeneity can unveil additional insights into the critical computational range and variabilities within RC frameworks.
   • Applying evolved or real biological networks as substrates could further elucidate intrinsic computational principles inherent in natural systems.

Conclusion

This paper intricately explores the computational dynamics of ECA, PLCA, and HHRBN within RC frameworks. By providing robust numerical and theoretical analyses, it emphasizes the delicate balance and intricate relationship between network topology, sensitivity, and computational performance. While ECA exhibits superior computational stability, PLCA and HHRBN reveal increased sensitivity coupled with higher collapse rates, signaling a complex interplay between topological disorder and computational rigidity. These insights pave the way for future studies into intermediate substrates and optimization of RC for advanced AI applications.