Hierarchical Cellular Automata: EngramNCA

• Hierarchical Cellular Automata (EngramNCA) are neural extensions that combine visible state updates with private memory channels to facilitate decentralized morphogenesis.
• The GeneCA module uses convolution-based perception and an MLP to update visible states while keeping the private gene memory unchanged.
• The GenePropCA module propagates private memory via context-sensitive gating, enabling the assembly of multi-scale, complex structures and improved stability.

Hierarchical Cellular Automata (EngramNCA) are a neural extension of classical cellular automata (CA) frameworks designed to support the emergence and transfer of complex, multi-scale morphologies via both visible and hidden, cell-internal memory channels. The EngramNCA paradigm is motivated by biological evidence that memory storage may rely not only on synaptic modifications but also on intracellular mechanisms. This architecture operationalizes distinct, decentralized memory layers in artificial self-organizing systems and provides a computational model for memory engram hypotheses (Guichard et al., 16 Apr 2025).

1. Model Architecture and State Representation

EngramNCA operates on a fixed 2D grid of size $H \times W$. Each cell $i$ at time $t$ possesses a composite state comprising:

• A publicly visible state $x_i^t \in \mathbb{R}^{d_x}$, where
  • $x_i^t = [v_i^t; h_i^t]$
  • $v_i^t \in \mathbb{R}^4$ are RGBA channels (observable as rendered images)
  • $h_i^t \in \mathbb{R}^{n_h}$ are “public hidden” channels
• A private, cell-internal memory or gene vector $m_i^t \in \mathbb{R}^{n_g}$, which is not visible to neighboring cells.

Typical experimental settings use a total state width $N=16$, with $n_h=4$ and $i$0, so $i$1, $i$2.

2. Update Rules: GeneCA and GenePropCA Modules

EngramNCA is defined as the serial composition of two neural cellular automaton modules at each time step:

GeneCA

GeneCA is responsible for the local growth of morphogenetic primitives. Cells update only their visible state $i$3, holding the private gene memory $i$4 constant.

• Each cell computes a local perception vector $i$5 using four linear convolutions (Identity, Sobel$i$6, Sobel$i$7, Laplacian) on the visible state $i$8.
• The cell state update is governed by a multilayer perceptron (MLP) parameterized by $i$9:

$t$0

$t$1

$t$2

where $t$3 are masks for asynchronous updating and cell aliveness.

GenePropCA

GenePropCA modulates and propagates the private memory $t$4 while the visible state $t$5 is held unchanged. The same perception $t$6 is used as input.

• The memory update is parameterized by $t$7:

$t$8

$t$9

$x_i^t \in \mathbb{R}^{d_x}$0

Ensemble Update Step

At each discrete time step $x_i^t \in \mathbb{R}^{d_x}$1, the model applies these modules in sequence:

1. $x_i^t \in \mathbb{R}^{d_x}$2
2. $x_i^t \in \mathbb{R}^{d_x}$3

3. Hierarchical Morphogenetic Behavior

EngramNCA supports spatial and scale hierarchies through the interplay of primitive growth and memory propagation:

• Primitive Instantiation (Level 1): GeneCA alone can robustly generate $x_i^t \in \mathbb{R}^{d_x}$4 distinct primitive shapes from isolated seeds, each differentiated by a unique binary gene code for $x_i^t \in \mathbb{R}^{d_x}$5.
• Landmark-Gated Gene Propagation (Level 2+): GenePropCA leverages visible morphologies to gate and direct the local propagation, mixing, or rewrite of the gene vector across the grid. Importantly, the private $x_i^t \in \mathbb{R}^{d_x}$6 vectors are never directly shared; they are instead conditionally rewritten based on patterns in $x_i^t \in \mathbb{R}^{d_x}$7, yielding decentralized and context-sensitive memory transfer.
• Multi-scale Assembly: Composing multiple rounds of (GeneCA $x_i^t \in \mathbb{R}^{d_x}$8 GenePropCA) enables the emergence of nested structures—e.g., merging primitives into articulated assemblages (such as a lizard from constituent parts or a fractal from line segments)—with additional meta-gene channels supporting coexisting, interleaved morphologies.

The formal evolution equations for the two channels are:

$x_i^t \in \mathbb{R}^{d_x}$9

$x_i^t = [v_i^t; h_i^t]$0

4. Training Objectives and Protocols

EngramNCA adopts a staged training procedure:

• GeneCA Pretraining: Trained on a pool of $x_i^t = [v_i^t; h_i^t]$1 primitive targets, with each run seeded by the corresponding binary code in $x_i^t = [v_i^t; h_i^t]$2 and optimized via pixelwise MSE on visible RGBA parts after $x_i^t = [v_i^t; h_i^t]$3 CA steps:

$x_i^t = [v_i^t; h_i^t]$4

• GenePropCA Training: After freezing GeneCA weights, GenePropCA is trained, with the GeneCA module running in a no-grad (fixed) mode, to assemble full morphological targets from previously learned primitives, using the same $x_i^t = [v_i^t; h_i^t]$5 loss.

No regularization on $x_i^t = [v_i^t; h_i^t]$6 (e.g., $x_i^t = [v_i^t; h_i^t]$7 or sparsity) is employed in the base implementation, though regularizations such as $x_i^t = [v_i^t; h_i^t]$8 or $x_i^t = [v_i^t; h_i^t]$9 are acknowledged as potential avenues for future experimentation.

Minibatch training mixes $v_i^t \in \mathbb{R}^4$0 gene codes per batch, and initialization assigns a single seed cell with $v_i^t \in \mathbb{R}^4$1 set to the binary gene code $v_i^t \in \mathbb{R}^4$2, all other cells starting with $v_i^t \in \mathbb{R}^4$3 and dead $v_i^t \in \mathbb{R}^4$4 channels.

5. Experimental Results and Ablations

• Scalability and Stability: GeneCA alone can reliably grow up to $v_i^t \in \mathbb{R}^4$5 non-interfering, coexisting primitives, including complex shapes such as lizard body parts and basic geometric forms.
• Composite Morphogenesis: GenePropCA enables the assembly of these primitives into higher-order morphologies—e.g., lizard torsos acquiring limbs, or lines forming deterministic fractals. Gene channel bit mixing not seen at train time generates novel, compositional forms.
• Meta-gene Switching: A single added meta-gene channel enables the same set of primitives to produce entirely distinct full morphologies (e.g., switching between lizard and butterfly-shaped composites).
• Performance Benchmarks: In tasks involving moving patterns, such as Lenia gliders, the combined EngramNCA architecture produces lower per-frame MSE, enhanced long-term spatiotemporal stability, and more coherent dynamics compared to conventional NCA baselines with the same parameterization.

Ablation studies demonstrate that privatization of additional channels leads to a smooth degradation in morphogenetic performance, but stable growth and recovery persist up to approximately $v_i^t \in \mathbb{R}^4$6 privatized channels (Guichard et al., 16 Apr 2025).

6. Theoretical Context: Connections to Hierarchical CAs

The hierarchical nature of EngramNCA's morphogenesis aligns with formal results in the study of number-conserving cellular automata (NCCA), in which particle moves and CA rules of neighborhood size $v_i^t \in \mathbb{R}^4$7 can be decomposed hierarchically into CA rules of successively smaller neighborhoods (Kong et al., 2019). In 1D NCCAs, the pattern sets $v_i^t \in \mathbb{R}^4$8 corresponding to value-1 states in the update rule $v_i^t \in \mathbb{R}^4$9 can be recursively bundled to form pattern sets of lower order, yielding a binary “bundle-tree” structure that organizes local dynamics at multiple scales.

A direct implication is that hierarchical CA frameworks such as EngramNCA may benefit from multiscale decompositions analogous to those employed in NCCAs, supporting memory stratification and context-dependent transfer. The use of private gene channels as explicit memory substrates, combined with context-sensitive gene propagation via landmark-morphology gating, provides an artificial analog to multi-level motion representations and hierarchical organization as studied in (Kong et al., 2019).

7. Significance and Implications

EngramNCA demonstrates that the integration of private, localized memory channels with modular propagation mechanisms enables robust, hierarchical, and coexisting morphogenetic processes that are difficult to realize in monolithic neural CA architectures. This model provides a computational substrate for exploring decentralized memory mechanisms parallel to biological engram theories—specifically, RNA-mediated memory transfer and intracellular memory engrams. The results suggest new methodologies for constructing adaptive, self-organizing artificial systems with stable distributed memory and dynamic recombination capabilities (Guichard et al., 16 Apr 2025).

A plausible implication is that combining hierarchical CA structures with explicit multi-scale memory propagation architectures may offer scalable blueprints for adaptive morphogenesis and decentralized computation in synthetic cellular systems, drawing on both formal CA hierarchy results and generative neural modeling principles.