---
title: Parallelized Hierarchical Connectome (PHC)
url: https://www.emergentmind.com/topics/parallelized-hierarchical-connectome-phc
type: topic
---

# Parallelized Hierarchical Connectome (PHC)

Parallelized Hierarchical Connectome (PHC) is a general framework that upgrades temporal-only State-Space Models (SSMs) into spatiotemporal recurrent networks by combining diagonal SSM parallelism with explicit intra-slice neural communication [2604.01295]. In the formulation introduced with the PHCSSM instantiation, the diagonal SSM core is mapped to a shared Neuron Layer, inter-neuronal communication is mapped to a shared Synapse Layer, and neurons are partitioned into hierarchical regions governed by a connectome topology. A Multi-Transmission Loop enables spatial recurrence within each temporal window while preserving $O(\log T)$ parallelism, and the framework is presented as the first model to unify recurrent spiking neural network dynamics with diagonal SSM parallelism while enforcing all five biological constraints and learnable lateral connections within a fully parallelizable training pipeline [2604.01295].

## 1. Diagonal state-space substrate and parallel scan

PHC is defined against the background of the standard discrete-time linear SSM
$$
h_t = A h_{t-1} + B x_t,\qquad
y_t = C h_t + D x_t,
$$
where $h_t \in \mathbb{R}^D$ is the hidden state, $x_t \in \mathbb{R}^{d^{in}}$ is the input, and $A \in \mathbb{R}^{D \times D}$. Modern SSMs, including S4, Mamba, and S5 in the exposition, restrict $A$ to be diagonal,
$$
A = \operatorname{diag}(\alpha_1,\ldots,\alpha_D),
$$
so that the state update decouples across dimensions:
$$
h_t^{(i)} = \alpha_i h_{t-1}^{(i)} + [B x_t]^{(i)},\qquad i=1,\ldots,D.
$$

This diagonal Jacobian form admits an $O(\log T)$ parallel-scan solution of the leaky-integrator recurrence
$$
y_t = \alpha y_{t-1} + f_t
$$
via the log-domain prefix-sum trick. Letting
$$
S_t = \sum_{k=1}^t \log \alpha_k
$$
and enforcing $f_t > 0$, the exposition gives
$$
y_t = \exp\bigl(\operatorname{logcumsumexp}(\log f - S)[t] + S_t\bigr) + h_0 \exp(S_t).
$$
Here, $\operatorname{logcumsumexp}$ is the inclusive prefix scan of its argument, implementable in $O(\log T)$ depth via a Hillis–Steele algorithm.

The central limitation identified by PHC is that conventional diagonal SSMs retain only temporal recurrence. They do not natively provide lateral or feedback interactions within a single timestep. PHC therefore preserves the diagonal recurrence that supports scan-based parallelism, but augments it with structured spatial communication. A common misconception is that such augmentation must reintroduce full temporal serialization; in the PHC formulation, that is explicitly not the case, because the temporal recurrences remain scan-solvable.

## 2. Decomposition into Neuron Layer and Synapse Layer

PHC decomposes the standard diagonal-SSM state transition into two half-steps at each transmission. The Neuron Layer (NL) contains purely diagonal intrinsic dynamics, including per-neuron leaky integration, adaptation, and refractory behavior. The Synapse Layer (SL) contains all inter-neuron communication, including lateral, feedforward, and feedback transmission, mediated by a shared weight matrix.

The notation assumes a batch of size $B$, sequence length $T$, neuron dimension $D$, and $R$ hierarchical regions. Neuron indices are $i,j \in \{1,\ldots,D\}$, region indices are $r,r' \in \{1,\ldots,R\}$, and $D_r$ denotes the number of neurons in region $r$. The variables $x_{t,i}^{(k)}$, $s_{t,i}^{(k)}$, $V_{f,t,i}$, $\theta_{adapt,t,i}$, $V_{res,t,i}$, $u_{t,i}$, and $R_{t,i}$ parameterize the neuronal and synaptic states, while $W_{syn} \in \mathbb{R}^{D \times D}$ is the unconstrained synaptic weight matrix, $M_{topo} \in \{0,1\}^{D \times D}$ is the binary topology mask, and
$$
W_{struct} = W_{syn} \odot M_{topo}
$$
is the effective structural weight matrix.

Within the NL, three parallel scans and spike generation implement adaptive leaky integrate-and-fire behavior:
$$
\alpha_{exc,i} = \sigma(\tau_{exc,i}) \cdot 0.99,\qquad
V_{f,t,i} = \alpha_{exc,i} V_{f,t-1,i} + f(I_{t,i}),
$$
$$
\alpha_{adapt} = \exp(-1/\tau_{adapt}),\qquad
\theta_{adapt,t,i} = \alpha_{adapt}\theta_{adapt,t-1,i} + f(\sigma(V_{f,t,i} - v_{th})),
$$
$$
\alpha_{inh,i} = \sigma(\tau_{inh,i}) \cdot 0.99,\qquad
s_{pre,t,i} = \Theta(V_{f,t,i} - v_{th} - \theta_{adapt,t,i}\beta_{adapt}),
$$
$$
V_{res,t,i} = \alpha_{inh,i} V_{res,t-1,i} + f(s_{pre,t,i} w_{reset,i}),
$$
$$
V_{final,t,i} = V_{f,t,i} - V_{res,t,i} - \theta_{adapt,t,i}\beta_{adapt},\qquad
s_{out,t,i} = \Theta(V_{final,t,i} - v_{th}).
$$
Here $\Theta$ is the Heaviside step with surrogate gradient, $f(\cdot)$ is a softplus for strict positivity, and all three recurrences use the logcumsumexp primitive.

Within the SL, the pre-synaptic stage implements Tsodyks–Markram short-term plasticity:
$$
u_{t+1,i} = u_{t,i} e^{-\Delta t/\tau_f} + U (1-u_{t,i}) S_{t+1,i},
$$
$$
R_{t+1,i} = R_{t,i} e^{-\Delta t/\tau_d} + (1-e^{-\Delta t/\tau_d}) - u_{t+1,i} R_{t,i} S_{t+1,i},
$$
with $S_{t,i} = s_{out,t-delay,i}$, and these recurrences are also cast as affine scans solved in $O(\log T)$.

The post-synaptic stage enforces zero diagonal, imposes Dale’s Law at the level of each presynaptic column, applies the topology mask, and computes structured transmission:
$$
I_{syn,t,i} = \sum_{j=1}^D W_{struct,ij}\,[s_{out,t-d,j}\,u_{t,j}\,R_{t,j}].
$$
The decomposition is significant because it isolates diagonal, scan-friendly intrinsic dynamics from dense but shared spatial connectivity. This suggests that PHC is not a rejection of diagonal SSM design, but a structural generalization of it.

## 3. Hierarchical region partitioning and connectome topology

A defining feature of PHC is the partition of the $D$ neurons into $R$ non-overlapping macro-regions of sizes $D_r$. The connectome is encoded by a block-structured binary mask
$$
M_{topo} = \operatorname{block\_diag}(M_{rr'}),
$$
with blocks $M_{rr'} \in \{0,1\}^{D_r \times D_{r'}}$ indicating whether neurons in region $r'$ may project to neurons in region $r$ [2604.01295].

Two canonical masks are specified. The feedforward-only mask $M_{topo}^0$ permits $E \rightarrow E$ and $E \rightarrow I$ from $R_0 \rightarrow R_1$, but no feedback $R_1 \rightarrow R_0$. The bidirectional mask $M_{topo}^1$ permits both $R_0 \rightarrow R_1$ and $R_1 \rightarrow R_0$. In graph-theoretic terms, the system is a graph $G=(V,E)$ with
$$
V=\{1,\ldots,D\},\qquad
E=\{(j \rightarrow i)\mid M_{topo}[i,j]=1\},
$$
and weighted adjacency $W_{struct}$.

The framework’s hierarchical language refers to this regional partition and block-level admissibility structure. It does not merely impose sparsity; it specifies which classes of inter-regional projections are allowed. A plausible implication is that the region mask functions as an architectural prior analogous to a connectome-inspired constraint, making the connectivity pattern itself part of the model class rather than a post hoc regularizer.

Because PHC places the connectome topology in the shared SL rather than across a stack of independent layers, the hierarchy is expressed spatially within each timestep. This differs from a conventional layered interpretation in which depth is accumulated only through serial composition over layers.

## 4. Multi-Transmission Loop and intra-slice spatial recurrence

PHC recovers within-timestep spatial depth through a Multi-Transmission Loop. Let $x_{sensory}$ denote the gated encoder input and initialize $x^{(0)} = x_{sensory}$. Then, for $k=1,\ldots,M$,
$$
(s^{(k)}, state_{NL}^{(k)}) = NL(x^{(k-1)}, state_{NL}^{(k-1)}),
$$
$$
(I_{syn}^{(k)}, state_{SL}^{(k)}) = SL(s^{(k)}, state_{SL}^{(k-1)}),
$$
$$
x^{(k)} = I_{syn}^{(k)} + \alpha_{drive} x_{sensory}.
$$
The loop terminates early when the Cauchy criterion holds,
$$
\frac{\|I_{syn}^{(k)} - I_{syn}^{(k-1)}\|}{\|I_{syn}^{(k)}\|+\epsilon} < \theta_{conv},
$$
or when $k = N_{max}$.

Algorithm 1 in the exposition runs this loop in parallel across $t$ by using the log-scan in the NL and STP recurrences, together with a single shared matrix multiply $W_{struct}$ in the SL. The resulting temporal depth is $O(\log T)$ per scan, the spatial depth is $M$ iterations, and the total depth is $O(M \cdot \log T)$ [2604.01295].

This loop is the mechanism by which PHC introduces intra-slice spatial recurrence without abandoning the parallel-scan structure inherited from diagonal SSMs. A common misunderstanding is to equate recurrence with obligatory backpropagation through time over a serial chain of timestep updates. In PHC, the recurrence is split: temporal recurrence remains diagonal and scan-solvable, while spatial recurrence is handled by repeated circulation over a fixed connectome within the same temporal window.

The framework therefore relocates a substantial portion of expressive depth from stacked temporal layers to repeated spatial transmissions over shared structure. The exposition states this directly in complexity terms: PHC collapses the vertical depth $L$ of independent SSM layers into spatial circulations $M$ over a fixed connectome.

## 5. Biological priors and plasticity mechanisms

PHC is designed to integrate neuro-physical priors that are described as typically intractable for standard SSMs. The exposition explicitly details adaptive leaky integrate-and-fire dynamics, Dale’s Law, short-term plasticity, and reward-modulated spike-timing-dependent plasticity, all within the same parallelizable framework.

Adaptive leaky integrate-and-fire behavior is implemented in the NL through the recurrences for membrane potential, adaptive threshold, refractory suppression, and spike generation. The model uses the Heaviside step with surrogate gradient, softplus-enforced positivity, and learnable parameters $\tau_{exc}$, $\tau_{adapt}$, $\tau_{inh}$, $\beta_{adapt}$, and $w_{reset}$.

Dale’s Law is enforced by clamping each presynaptic column of $W_{syn}$ to be either all nonnegative for excitatory neurons or all nonpositive for inhibitory neurons:
$$
W_{:,j}^{(E)} \leftarrow \max(W_{:,j}^{(E)},0),\qquad
W_{:,j}^{(I)} \leftarrow \min(W_{:,j}^{(I)},0).
$$
The specified excitatory/inhibitory ratio is approximately $0.8/0.2$.

Short-term plasticity uses the Tsodyks–Markram formulation with state variables $u_t \in [0,1]$ and $R_t \in [0,1]$. Because these updates are recast as affine scans, the framework maintains the same scan-based temporal parallelism that motivates diagonal SSMs in the first place.

Reward-modulated spike-timing-dependent plasticity introduces eligibility traces
$$
x_{pre,t} = \alpha_+ x_{pre,t-1} + s_t,\qquad
x_{post,t} = \alpha_- x_{post,t-1} + s_t,
$$
with $\alpha_\pm = \exp(-1/\tau_\pm)$, followed by accumulated weight change
$$
\Delta W =
\frac{A_+ \sum_t s_t x_{pre,t-1}^{\top} - A_- \sum_t x_{post,t} s_t^{\top}}{B \cdot T}.
$$
Reward gating and synapse-type masking are then applied through
$$
W_{syn} \leftarrow W_{syn} + \eta_{hebb}\, r\, \Delta W \odot M_{topo} \odot M_{type},
$$
followed by reapplication of Dale’s Law, with
$$
r = 2 \cdot (\text{batch accuracy}) - 1 \in [-1,+1].
$$

The significance of these mechanisms is not only biological plausibility. The paper’s framing suggests that biologically grounded inductive biases may serve as a principled route to parameter-efficient sequence modeling. That claim is presented not as a biological argument alone, but as an architectural one: the priors are embedded in a model class that remains fully parallelizable during training.

## 6. Complexity, training pipeline, and empirical positioning

The complexity comparison given in the exposition contrasts stacked diagonal SSMs and PHC directly. A stacked diagonal-SSM with $L$ layers has a diagonal core of $D$ time-constants per layer and a dense inter-layer MLP of size $D^2$, for a total parameter count of $\Theta(D^2 L)$ and compute per sequence of $O(T D^2 L)$. PHC, by contrast, uses one shared NL with $O(D)$ time-constants and one shared SL matrix $W_{syn}$ of size $\Theta(D^2)$, reused across $M$ transmission steps. Its total parameter count is $\Theta(D^2)$, its overall work per sequence is $O(M \cdot T \cdot D^2)$, and its depth is $O(M \cdot \log T)$ [2604.01295].

The reported implementation notes specify $R=2$ regions, equal partition $D_0=D_1=D/2$, and an excitatory/inhibitory ratio of $0.8/0.2$ in each region. The topology mask choice, either $M_{topo}^0$ or $M_{topo}^1$, is selected per task by validation. Additional hyperparameters include synaptic delay $d$, STP parameters $U$, $\tau_f$, and $\tau_d$, loop parameters $M_{max}$, $\alpha_{drive}$, and $\theta_{conv}$, an encoder consisting of linear $W_{enc,D \rightarrow D}$ plus LayerNorm with an input mask $m_{in}$, and a readout defined as mean-over-time of $V_{final}$ on designated readout neurons followed by RMSNorm and linear decoder $W_{dec}$.

Training uses cross-entropy together with $\lambda_{rate}$ for a mean firing-rate penalty and $\lambda_{volt}$ for a voltage penalty. All recurrences in the NL and STP components are solved via log-domain prefix-sums in $O(\log T)$, eliminating BPTT over time. R-STDP updates are applied after each batch outside the autograd graph, and the framework is described as implementable in JAX/Flax or PyTorch with custom prefix-sum kernels. For inference, the log-scan can be replaced with sequential step-by-step updates for energy-efficient spiking-platform execution.

The empirical results reported for PHCSSM are on physiological benchmarks from the UEA multivariate time-series archive. The model is stated to achieve performance competitive with state-of-the-art SSMs while reducing parameter complexity from $\Theta(D^2 L)$ for $L$-layer stacked architectures to $\Theta(D^2)$. Within the paper’s interpretation, these findings suggest that biologically grounded inductive biases offer a principled route to parameter-efficient sequence modeling and open diagonal SSMs to spatiotemporal recurrence.

Source: https://www.emergentmind.com/topics/parallelized-hierarchical-connectome-phc