G2GNet: Modular Sparse Neural Architecture

• G2GNet is a neuro-inspired ANN architecture featuring modular ensemble-to-ensemble connectivity with block-diagonal weight masks that mimic biological cortical circuits.
• It employs dynamic sparse training with Hebbian-inspired rewiring to maintain high sparsity while reducing parameter counts drastically.
• Empirical evaluations show G2GNet matching or exceeding dense model accuracies on benchmarks like Fashion-MNIST, CIFAR-10, and CIFAR-100 with improved computational efficiency.

G2GNet is a neuro-inspired artificial neural network (ANN) architecture designed to enforce sparse, modular, ensemble-to-ensemble connectivity across feedforward layers. Drawing direct motivation from electrophysiological studies of the mouse visual cortex, particularly the communication pattern between Layer 4 and Layer 2/3 in V1, G2GNet imposes a biologically grounded structural prior on artificial networks. The architecture features a group-based block-diagonal adjacency structure, dynamic sparse training with Hebbian-inspired rewiring, and achieves high sparsity ratios while maintaining or exceeding the predictive performance of dense models on standard vision benchmarks (Konstantaropoulos et al., 19 Aug 2025).

1. Modular Ensemble-to-Ensemble Connectivity

G2GNet replaces the standard dense layer-to-layer connections in multi-layer perceptrons with a pattern based on neuron ensembles, or "groups." Formally, for each hidden layer $i$ containing $N_i$ neurons, the layer is divided into $G_i$ non-overlapping ensembles, each of size $N_i/G_i$. The successor layer $i+1$ has $G_{i+1}$ groups. Connectivity is structured around "pathways": group $k$ in layer $i$ connects densely (probability $p$) to group $k$ in layer $N_i$0, while connections between non-matched groups occur at much lower probability $N_i$1. This yields a block-diagonal binary adjacency mask per layer pair, with most signal flow constrained to a small number of high-reliability channels between functionally aligned ensembles.

This structural bias reflects biological evidence: mouse V1 feedforward transmission is mediated by co-active, small ensembles rather than single-unit connections. Postsynaptic firing probability exhibits a nonlinear, ReLU-like threshold at a critical fraction ($N_i$2) of an ensemble co-firing, indicating strong intra-ensemble coupling and sparse inter-ensemble links (Konstantaropoulos et al., 19 Aug 2025).

2. Sparse Connectivity and Mathematical Formulation

G2GNet’s connectivity is initialized as follows. For each feedforward layer transition, let $N_i$3 be the full-precision weight matrix and $N_i$4 the binary mask. The mask is defined groupwise such that

$N_i$5

with $N_i$6 and $N_i$7 empirically yielding 75–85% sparsity. The effective weight matrix is $N_i$8, i.e., forward and backward passes use the masked weights. This ensemble-based, block-sparse initialization imposes modular structure and reduces parameter counts by an order of magnitude compared to full dense equivalents. In one configuration with hidden MLP layers of width 1024 and $N_i$9, overall parameter count drops from $G_i$03.15M (dense) to $G_i$10.81M (G2GNet), while structured sparsity enhances hardware acceleration via block-sparse operations.

3. Dynamic Sparse Training and Hebbian-Inspired Rewiring

To complement static modularity, G2GNet incorporates dynamic sparse training (DST) to promote plasticity and robustness. Every $G_i$2 iterations (typically 1000), a fraction $G_i$3 (e.g., 2.5%) of active edges are pruned, and an equal number of edges regrown, maintaining fixed sparsity throughout training. Edge selection for pruning/growth is governed by Hebbian-inspired criteria: activations of neuron pairs $G_i$4 are correlated over a batch to compute

$G_i$5

Edges with the lowest $G_i$6 are pruned, while regrowth favors previously inactive pairs with the highest $G_i$7. This mimics biological synaptic plasticity, preferentially reinforcing strongly co-active neuron groups. Optionally, incremental updates can take the form $G_i$8.

The combination of static block-diagonal structure and DST enables the network to continually explore and exploit new sparse subgraphs while preserving global modular organization, shown to support generalization and stability.

4. Performance, Accuracy–Sparsity Trade-offs, and Computational Efficiency

Empirical evaluation demonstrates that G2GNet matches or outperforms dense fully-connected models on image classification tasks while utilizing only a fraction of the parameters and FLOPs. Key results include (Konstantaropoulos et al., 19 Aug 2025):

Dataset	Dense FC Param-Match	Dense FC Width-Match	G2GNet (Static+DST)
Fashion-MNIST	90.2%	—	90.4–91.1%
CIFAR-10	68.4%	71.5%	71.5–72.7%
CIFAR-100	38.7%	40.8%	42.2–42.7%

Operation at $G_i$9–$N_i/G_i$0 (75–85% overall sparsity) yields the highest test accuracy. Increasing $N_i/G_i$1 beyond this saturates or degrades performance, likely due to loss of modular specialization. Performance gains are robust to the choice of ensemble grouping (index or mixer). Computational cost is reduced proportionally with the number of nonzero weights; structured block sparsity further facilitates hardware optimization.

5. Biological Principles and Architectural Rationale

G2GNet closely parallels architectural motifs observed in cortical circuits. It embodies modularity via neuron ensembles, hierarchy through layerwise group-to-group pathways, and efficiency from sparse interconnectivity. The intra-ensemble reliability is enabled by high within-pathway connectivity ($N_i/G_i$2), while architectural robustness is supported by very sparse cross-group shortcuts ($N_i/G_i$3 small). This supports both functional specialization and metabolic efficiency, analogous to observed biological trade-offs.

The DST mechanism, and Hebbian-inspired rewiring in particular, are motivated by synaptic plasticity and dynamic functional remapping seen in vivo. The adoption of activation correlation as the growth/pruning criterion is a direct reflection of “fire together, wire together” dynamics of biological networks.

6. Future Directions and Potential Extensions

Proposed extensions of G2GNet include dynamic or learning-based assignment of neurons to ensembles, emulating experience-driven ensemble formation mechanisms in cortex. Application of the G2G communication pattern to other architectural primitives—such as convolutional layers (using channels or receptive fields as ensembles) and attention mechanisms—represents a logical direction. Beyond this, integrating local (biologically plausible) learning rules remains an open research trajectory for further aligning ANN efficiency with neural circuit principles (Konstantaropoulos et al., 19 Aug 2025).

A plausible implication is that G2GNet’s blend of static modular bias and dynamic synaptic plasticity may generalize to domains requiring continual adaptation, robustness to pruning, or highly efficient deployment on specialized hardware.