Randomly Wired Neural Networks

• Randomly Wired Neural Networks are defined by stochastic or heuristic wiring patterns that replace traditional hand-designed connectivity.
• They encompass various models such as graph-wired CNNs, fixed random-weight systems, and readout-trained networks to balance efficiency and performance.
• RWNNs demonstrate robustness across domains like image recognition, climate emulation, and reinforcement learning, with pruning and rewiring enhancing computational efficiency.

Searching arXiv for recent and foundational papers on Randomly Wired Neural Networks. Searching arXiv for "Randomly Wired Neural Networks RandWire RicciNets climate emulation RAN-GNNs". Randomly Wired Neural Networks (RWNNs) are neural architectures in which the wiring pattern, and in some strands of the literature also the weights or neuron-local dynamics, are generated by stochastic or heuristic procedures rather than being entirely hand-designed or fully learned end to end. In the image-recognition line inaugurated by random graph generators, an RWNN is typically a feedforward convolutional network whose hidden connectivity is obtained by sampling a graph and mapping it to a directed acyclic computational graph; in the broader randomized-network literature, RWNNs also include models with fixed random weights, readout-only training, reservoir-style recurrence, or neuron-centric parameterization (Xie et al., 2019, Gallicchio et al., 2020). The field therefore spans a unified question—what aspects of neural computation can be delegated to random or weakly constrained structure—while differing substantially in what is randomized and what remains trainable.

1. Definitions, scope, and competing usages

A precise starting point is the stochastic-network-generator view. In the image-recognition formulation, a network generator is a mapping $g:\Theta \to N$ from generator parameters to architectures, and a stochastic generator augments this with a seed $s$, inducing a distribution over architectures through $n=g(\theta,s)$ or, more explicitly, a latent graph $G=(V,E)$ sampled from $p(G \mid \theta)$ (Xie et al., 2019). This framing places RWNNs and neural architecture search within the same formal space, while distinguishing RWNNs by their reliance on classical random graph families rather than tightly constrained searched motifs.

The broader survey literature adopts a wider definition. “Randomized Neural Networks” are architectures in which “the majority of connections are fixed, either in a stochastic or a deterministic fashion,” with training restricted to a reduced set of weights, often the final readout (Gallicchio et al., 2020). Within that scope, RWNNs can combine random topology and random weights, or use deterministic sparse wiring together with fixed hidden transformations. This broader usage explains why the term encompasses both graph-wired convolutional networks and dense feedforward networks whose weight matrices are sampled once and frozen.

Family	What is randomized	Representative paper
Graph-wired CNNs	Hidden-stage connectivity graph	(Xie et al., 2019)
Frozen-weight feedforward or spiking models	Hidden weights, often with fixed dense adjacency	(Pedersen et al., 2023, Dai et al., 19 May 2025)
Readout-trained randomized systems	Hidden weights and sometimes wiring, readout trained	(Gallicchio et al., 2020)

This diversity is not merely terminological. It marks distinct research programs: architectural randomization for representation learning, random-feature and reservoir-style efficiency, and graph-theoretic analysis of topology before or during training. A recurring misconception is to treat all RWNNs as random-topology CNNs; the literature surveyed here shows that the term is used for random graph wiring, fixed random weights, neuron-level heterogeneity, and randomly connected recurrent networks (Zou et al., 2023).

2. Random graph generators and graph-to-network mappings

The most influential graph-wired formulation samples hidden-stage graphs from classical random graph models and then maps them to neural computation. Three generators recur: Erdős–Rényi $G(n,p)$, Barabási–Albert preferential attachment, and Watts–Strogatz small-world graphs (Xie et al., 2019). In the original image-recognition study, undirected sampled graphs are converted to DAGs by indexing nodes and orienting every edge from the lower to the higher index, after which unique input and output nodes are added so that the graph supports a well-defined feedforward computation (Xie et al., 2019).

Node semantics are standardized rather than searched. Each node aggregates predecessor tensors through a weighted sum with positive learnable weights, applies a uniform transformation, and distributes copies to all outgoing edges. In the original formulation,

$$x_i = f_i\Big( \sum_{j \in \mathrm{pred}(i)} w_{ij}\, x_j \Big),\quad w_{ij}=\sigma(\alpha_{ij}),$$

with $f_i$ instantiated as a ReLU–Conv–BN triplet and edges themselves remaining parameter-free (Xie et al., 2019). Additive aggregation keeps channel count invariant with respect to in-degree, so comparisons across wirings are not confounded by inflated parameter count or FLOPs.

Among the generator families, Watts–Strogatz proved especially effective. The study of image recognition on ImageNet reported that several ER, BA, and WS variants were competitive, but WS with moderate rewiring was particularly strong; in the small-compute regime, WS$(4,0.75)$ achieved $74.7 \pm 0.25$ top-1 and $s$0 top-5 at about $s$1M FLOPs and about $s$2M parameters, while regular-compute WS models reached $s$3 top-1 at about $s$4B FLOPs and $s$5 at about $s$6B FLOPs, outperforming ResNet and ResNeXt baselines at similar compute (Xie et al., 2019). The same backbones transferred effectively to COCO detection, again exceeding ResNet and ResNeXt at comparable compute (Xie et al., 2019).

Later work preserved this mapping but used it as a substrate for compression and task transfer. RicciNets adopts the RandWire setup and uses WS$s$7 as its baseline computational graph, again orienting edges by increasing node label to obtain a DAG (Glass et al., 2020). A 2025 chest x-ray study used ER, WS, and BA hidden-stage blocks with $s$8 nodes and $s$9 channels between standard stem and classifier components, preserving the same basic aggregation–transformation–distribution semantics (Elumalai et al., 30 Aug 2025). In both cases, the graph is treated as an architectural object that can be pruned or analyzed before or after training.

3. Randomization beyond graph topology

A second major branch uses “randomly wired” to mean that hidden connectivity is conventional but hidden weights or neuron-level transformations are fixed or only lightly trained. In “Learning to Act through Evolution of Neural Diversity in Random Neural Networks,” the networks are standard layered feedforward architectures in which all inter-layer connections are dense and directed from layer $n=g(\theta,s)$0 to $n=g(\theta,s)$1, but the synaptic weights $n=g(\theta,s)$2 are sampled i.i.d. from $n=g(\theta,s)$3 and then frozen (Pedersen et al., 2023). Neural computation is shifted to a neuron-specific $n=g(\theta,s)$4 parameter matrix $n=g(\theta,s)$5 that transforms a local vector $n=g(\theta,s)$6 into a propagated activation and a recurrent state,

$n=g(\theta,s)$7

while the random synaptic weights remain untrained (Pedersen et al., 2023). Evolution over these neuron-centric parameters alone produced competitive scores on CartPoleSwingUp, BipedalWalker-v3, and CarRacing-v0, with recurrent neurons reaching $n=g(\theta,s)$8 on CarRacing-v0, the best among the studied configurations (Pedersen et al., 2023).

RanSNN translates the same logic into spiking computation. Its hidden spiking layers are densely connected between consecutive layers, with randomness again confined to the inter-layer weights rather than to the presence or absence of edges; effectively, the adjacency matrices are all ones and the sampled weights are fixed (Dai et al., 19 May 2025). For MNIST-style experiments, the input-to-hidden weights are sampled from $n=g(\theta,s)$9 with $G=(V,E)$0, the hidden layer contains $G=(V,E)$1 LIF neurons, and only the final readout is trained with Adam on cross-entropy (Dai et al., 19 May 2025). On four benchmark datasets, training times were reduced by roughly two orders of magnitude relative to surrogate-gradient baselines; for example, on MNIST the reported times were $G=(V,E)$2s for surrogate gradients versus $G=(V,E)$3s for RanSNN, while accuracy remained competitive at $G=(V,E)$4 (Dai et al., 19 May 2025).

A third variant randomizes inter-layer dense connectivity rather than within-layer weights. In climate model emulation, “RandDense” replaces serial dense blocks with a random DAG over dense layers. The adjacency is sampled uniformly from a lower-triangular binary matrix and then repaired so that every layer has at least one inbound and one outbound edge; incoming tensors are combined by a weighted sum with learnable positive scalars, followed by ReLU and a dense transform (Yik et al., 2022). Across $G=(V,E)$5 architecture–budget–task combinations on ClimateBench, only one case showed a statistically significant deficit relative to serial baselines, whereas $G=(V,E)$6 showed statistically significant improvement; gains were largest for simpler architectures, reaching up to $G=(V,E)$7 for multilayer perceptrons (Yik et al., 2022).

Random wiring has also been adapted to GNNs. RAN-GNNs sample an ER upper-triangular architecture DAG over GNN modules, interpret every path as a different effective depth, and learn sigmoid-gated edge weights $G=(V,E)$8 for the aggregation

$G=(V,E)$9

Theoretical analysis shows that the architecture behaves as an ensemble of paths with varied receptive-field radii, and empirical results on ZINC, CIFAR10 superpixels, and OGBG-MOLPCBA show consistent gains over matched-depth baselines (Valsesia et al., 2021).

4. Topology-aware pruning and rewiring

Because graph-wired RWNNs expose architecture as an explicit graph before training, they invite topological pruning methods that operate without reference to learned weight magnitudes. RicciNets is the first paper in the supplied corpus to prune RWNNs directly. It defines a node-mass distribution over a node $p(G \mid \theta)$0 and its neighborhood $p(G \mid \theta)$1,

$p(G \mid \theta)$2

uses this to compute discrete Ollivier–Ricci curvature

$p(G \mid \theta)$3

and evolves edge weights under Ricci flow before pruning edges whose converged weights fall below the mean (Glass et al., 2020). A lightweight REINFORCE controller outputs $p(G \mid \theta)$4, with $p(G \mid \theta)$5 determined by normalization, and optimizes an accuracy–FLOPs reward (Glass et al., 2020). On CIFAR-10, the method reported almost $p(G \mid \theta)$6 reduction in FLOPs per pass with no degradation in performance, average accuracy gains of about $p(G \mid \theta)$7 over baseline across $p(G \mid \theta)$8, and superior performance to lowest-magnitude pruning at comparable compression: $p(G \mid \theta)$9 top-1 with $G(n,p)$0 weights remaining, versus $G(n,p)$1 for lowest-magnitude pruning (Glass et al., 2020).

The later chest x-ray case study broadens curvature-guided pruning beyond ORC. It compares Forman–Ricci curvature, Ollivier–Ricci curvature, and edge betweenness centrality on ER, WS, and BA RWNNs, using binary search over pruning fraction while requiring accuracy, specificity, and sensitivity to remain at or above the baseline (Elumalai et al., 30 Aug 2025). ORC generally yields the largest compression and speedup in ER and BA networks, whereas FRC is computationally cheaper and competitive in WS graphs. The study also reports a structural trade-off: pruning increases modularity and decreases global efficiency, with ORC usually producing the strongest increase in modularity and the largest decrease in efficiency (Elumalai et al., 30 Aug 2025).

A distinct line studies rewiring rather than pruning. In directed random networks, adaptive rewiring based on advection and consensus kernels,

$G(n,p)$2

produces “convergent–divergent units” consisting of convergent hubs, densely interconnected intermediate processing nodes, and divergent hubs (Rentzeperis et al., 2021). Balanced mixing of the two criteria shortens directed path length and increases reachability. This suggests that, even before considering task loss, directed RWNN topology can be sculpted toward efficient communication motifs.

At a finer scale, preferential-attachment rewiring at initialization reorganizes dense layer weights while preserving their values. The procedure rewires complete bipartite layer graphs so as to reduce variance and tail-heaviness of neuronal strength distributions, without changing the global weight distribution (Scabini et al., 2022). Across LeNet, ResNet18, ResNet50, SL-ViT, and ConvMixer, this micro-level rewiring generally improved early training behavior and produced modest but consistent test-accuracy gains, indicating that random organization inside dense layers is itself an architectural prior (Scabini et al., 2022).

5. Theoretical frameworks and systems considerations

RWNN research is closely tied to older theories of randomized neural systems. The deep-randomized-networks survey places RVFL, ELM, randomized convolutional filters, ESN, LSM, and DeepESN under a common lens in which hidden mappings are fixed and only a reduced parameter set is trained (Gallicchio et al., 2020). In feedforward form, this leads to closed-form or convex readout training, such as ridge regression on a random feature map $G(n,p)$3, with

$G(n,p)$4

or its dual form (Gallicchio et al., 2020). In recurrent settings, the same chapter emphasizes stability via the echo state property, using conditions such as $G(n,p)$5 and, for strict contraction, $G(n,p)$6 (Gallicchio et al., 2020). This literature treats RWNNs as analyzable objects before training, not only as trainable architectures.

For fully connected recurrent random networks, dynamical mean-field theory provides an exact large-$G(n,p)$7 reduction. In the bidirectionally correlated-coupling model,

$G(n,p)$8

with $G(n,p)$9 and $$x_i = f_i\Big( \sum_{j \in \mathrm{pred}(i)} w_{ij}\, x_j \Big),\quad w_{ij}=\sigma(\alpha_{ij}),$$0, the large-$$x_i = f_i\Big( \sum_{j \in \mathrm{pred}(i)} w_{ij}\, x_j \Big),\quad w_{ij}=\sigma(\alpha_{ij}),$$1 dynamics reduce to an effective one-neuron stochastic process with a self-consistent memory term and Gaussian field (Zou et al., 2023). Linearization and the elliptic law imply a stability threshold $$x_i = f_i\Big( \sum_{j \in \mathrm{pred}(i)} w_{ij}\, x_j \Big),\quad w_{ij}=\sigma(\alpha_{ij}),$$2 for smooth activations such as $$x_i = f_i\Big( \sum_{j \in \mathrm{pred}(i)} w_{ij}\, x_j \Big),\quad w_{ij}=\sigma(\alpha_{ij}),$$3, while a static cavity analysis gives $$x_i = f_i\Big( \sum_{j \in \mathrm{pred}(i)} w_{ij}\, x_j \Big),\quad w_{ij}=\sigma(\alpha_{ij}),$$4 for ReLU (Zou et al., 2023). A complementary functional-field-theory tutorial derives the classic Sompolinsky–Crisanti–Sommers chaos transition for dense asymmetric random networks with $$x_i = f_i\Big( \sum_{j \in \mathrm{pred}(i)} w_{ij}\, x_j \Big),\quad w_{ij}=\sigma(\alpha_{ij}),$$5, again locating the $$x_i = f_i\Big( \sum_{j \in \mathrm{pred}(i)} w_{ij}\, x_j \Big),\quad w_{ij}=\sigma(\alpha_{ij}),$$6 threshold near $$x_i = f_i\Big( \sum_{j \in \mathrm{pred}(i)} w_{ij}\, x_j \Big),\quad w_{ij}=\sigma(\alpha_{ij}),$$7 and expressing the autocorrelation through a self-consistent potential formulation (Schuecker et al., 2016). These results do not describe graph-wired CNNs directly, but they provide the principal analytical vocabulary for random recurrent wiring: spectral radius, criticality, autocorrelation dynamics, and Lyapunov stability.

Irregular wiring also creates systems-level challenges. Because RWNNs and NAS graphs contain wide fan-in and fan-out motifs, the schedule of node execution can greatly alter peak activation memory. SERENITY formalizes the problem on a DAG $$x_i = f_i\Big( \sum_{j \in \mathrm{pred}(i)} w_{ij}\, x_j \Big),\quad w_{ij}=\sigma(\alpha_{ij}),$$8 with live-set memory

$$x_i = f_i\Big( \sum_{j \in \mathrm{pred}(i)} w_{ij}\, x_j \Big),\quad w_{ij}=\sigma(\alpha_{ij}),$$9

and uses dynamic programming over zero-indegree frontiers to find schedules that minimize $f_i$0 (Ahn et al., 2020). On irregularly wired networks including RandWire, the method achieved a $f_i$1 peak-memory improvement over TensorFlow Lite, rising to $f_i$2 with additional semantics-preserving graph rewriting, with less than one minute overhead on average (Ahn et al., 2020). This is a reminder that RWNNs are not only architectural objects but also compilation and deployment problems.

6. Empirical domains, persistent advantages, and open limitations

RWNNs have been evaluated across image recognition, object detection, climate emulation, reinforcement learning, medical imaging, graph learning, and spiking classification. In computer vision, random graph-wired CNNs on ImageNet and COCO established that sampled WS, ER, and BA generators can be competitive with or exceed manually designed backbones at similar FLOPs (Xie et al., 2019). In climate emulation, replacing standard dense heads with random DAGs improved performance in most conditions while leaving inference speed essentially unchanged (Yik et al., 2022). In graph learning, random architecture DAGs over GNN layers consistently improved capacity relative to deeper stacks, reaching or surpassing strong baselines on OGBG-MOLPCBA when combined with Virtual Node and FLAG (Valsesia et al., 2021). In spiking classification, fixed random hidden layers with trained readouts delivered competitive accuracy with training times reduced by two orders of magnitude (Dai et al., 19 May 2025). In RL control, evolved neuron diversity over random frozen weights yielded competitive episodic returns without synaptic optimization (Pedersen et al., 2023).

The strongest recurring empirical advantage is not a single accuracy number but an operating regime. RWNNs repeatedly appear where one wants architectural diversity, strong inductive bias from graph structure, pre-training compression, or restricted optimization budgets. Curvature-guided pruning is explicitly useful when compression should be decided before full training (Glass et al., 2020). Frozen-random-weight models are useful when end-to-end training is costly or unstable, as in SNNs (Dai et al., 19 May 2025). Random dense DAGs appear most beneficial when standard heads are under-expressive relative to the task, as in low-budget climate MLPs (Yik et al., 2022).

The limitations are equally consistent. Generator choice matters: WS tends to outperform ER and BA in image-recognition CNNs, and poor settings degrade performance (Xie et al., 2019). Compression is not unbounded: RicciNets reports a restricted compression range, with severe compression tending to produce chain-like topologies and sharp accuracy drops (Glass et al., 2020). Fixed random transformations can degrade signals and may need larger width, synaptic tuning, or adaptive plasticity on harder tasks (Pedersen et al., 2023, Dai et al., 19 May 2025). Random dense DAGs can underperform in some high-capacity settings, as seen in the $f_i$3M-parameter MLP DTR case in climate emulation (Yik et al., 2022). Theoretical guarantees also remain fragmented: the readout-trained and recurrent-randomized traditions have mature analyses, while the graph-wired CNN line still relies more heavily on empirical comparison and graph-theoretic intuition than on closed-form theory (Gallicchio et al., 2020, Xie et al., 2019).

Taken together, the literature supports a broad but precise encyclopedia definition. RWNNs are neural systems in which connectivity, weights, or local computational modules are generated by random or weakly constrained mechanisms and then exploited, analyzed, pruned, or lightly trained. In one lineage they are DAG-mapped random graph CNNs; in another they are frozen random-feature or reservoir-like systems; in newer variants they are neuron-diverse, spiking, graph-ensemble, or curvature-pruned models. The common premise is that useful computation can emerge from structured randomness, provided that the randomization axis, the mapping to operators, and the downstream optimization procedure are chosen to match the task and the resource regime (Gallicchio et al., 2020, Xie et al., 2019).