---
title: Modular Neural Computer Architecture
url: https://www.emergentmind.com/topics/modular-neural-computer
type: topic
---

# Modular Neural Computer Architecture

to=arxiv_search.query 派奖中  天天中彩票出票json_string={"query":"id:2603.13323 OR ti:\"Modular Neural Computer\"","max_results":5,"sort_by":"relevance"}  สำนักเลขานุการองค์กร
to=arxiv_search.query  สล็อตโനjson_string={"query":"id:2604.06425 OR id:2009.11443 OR id:2210.08031 OR id:1811.05249 OR id:2409.05780","max_results":10,"sort_by":"relevance"} 
Modular Neural Computer (MNC) denotes a memory-augmented neural architecture for exact algorithmic computation on variable-length inputs. The model combines an external associative memory of scalar cells, explicit read and write heads, a controller multi-layer perceptron (MLP), and a homogeneous set of functional MLP modules. Rather than learning an algorithm end to end from data, it realizes a given algorithm through analytically specified neural components with fixed interfaces and exact behavior. Control flow is represented inside the neural computation through one-hot module gates, where inactive modules are inhibited, and computation unfolds as a sequence of memory transformations generated by a fixed graph [2603.13323].

## 1. Conceptual position within neural computation

The MNC occupies a specific position within the broader landscape of neural computation. It is modular, memory-augmented, and neural, but its central purpose is exact, deterministic algorithmic execution rather than statistical approximation. In this respect it differs from the broader “Neural Computers” program, in which Neural Computers internalize the core roles of computation, memory, and I/O within a single neural runtime state, and aim to make the model itself the running computer [2604.06425]. The MNC instead uses explicit external memory, explicit heads, and analytically specified modules so that a given algorithm is realized step-for-step.

A common misconception is that modular neural computation is necessarily synonymous with end-to-end learned routing or emergent specialization. The MNC is not trained to discover an algorithm from traces. There is no end-to-end training of the system for the task; all network modules, read/write logic, and control are fully analytically specified for a given algorithm. This makes the architecture closer to a neuro-symbolic compilation scheme than to a purely subsymbolic learner [2603.13323].

This distinction places the MNC at one pole of a spectrum. At another pole are learned runtimes and learned modular systems; between them are memory-augmented and controller-based architectures that learn how to select, compose, or retrieve modules. A plausible implication is that “modular neural computer” names not only a single architecture, but also a design problem: how to combine modular computation, persistent state, and explicit control without collapsing either into a conventional program interpreter or into a monolithic end-to-end network.

## 2. Architectural organization

The MNC is built around an external associative memory consisting of a vector of scalar cells \( \{M[a] \mid a \in A\} \), where \( A = \{0,1,\ldots,S-1\} \). Addressing is performed associatively: addresses are presented as one-hot vectors, and read and write operations use a key-based softmax lookup. With \( K = I_S \) and a large temperature \( \tau \), integer queries perform hard lookups, while fractional queries interpolate. Writes update memory according to
$$
M^{t+1}(a) = \alpha(a)\cdot v + (1-\alpha(a))\cdot M^t(a).
$$
All mutable state resides in memory; internal activations are not relied upon for persistence [2603.13323].

At every computation step, the controller MLP receives a small set of control scalars \( c = (c_1,\ldots,c_{n_c}) \), frequently including the current index, total input length, constants, or running flags. It emits three outputs: a one-hot gate vector \( \mathbf{g}_t = (g_1,\ldots,g_K) \), a fixed number of read addresses \( \mathbf{r}_t \), and a fixed number of write addresses \( \mathbf{w}_t \). The functional modules are homogeneous MLPs with a fixed interface,
$$
F_k:(g_k,x_1,\ldots,x_{n_r}) \mapsto (u_1,\ldots,u_{n_w}),
$$
so every module consumes the same number of inputs and produces the same number of outputs. Only the active module, for which \( g_k = 1 \), produces a nonzero output; all other modules return zero. The overall module output is therefore merged by addition,
$$
y_i = \sum_{k=1}^{K} u_{k,i}.
$$

This organization yields a fixed computation graph reused across timesteps. Only memory content, read and write addresses, and one-hot gates vary during execution. The architecture is modular because the functional units are separated into architectural peers with a common signature, memory-centric because all persistent variables are explicit in memory, and deterministic because the entire execution trace is specified by the constructed controller, the fixed modules, and the current memory state [2603.13323].

## 3. Execution semantics and algorithmic case studies

Each MNC step consists of four phases. First, the controller reads memory and emits the active module gate together with read and write addresses. Second, the specified memory locations are read. Third, the active module applies its analytically specified transformation. Fourth, the resulting values are written back to the controller-specified memory locations. Computation is therefore a sequence of memory transformations, not a hidden-state recurrence in the usual sense [2603.13323].

The architecture is illustrated through three case studies. In minimum-value computation, memory stores the array, the current index, the length, the current minimum, a zero constant, and a flag. The system uses three modules: Init, Update, and Stop. The Update module compares the current minimum with the next array element using
$$
\min(x_1,x_2)=\frac{x_1+x_2-|x_1-x_2|}{2}.
$$
Execution is a fully deterministic iteration over the array.

In in-place sorting, the MNC performs adjacent-exchange sorting. Memory stores the array, the current pair index \( i \), the pass limit \( p \), a zero constant, and a running flag. Three modules are used again: Pair-Process, Pass-Transition, and Stop. Pair-Process sorts \((a_i,a_{i+1})\) in place, increments the index, and writes the updated values and control state back into memory. Pass-Transition decreases the pass limit and resets the index. Stop writes the halting condition.

In A* search on a fixed problem instance, memory is partitioned into a fixed graph description, control state, and search-node records. Six modules correspond to the search phases: InitRoot, StartOpenScan, ScanOpenNode, FinishOpenScan, GoalTest, and ExpandAction. Open and closed flags, parent pointers, costs, and scan state remain explicit in memory throughout execution. These case studies show that algorithmic procedures can be compiled into modular neural components with external memory while preserving deterministic behavior and explicit intermediate state [2603.13323].

## 4. Relation to learned modular architectures

The MNC contrasts sharply with learned modular systems in which both the functions of modules and their usage are optimized from data. In “Modular Networks: Learning to Decompose Neural Computation,” a controller selects \( K \) modules from a pool of \( M \) available modules for each input, both the decomposition and modules are learned end-to-end, and training uses a stochastic generalized Expectation-Maximization procedure, specifically a Viterbi EM variant, that does not require extra regularization to prevent module collapse [1811.05249]. There, modularity serves conditional computation and specialization; in the MNC, modularity serves exact execution of an already specified algorithm.

“Neural Attentive Circuits” extends the learned-modularity perspective by jointly learning both module parameterization and a sparse connectivity of neural modules without using domain knowledge. NACs support a large number of modules, use a circuit generator and a circuit executor trained end-to-end, and improve upon a strong non-modular baseline in low-shot adaptation on CIFAR and CUBs by about 10%, OOD robustness on Tiny ImageNet-R by about 2.5%, and can achieve an 8x speedup at inference time while losing less than 3% performance [2210.08031]. This is a different notion of modular neural computation: the graph and module behavior are learned, sparsity is induced through attention and graph priors, and the objective is robustness, efficiency, and general-purpose behavior across modalities.

The comparison clarifies an important taxonomy. In the MNC, modules are architectural peers with exact behavior and fixed interfaces. In learned modular networks, modules are latent computational resources whose specialization and routing emerge during optimization. This suggests a useful distinction between analytic modular neural computers and learned modular neural architectures.

## 5. External memory, program storage, and algorithm induction

A second line of related work concerns how neural systems store and retrieve programs. Neurocoder introduces the stored-program principle into neural networks by using external memory to create, store, and retrieve modular programs. It stores modular neural program components through a low-rank decomposition based on Singular Value Decomposition, uses a Program Controller with recurrent, multi-head differentiable attention to compose a Working Program for each input, and treats a Neural Program as a datum in memory, paving the ways for modular, recursive and procedural neural programming [2009.11443]. Relative to Neurocoder, the MNC uses external memory not to retrieve compressed learned programs, but to hold the entire mutable execution state of an analytically compiled procedure.

A related but distinct architecture is Main, a modular neural network that learns algorithms from input-output examples. Main consists of a neural controller that interacts with a variable-length input tape and learns to compose modules together with their corresponding argument choices. It uses a general domain-agnostic mechanism for selection of modules and their arguments, a general input tape layout together with a parallel history tape, and a memoryless controller with a length-invariant self-attention based input tape encoding to allow for random access to tape locations. Trained end-to-end using reinforcement learning, Main can learn policies that generalize perfectly to inputs of much longer lengths than the ones used for training [2003.04227].

These architectures frame the MNC’s contribution more precisely. The MNC is memory-augmented like Neurocoder and tape-like architectures, but it does not rely on learning to recover an execution procedure. Instead, the controller, module library, and memory transformations are synthesized so that execution is exact, inspectable, and deterministic [2603.13323].

## 6. Interpretability, modularity, and unresolved questions

One of the central properties of the MNC is inspectability. All intermediate states and progress are externally visible in memory; nothing is hidden in activations. Each module corresponds directly to a recognizable phase of a standard algorithm, and the computation trace is reproducible for a given input and initial state [2603.13323]. In this respect, the architecture avoids a common ambiguity in modular neural systems: whether topological modularity corresponds to functional modularity.

That ambiguity is explicit in the broader modularity literature. A review of modularization techniques states that topological modularity is a necessary, but not sufficient, condition for functional modularity [1904.12770]. Empirical analyses of trained multilayer perceptrons similarly show that training and weight pruning produce networks that are more modular than randomly initialized ones, and that they are much more modular when trained with dropout [2003.04881]. Those findings establish that modular structure can emerge from learning, but they do not by themselves guarantee that modules map cleanly onto algorithmic phases or interpretable subroutines. The MNC sidesteps that issue by construction.

The main limitations appear at the boundary between exact compiled execution and general-purpose neural computation. Neural Computers, in the broader sense, are still confronted by routine reuse, controlled updates, and symbolic stability as open problems [2604.06425]. The MNC resolves symbolic stability for the algorithms it realizes by analytically specified components with fixed interfaces and exact behavior, but it does not address how such modules should be learned, generalized, or reprogrammed across tasks. This suggests a persistent research tension: explicit modular neural computers provide deterministic behavior and explicit intermediate state, whereas learned neural computers aim for adaptable runtimes and durable capability reuse. The long-term synthesis of these two traditions remains open.

Source: https://www.emergentmind.com/topics/modular-neural-computer