Neural Computers: Unified Computation & Memory

• Neural Computers are architectures that merge memory, computation, and I/O into a single neural runtime, eliminating traditional program counters and external memory structures.
• They employ techniques like differentiable memory, modular controllers, and end-to-end learning to perform tasks such as arithmetic, search, and algorithmic generalization.
• NCs are implemented in various substrates including neuromorphic hardware and bio-inspired circuits, promising energy efficiency, reprogrammability, and integrated multimodal processing.

Neural Computers (NCs) are a class of machine architectures in which computation, memory, and I/O are unified within a single, learned or neurally realized runtime substrate. Unlike conventional von Neumann computers, explicit program-executing agents, or world models trained purely as predictors, Neural Computers embody both the dynamic state and execution logic within neural or neural-inspired systems, including artificial neural networks, biological neurons, and mixed analog-digital substrates. NCs span theoretical, engineered, and biological realizations, extending from memory-augmented neural networks and differentiable neural computers (DNCs), through reservoir-based architectures and modular neural controllers, to fully embodied “neurons-on-a-chip” devices and brain-inspired optimization engines. The concept of a Completely Neural Computer (CNC) represents a mature, Turing-complete, and stably reprogrammable instance of this paradigm, presaging a shift toward models whose runtime state is the computer itself (Zhuge et al., 7 Apr 2026).

1. Definition, Motivation, and High-Level Abstractions

A Neural Computer (NC) is formally defined as a system comprising a pair of neural modules $(F_\theta, G_\theta)$ and a latent state $h_t$ that encodes working memory, context, and I/O within the same substrate. Time evolution proceeds according to the update equations: $$h_t = F_\theta(h_{t-1}, x_t, u_t), \quad x_{t+1} \sim G_\theta(h_t)$$ where $F_\theta$ updates the runtime state given previous state $h_{t-1}$, current input $x_t$, and action/conditioning $u_t$, while $G_\theta$ decodes the next observable (e.g., video frame, CLI/GUI output) (Zhuge et al., 7 Apr 2026).

Distinctive features:

• Unified latent runtime: There is no externally engineered program counter, memory bank, or device driver stack. Memory and computation are fused in the latent state.
• Executable model state: The model acts as both predictor and executor—contrast to conventional world models, which only learn environment dynamics.
• Programming interface: NCs potentially enable programming by demonstration, prompt, or constraint, rather than by code.

The long-term vision is the "Completely Neural Computer" (CNC), which achieves (i) Turing completeness, (ii) universal programmability (installation and durable reuse of routines), (iii) execution consistency (running behaviors only change when explicitly reprogrammed), and (iv) transparent machine-native semantics (Zhuge et al., 7 Apr 2026).

2. Computational Mechanisms and Memory Architectures

NCs subsume a spectrum of memory and control architectures, united by the principle of neural state manipulation:

• Differentiable Neural Computers (DNCs), Neural Turing Machines: Memory-augmented systems with explicit content-addressable memory, discrete or differentiable read/write heads, and a controller (often an RNN or MLP). DNCs support dynamic address allocation, temporal links for traversing data structures, and soft or hard attention (Tanneberg et al., 2021, Leon, 4 Mar 2026).
• Reservoir Memory Machines (RMMs): An echo state network (ESN) with a fixed random recurrent core and an explicit external memory of $L$ slots. At each step, a classifier determines when to write to or retrieve from memory. RMMs can efficiently simulate any regular language and tasks (copy, recall, FSM simulation) with minimal training data and convex training (Paaßen et al., 2020).
• Stateless DNCs and Transformers: Causal Transformer layers are mathematically equivalent to stateless DNCs with write-once external memory, memoryless controller, and multi-head content-based addressing. Keys and values are computed in feedforward fashion; evidence accumulation and memory recycling are absent. Cross-attention in encoder–decoder models corresponds to DNCs reading from independent memories (Tang et al., 27 Feb 2026).
• Modular Neural Computer (MNC): Designed for exact algorithmic computation, with an associative external memory (one-hot/scalar), one-hot gated modules (MLPs), and analytically specified control flow. All intermediate control, addressing, and module selection are encoded precisely, yielding deterministic behavior and transparent invariants (Leon, 4 Mar 2026).
• Biological/Spiking and Neuromorphic Substrates: NCs can be directly realized in biological hardware using living or synthetic neurons ("neurons-on-a-chip"), with sequential logic gates, flip-flops, and buffers constructed from spiking circuits, synaptic conductance, and buffer neurons (Basso et al., 2024). Larger neuromorphic compute substrates (e.g., IBM Neural Computer, Loihi, TrueNorth) generalize these principles to electronic or mixed-signal settings (Narayanan et al., 2020).
• Cortical-Inspired Multi-Step Devices: Zylberberg et al. proposed a biologically plausible, cortical circuit with production, memory, pointer, domain, and operation subnetworks, implementing variable assignment, equality testing, retrieval, and control flow for variable-oriented reasoning (Zylberberg et al., 2013).

The table below summarizes core architectural classes:

Architecture	Memory Mechanism	Control/Module
DNC/NTM	Addressable ext. mem	RNN controller
RMM	External slots	Fixed reservoir + classifier
Transformer (sDNC)	Write-once (stack)	Feedforward
MNC	Associative (one-hot)	Gated MLP modules
Neuronal devices	Synaptic weights	Production neurons

3. Learning, Abstraction, and Algorithmic Generalization

NCs license both learned and hand-designed algorithmic computation:

• End-to-end learning of runtime: Current NCs (as video models) are trained directly on end-to-end I/O traces (e.g., terminal sessions, GUI explorations), acquiring alignment of generated outputs to actions and primitive reasoning (e.g., window control, cursor movement). Key objective functions include diffusion-based generative losses (video models) and temporal contrastive losses for action alignment (Zhuge et al., 7 Apr 2026).
• Abstraction and control–data separation: Neural Harvard Computer (NHC) introduced a strict abstraction barrier between algorithmic (control flow, memory, bus) and data (input, ALU) modules. Evolutionary Natural Evolution Strategies (NES) search enables networks to reliably discover and generalize algorithms (sorting, arithmetic, search, copy) to longer sequences, new domains, and representations, exceeding the capabilities of standard DNCs (Tanneberg et al., 2021).
• Modular, analyzable programs: MNCs instantiate given algorithms as fixed graphs of compositional, analytically specified neural modules. This enables formally correct, exact execution, in contrast to learned, approximate solutions, but at the expense of generality and learnability (Leon, 4 Mar 2026).

Major challenges include:

• Attaining Turing completeness and unbounded dynamic memory in trainable systems, currently limited by fixed-latent neural models (Zhuge et al., 7 Apr 2026).
• Stable “installation” and reuse of routines—unlike current prompt-based models, which lack durable capability partitioning and persistent programming interfaces.
• Machine-native programming frameworks: beyond Weak API imitation toward register-based, differentiable controllers with auditability, explicit memory management, and neural-native combinators.

4. Physical Realizations and Embodied Computation

NCs include a wide spectrum of physical embodiments:

• Neuromorphic hardware: Systems such as IBM Neural Computer (INC), Loihi, TrueNorth, DYNAP, and BrainScaleS leverage FPGA, mixed-signal, or memristive elements to realize spiking neuron, event-driven, or parallel network architectures with customizable topologies, addressing both deep learning and event-driven, irregular workloads (Narayanan et al., 2020, Talbi, 22 May 2025).
• Neuronal logic circuits: Synthetic bio-neural platforms demonstrate all requisite digital logic gates (AND, NAND, NOT, flip-flops, latches), supporting binary signal propagation, synchronization (via buffer neurons), and metabolic self-regulation. Binary state is encoded by sustained spiking vs. silence. Metabolic burden per operation is quantified (0.5–0.8 a.u.) and compared to silicon logic. Gate operations occur in tens to hundreds of ms (six orders of magnitude slower than silicon) but with modularity, bio self-repair, and energy efficiency as key trade-offs (Basso et al., 2024).
• Spiking SNN metaheuristics (Nheuristics): Optimization algorithms for NP-hard problems (QUBO, TSP, CSP), swarm intelligence, and evolutionary search are mapped onto SNN architectures, leveraging the energy efficiency, latency, and event-driven operation of neuromorphic substrates. Rate or temporal encoding, spike-timing dependent plasticity (STDP), and distributed representations are central (Talbi, 22 May 2025).
• Memristive neuromorphic deep nets: “Group Scissor” delivers scalable deployment of large neural networks onto crossbar arrays via rank clipping (low-rank training) and group connection deletion (structured pruning), maximizing area and routing efficiency on neuromorphic chips (Wang et al., 2017).

5. Experimental Results and Comparative Analysis

Key empirical findings:

• Primitives acquired in learned NCs: Video-diffusion based NCs attain high-fidelity I/O alignment (PSNR up to 40.8 dB, SSIM 0.989), stateful cursor manipulation (placement accuracy 98.7%), and short-horizon control. Arithmetic performance in end-to-end form remains low unless externally conditioned, indicating that rote execution is separable from underlying reasoning (Zhuge et al., 7 Apr 2026).
• Algorithmic generalization with abstraction: NHC achieves perfect scaling and strong generalization to out-of-distribution task configurations and representations, solving tasks (search, sort, arithmetic) to lengths/complexities well beyond training, with 73–100% perfect run rates. Standard DNC baselines fail to reliably scale (Tanneberg et al., 2021).
• Reservoir Memory Machines outperform ESNs and train with negligible cost: LDN-RMMs solve all regular/fixed algorithmic tasks accurately, with convex training in seconds, and provide formal guarantees for regular language recognition and variable-length sequence manipulation (Paaßen et al., 2020).
• Neuronal logic on bio-hardware: All digital logic elements (AND, NAND, SR latches, D flip-flops) are demonstrated in networks of 3–4 Izhikevich model neurons, with operational timings set by synaptic time constants. Energy-per-operation is far lower than conventional electronics; scalability is currently limited to ~10²–10⁴ neuron modules (Basso et al., 2024).

Empirical performance metrics and comparison table:

System/Task	Key Metric	Result
Diff. Video NC	SSIM (GUIWorld, CUA)	0.885
Diff. Video NC	Cursor accuracy	98.7%
NHC	Sort (ℓ=1000)	73–87% perfect runs
RMM	Copy/Recall, tasks	Perfect/near-perfect RMSE
Bio Neuronal Logic	Flip-flop op time	~100 ms per edge
Bio Neuronal Logic	Metabolic op cost	0.5–0.8 a.u./neuron/op

6. Limitations, Open Questions, and Future Directions

Unresolved issues and directions for research include:

• Turing completeness and dynamic memory: Existing NCs with frozen architectures cannot encode unbounded computation or storage; explicit architectural augmentation is needed.
• Capability installation and governance: Mature CNCs require explicit interfaces for routine registration, controlled updates, behavior partitioning, and traceable reprogramming (Zhuge et al., 7 Apr 2026).
• Interpretability and auditability: Neural programming and execution must offer inspection and rollback mechanisms for critical applications.
• Physical scalability: Neuromorphic and neuronal logic circuits face challenges with wiring, speed, and faithful scaling. Integration with conventional CMOS and emergence of hybrid platforms (e.g., CMOS–neuron chips) offer a plausible path.
• Program synthesis and modularization: MNC and modular architectures suggest a trajectory toward neural program synthesis: automated induction of module interfaces and control graphs that preserve formal correctness and generalizability (Leon, 4 Mar 2026, Tanneberg et al., 2021).
• Energy, speed, and sustainability trade-offs: Bio-hardware (neurons, synapses) provides extreme energy and thermal advantages; speed remains orders of magnitude lower than current electronics. Synchronization and buffering overheads require formal design principles in deep logic (Basso et al., 2024).

7. Broader Impact and Paradigm Implications

Neural Computers instantiate an emerging paradigm for computation:

• Unified, learned execution blurs the boundary between program, memory, and data, grounding the “computer” in a high-dimensional neural substrate rather than an engineered stack.
• The programming interface shifts from imperative code to demonstration, constraint, and prompt, potentially making interaction with computation more accessible and specification-centric.
• Multimodal processing and compositionality become native axes of computation, simplifying fusion of perception, action, and reasoning tasks.
• If CNCs reach operational maturity, routine automation, multimodal integrated workflows, and dynamic system updates could see substantial advances, presaging new machine architectures in both physical and virtual domains (Zhuge et al., 7 Apr 2026).

Interpretability, governance, and safety grow in importance as the locus of computation shifts from discrete, inspectable registers to opaque latent state, demanding further research on architecture, specification, and verification.

References:

• (Zhuge et al., 7 Apr 2026) Neural Computers
• (Leon, 4 Mar 2026) Modular Neural Computer
• (Tanneberg et al., 2021) Evolutionary Training and Abstraction Yields Algorithmic Generalization of Neural Computers
• (Paaßen et al., 2020) Reservoir Memory Machines as Neural Computers
• (Tang et al., 27 Feb 2026) Transformers are Stateless Differentiable Neural Computers
• (Basso et al., 2024) Embodied Biocomputing Sequential Circuits with Data Processing and Storage for Neurons-on-a-chip
• (Wang et al., 2017) Group Scissor: Scaling Neuromorphic Computing Design to Large Neural Networks
• (Talbi, 22 May 2025) Neuromorphic-based metaheuristics: A new generation of low power, low latency and small footprint optimization algorithms
• (Zylberberg et al., 2013) A neuronal device for the control of multi-step computations
• (Narayanan et al., 2020) Overview of the IBM Neural Computer Architecture