Neural Generative Coding Overview

• Neural Generative Coding is a computational framework that constructs hierarchical generative models using explicit state and error neurons with locally-driven, Hebbian-style updates.
• Its iterative inference mechanism minimizes prediction errors across layers, refining latent representations for enhanced reconstruction and generalization.
• Extensions like Conv-NGC and Active NGC adapt the framework for visual and reinforcement learning, demonstrating robustness and efficiency over traditional backpropagation approaches.

Neural Generative Coding (NGC) is a computational framework for learning hierarchical, explicit generative models using principles derived from predictive processing and biological neural circuits. Unlike backpropagation-based systems, NGC employs strictly local, Hebbian-style update rules and iterative inference dynamics involving prediction errors, making it both biologically plausible and applicable to a range of unsupervised and reinforcement learning settings (Ororbia et al., 2020, Ororbia et al., 2021, Ororbia et al., 2022).

1. Theoretical Foundations and Model Objective

NGC constructs an explicit probabilistic generative model characterized by a layered structure of “state” and “error” variables. Each layer in the hierarchy consists of state neurons $z^\ell$ representing latent variables and error neurons $e^\ell$ encoding mismatches between predicted and actual states. The joint density over observed data $x$ and latent states $\{z^\ell\}_{\ell=1}^L$ is defined as

$$p(x,z^1,\ldots,z^L) = p(x|z^1)\prod_{\ell=1}^{L-1} p(z^\ell|z^{\ell+1})p(z^L),$$

where $p(x|z^1)$ is typically Bernoulli (for binarized data) and $p(z^\ell|z^{\ell+1})$ is Gaussian with mean given by top–down generative weights. The learning objective is to maximize the complete-data log-likelihood, which decomposes into a sum of terms capturing reconstruction fidelity and hierarchical consistency (Ororbia et al., 2020).

In Conv-NGC, this log-likelihood is operationalized as a multi-layer energy function: $$E(x,\{z^\ell\}) = \sum_{\ell=0}^L \frac{1}{2} \|z^\ell - \bar z^\ell\|_2^2,$$ where $\bar z^\ell$ is the top–down prediction of layer $\ell$ (Ororbia et al., 2022).

2. Network Architecture and Components

The canonical NGC network comprises $L$ layers of state neurons, each paired with error neurons. Interlayer connections are mediated by forward (generative) weights $W^{\ell+1}$ and error weights $E^{\ell+1}$. In Conv-NGC, these weights generalize to convolutional/deconvolutional kernels, and layers are realized as feature maps with spatial structure.

Key components include:

• State Pathway: Carries hierarchical representations $z^\ell$.
• Error Pathway: Propagates bottom-up prediction errors $e^\ell$ for local learning.
• Synaptic Modulation: Synaptic scaling factors $M_W$, $M_E$ maintain weight stability and normalization.
• Lateral Competition: Inhibition within state layers enforces representational sparsity and decorrelation (Ororbia et al., 2020).

3. Iterative Inference and Local Learning Rules

NGC eschews global backpropagation in favor of iterative inference combining top–down predictions with bottom–up error-driven corrections. For each data point:

• The bottom layer $z^0$ is clamped to the observation.
• Higher layers initialize latents, then repeatedly refine them by:
  • Predicting child layer $\ell-1$ via the generative map,
  • Calculating local prediction errors $e^{\ell-1} = z^{\ell-1} - \bar z^{\ell-1}$,
  • Updating $z^\ell$ by combining top–down, bottom–up, lateral, and leak terms:

  $$z^\ell_i \leftarrow z^\ell_i + \beta\left[ \sum_j E^\ell_{ij}e^{\ell-1}_j - e^\ell_i - \sum_{j\ne i}V^\ell_{ij}\phi^\ell(z^\ell_j) + V^\ell_{ii}\phi^\ell(z^\ell_i) - \gamma z^\ell_i\right]$$ - This dynamic continues for $T$ steps until settled states minimize prediction errors (~free-energy).

Synaptic updates are local, Hebbian-type rules, for example: $$\Delta W^\ell = e^\ell[\,\phi^{\ell+1}(z^{\ell+1})\,]^T$$ Where error neurons modulate weight changes proportional to presynaptic activity (Ororbia et al., 2020, Ororbia et al., 2022).

4. Extensions: Convolutional and Active NGC

Convolutional NGC (Conv-NGC) adapts the framework to natural images by translating state and error neurons into feature maps and using convolutional/deconvolutional kernels for top–down and bottom–up processing. Inference corresponds to iterative layerwise refinement; learning remains entirely local and is performed primarily via Adam optimizer on layerwise errors (Ororbia et al., 2022). Conv-NGC demonstrates strong out-of-distribution reconstruction and sample efficiency due to dynamic inference and parameter parsimony.

Active NGC (ANGC) extends NGC principles to reinforcement learning, particularly backpropagation-free control:

• The architecture deploys separate, structurally identical NGC circuits for a Q-controller and a forward (world) model.
• Inference and learning remain local; Q-learning target estimation is implemented via NGC-inference-based regression.
• Action selection and planning are posed as inference under a hybrid extrinsic (instrumental) and epistemic (curiosity) reward:

$r_t = \alpha_i r^i_t + \alpha_e r^e_t$

where $r^e_t$ is the sum of generator error norms (epistemic surprise).

• ANGC achieves competitive or superior performance to DQN, ICM, and RnD on classical control and robotic tasks, especially under sparse reward regimes (Ororbia et al., 2021).

5. Algorithmic Workflow

The NGC learning algorithm consists of the following stages:

1. Initialization: Clamp input, initialize latents.
2. Top–Down Prediction: Propagate predictions downward through generative weights.
3. Error Calculation: Compute prediction errors across layers.
4. Iterative Inference: Dynamically refine latent states through local update rules.
5. Local Hebbian Updates: Adjust weights using layer-local products of errors and activations.
6. Synaptic Scaling: Normalize columns of $W$ after each update for stability.

Active NGC introduces interleaved updates between controller and generator circuits, following experience replay for reinforcement learning (Ororbia et al., 2021).

6. Empirical Performance and Analysis

Quantitative evaluation covers image reconstruction, likelihood estimation, classification, pattern completion, and RL benchmarks:

• Generative Modeling: On MNIST, KMNIST, and FMNIST, NGC attains leading test binary cross-entropy (BCE) and competitive log-likelihoods versus VAEs, GAN-AE, and other baselines. NGC–PDH achieves the lowest BCE on MNIST and high representational sparsity, supporting improved downstream classification and pattern completion (Ororbia et al., 2020).
• Convolutional Tasks: Conv-NGC outperforms convolutional autoencoders trained with backpropagation in MSE and SSIM for Color-MNIST, CIFAR-10, and SVHN, and is particularly robust to out-of-distribution shifts (e.g., SVHN→CIFAR-10; SSIM=0.98 for Conv-NGC vs. 0.69 for Conv-AE). Robustness is attributed to iterative refinement rather than single network mappings (Ororbia et al., 2022).
• Reinforcement Learning: ANGC demonstrated faster solution and lower variance in CartPole and MountainCar relative to DQN, and is competitive with ICM/RnD for more complex tasks such as LunarLander and Robot-Arm (Ororbia et al., 2021).

7. Strengths, Limitations, and Biological Relevance

Strengths:

• No weight-transport or global gradients; strictly local updates and synaptic scaling.
• Biological plausibility through explicit error pathways and dynamics reminiscent of cortical predictive coding.
• Modular adaptability to diverse settings—unsupervised modeling, reinforcement learning, image domains.
• Superior out-of-distribution generalization and parameter efficiency in Conv-NGC.

Limitations:

• Iterative inference incurs computational overhead (typically $K=10$–$60$ steps).
• Doubled parameter count due to dual weight matrices $(W,E)$ per layer.
• High-dimensional data require specialized architectures (e.g., Conv-NGC for images).
• Scaling to continuous actions or partial observability necessitates architectural extensions.

Potential extensions: Amortized inference for faster prediction, convolutional/temporal NGC for richer data structures, and coupling with policy gradients for continuous-control tasks (Ororbia et al., 2020, Ororbia et al., 2022, Ororbia et al., 2021).

NGC provides both a theoretical and empirical alternative to conventional deep learning approaches, emphasizing local computation and biologically inspired algorithms, with demonstrated competence across generative modeling and reinforcement learning domains.