Predictive Coding Framework

• Predictive coding is a framework that minimizes the gap between sensory inputs and model predictions using hierarchical generative structures.
• It operates by minimizing variational free energy through local, gradient-based error correction, aligning bottom-up signals with top-down predictions.
• Applications span vision, memory, control, compression, and associative learning, bridging neural circuit models and deep network architectures.

Predictive coding is a computational and neuroscientific framework in which hierarchical systems minimize the error between sensory inputs and top-down predictions generated by internal generative models. This inference principle underlies both neural and artificial adaptive systems, supporting perception, action, memory, and learning. Predictive coding realizes approximate Bayesian inference via local, recurrent error correction and has been linked to efficient coding, unsupervised learning, and flexible deep network training.

1. Hierarchical Generative Models and Variational Free Energy

Predictive coding posits that a system consists of a hierarchy of latent variables $\{x_L, x_{L-1}, \dots, x_0\}$ governed by a generative model: $$p(x_0, \ldots, x_L) = p(x_L) \prod_{l=0}^{L-1} p(x_l | x_{l+1})$$ where each conditional is typically Gaussian: $$p(x_l \mid x_{l+1}) = \mathcal{N}(x_l; \theta_{l+1} f(x_{l+1}), \Sigma_l)$$ with learnable parameters $\theta_{l+1}$ and element-wise nonlinearity $f$ (Millidge et al., 2022, Jiang et al., 2021). The core inference and learning principle is minimization of the variational free energy (VFE), which for a Laplace (delta) posterior reduces to the sum of squared prediction errors: $$F = \frac{1}{2} \sum_{l=0}^{L-1} \big\| x_l - \theta_{l+1} f(x_{l+1}) \big\|^2$$ This energy is minimized over both latent activities and parameters, driving the system to states that best explain observed data under the generative model (Millidge et al., 2022, Marino, 2020, Jiang et al., 2021).

2. Inference and Learning Dynamics: Local Error Minimization

Inference proceeds by local gradient descent on the free energy with respect to each latent variable: $$\frac{dx_l}{dt} = -\frac{\partial F}{\partial x_l} = -\epsilon_l + \theta_l^\top [f'(x_l) \odot \epsilon_{l-1}]$$ where $\epsilon_l = x_l - \theta_{l+1} f(x_{l+1})$ are prediction errors and $\odot$ is the Hadamard product. Weight updates are also local: $$\Delta\theta_{l+1} = \eta\,\epsilon_l\,f(x_{l+1})^\top$$ requiring only pre- and post-synaptic variables (Millidge et al., 2022, Marino, 2020, Salvatori et al., 2021, Millidge et al., 2020). These rules generalize to arbitrary directed acyclic graphs and allow unsupervised, supervised, and associative learning within a unified framework.

3. Functional Architecture: Prediction, Error, and Hierarchy

Predictive coding architectures are characterized by:

• Top-down feedback connections transmitting predictions generated at each hierarchical level: $$p(x_0, \ldots, x_L) = p(x_L) \prod_{l=0}^{L-1} p(x_l | x_{l+1})$$0
• Feedforward pathways conveying local prediction errors, $$p(x_0, \ldots, x_L) = p(x_L) \prod_{l=0}^{L-1} p(x_l | x_{l+1})$$1, upward for correction (Jiang et al., 2021).
• Separation of value ("state") units and error units within each layer.
• Bidirectional models that integrate both generative (top-down) and discriminative (bottom-up) predictive pathways via a composite energy function, supporting robust inference and multimodal learning (Oliviers et al., 29 May 2025).
• Sparsity and acyclicity constraints can be imposed for causal discovery, with interventions implemented by zeroing prediction errors at manipulated nodes (Salvatori et al., 2023).

This architecture supports convergent inference, in which representations are recursively updated to minimize mismatches between expectation and sensory evidence.

4. Connections to Variational Inference, VAEs, and Deep Learning

The variational free energy minimized in predictive coding is formally equivalent to the evidence lower bound (ELBO) in variational autoencoders (VAEs): $$p(x_0, \ldots, x_L) = p(x_L) \prod_{l=0}^{L-1} p(x_l | x_{l+1})$$2 with the bottom-up prediction error corresponding to the reconstruction loss and the top-down error to the KL term (Marino, 2020, Millidge et al., 2022). Amortized inference in VAEs replaces iterative gradient-based updates with a learned encoder, and recent work extends predictive coding with amortized and hybrid inference strategies that unify fast feedforward and slow recurrent processing (Tschantz et al., 2022).

Predictive coding also encompasses a breadth of machine learning paradigms. It approximates backpropagation in multilayer networks under suitable initialization and boundary conditions (Salvatori et al., 2021, Millidge et al., 2022), and has been shown to yield exact weight updates on deep, convolutional, and recurrent architectures (Salvatori et al., 2021). Adjustments such as learning distinct top-down weights or relaxing error-unit pairings retain performance and further biological plausibility (Millidge et al., 2020).

5. Computational and Biological Properties

Predictive coding delivers several computational and biological advantages:

• Local, parallel learning: All updates require only local variables, enabling asynchronous, hardware-efficient, and biologically plausible implementations (Millidge et al., 2022, Millidge et al., 2020).
• Associative and generative flexibility: A single network structure can perform classification, generation, denoising, inpainting, data imputation, and associative recall, depending only on the clamping of inputs and outputs (Salvatori et al., 2021, Millidge et al., 2022).
• Trust-region adaptation: PC can outperform vanilla backpropagation in saddle-escaping and robustness near minima by acting as an adaptive trust-region method, interpolating between first- and second-order gradient directions determined by the local Fisher information (Innocenti et al., 2023).
• Causal inference: Predictive coding networks can perform do-queries and causal discovery by modifying the inference process, including learning graph structure end-to-end from data (Salvatori et al., 2023).
• Uncertainty quantification: Bayesian predictive coding (BPC) extends PC to maintain parameter posteriors, yielding closed-form, Hebbian-local updates and calibrated uncertainty in predictions, comparable to or exceeding other Bayesian deep learning methods in both accuracy and convergence (Tschantz et al., 31 Mar 2025).

6. Applications: Perception, Memory, Control, and Compression

Predictive coding has been operationalized in multiple domains:

• Vision and sensory models: Predictive coding accounts for classical and extra-classical effects in early sensory areas, explains contextual modulation, and supports hierarchical learning of feature maps in cortex (Jiang et al., 2021).
• Associative memory: Hierarchical predictive coding networks achieve robust auto- and hetero-associative memory, outperforming backpropagation-trained autoencoders and modern Hopfield networks in denoising, partial completion, and multi-modal retrieval (Salvatori et al., 2021).
• Control and reinforcement learning: PC provides a unified model for perception, inference, and motor control via extensions to spatiotemporal predictive coding and active inference, supporting tasks such as path planning, stabilization, and representation learning for sparse rewards (Millidge et al., 2022, Lu et al., 2019, Kuo et al., 24 Oct 2025).
• Video and image compression: Classical predictive coding principles are at the heart of state-of-the-art video codecs. Deep neural implementations, such as the Residual Deep Animation Codec, outperform standard codecs by learning structured motion predictors and applying temporal predictive coding to residuals (Konuko et al., 2023, Huang et al., 2019).
• 3D environment modeling: Predictive coding objectives leveraging masked prediction of spatial zones enable embodied agents to form robust environment-level representations that generalize efficiently to new navigation and manipulation tasks (Ramakrishnan et al., 2021).

7. Biological Implementation and Empirical Evidence

Predictive coding maps naturally onto cortical microcircuits:

• Pyramidal neurons in deep layers generate predictions, while superficial error units signal mismatches (Jiang et al., 2021, Marino, 2020).
• Local Hebbian plasticity implements synaptic updates, and inhibitory interneuron dynamics correspond to adaptive normalization and precision gating (Marino, 2020).
• Extensions to spiking neural networks reveal that spike-timing dependent plasticity alone suffices to realize predictive coding, with error minimization manifesting as learned suppression of predictable stimuli via inhibitory circuits (Masumori et al., 2019).

Empirical studies show that prediction-error minimization accounts for diverse physiological phenomena, including receptive field structure, cross-modal integration, adaptation to changing uncertainty, and associative memory formation (Jiang et al., 2021, Salvatori et al., 2021, Marino, 2020).

---

References

• (Millidge et al., 2022) Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?
• (Marino, 2020) Predictive Coding, Variational Autoencoders, and Biological Connections
• (Salvatori et al., 2021) Predictive Coding Can Do Exact Backpropagation on Convolutional and Recurrent Neural Networks
• (Millidge et al., 2020) Relaxing the Constraints on Predictive Coding Models
• (Jiang et al., 2021) Predictive Coding Theories of Cortical Function
• (Salvatori et al., 2021) Associative Memories via Predictive Coding
• (Tschantz et al., 31 Mar 2025) Bayesian Predictive Coding
• (Innocenti et al., 2023) Understanding Predictive Coding as an Adaptive Trust-Region Method
• (Oliviers et al., 29 May 2025) Bidirectional predictive coding
• (Tschantz et al., 2022) Hybrid Predictive Coding: Inferring, Fast and Slow
• (Salvatori et al., 2023) Predictive Coding beyond Correlations
• (Konuko et al., 2023) Predictive Coding For Animation-Based Video Compression
• (Lu et al., 2019) Predictive Coding for Boosting Deep Reinforcement Learning with Sparse Rewards
• (Ramakrishnan et al., 2021) Environment Predictive Coding for Embodied Agents
• (Masumori et al., 2019) Predictive Coding as Stimulus Avoidance in Spiking Neural Networks
• (Kuo et al., 24 Oct 2025) Predictive Coding Enhances Meta-RL To Achieve Interpretable Bayes-Optimal Belief Representation Under Partial Observability
• (Ratzon et al., 12 Nov 2025) Multi-step Predictive Coding Leads To Simplicity Bias
• (Huang et al., 2019) Predictive Coding Networks Meet Action Recognition