Hierarchical Predictive Coding

• Hierarchical predictive coding is a layered framework where higher levels predict lower-level inputs and prediction errors drive local learning.
• It incorporates precision-weighted inference, using uncertainty estimates to locally scale synaptic updates and state adjustments.
• HPC principles bridge neuroscience and machine learning, underpinning models from cortical circuits to efficient deep network architectures.

Hierarchical predictive coding (HPC) is a layered inference-and-learning framework in which higher levels of a generative hierarchy predict the activity of lower levels, while lower levels transmit prediction errors that update latent states, synaptic parameters, and, in precision-weighted formulations, uncertainty estimates. In computational neuroscience, HPC is a theory of neocortical function grounded in hierarchical Bayesian inference and variational free energy; in machine learning, it is a local, distributed optimization scheme that can be formulated with deterministic state dynamics, learnable precisions, and deep parametric mappings (Jiang et al., 2021, Ofner et al., 2021, Hosseini et al., 2020).

1. Core architecture and message passing

Classical HPC assumes that each cortical or network level encodes latent causes for the level below. In the Rao–Ballard form reviewed in "Predictive Coding Theories of Cortical Function" (Jiang et al., 2021), the variables at level $l$ are observed activity $x^{l}$, latent causes $z^{l}$, and generative weights $U^{l}$. The hierarchical generative model is

$$x^{l} = f\!\big(U^{l} z^{l}\big) + n_{x}^{l}, \quad n_{x}^{l} \sim \mathcal{N}(0,\Sigma_{x}^{l}),$$

and

$$z^{l} = f\!\big(U^{l+1} z^{l+1}\big) + n_{z}^{l}, \quad n_{z}^{l} \sim \mathcal{N}(0,\Sigma_{z}^{l}).$$

The associated local prediction errors are

$$\varepsilon_{x}^{l} = x^{l} - f(U^{l} z^{l}), \quad \varepsilon_{z}^{l} = z^{l} - f(U^{l+1} z^{l+1}).$$

This establishes the canonical bidirectional organization: feedback pathways convey predictions, and feedforward pathways convey prediction errors. The review "Hierarchical Predictive Coding Models in a Deep-Learning Framework" names the resulting communication pattern the Rao–Ballard protocol, in which representation units and error units are explicitly separated and interact through prediction, prediction-error, lateral target, and lateral target-error links (Hosseini et al., 2020).

Deterministic layered HPC uses analogous notation. In "Predictive coding, precision and natural gradients" (Ofner et al., 2021), each layer $l$ maintains a mean activity $\mu_l$ and parameters $\theta_l$, predicts the activity of the layer below, and computes layerwise errors as

$x^{l}$0

The same work states that HPC is a deterministic, layered inference-and-learning framework that minimizes locally computed, precision-weighted prediction errors throughout a hierarchy. Each layer estimates the precision of its own errors and uses that precision to scale both inference and learning rates, so uncertainty is not an external annotation but part of the hierarchy’s state (Ofner et al., 2021).

2. Free energy, inference, and local learning

Under Gaussian assumptions, HPC minimizes a sum of prediction-error terms. A standard hierarchical energy is

$x^{l}$1

or, with matrix precisions,

$x^{l}$2

where $x^{l}$3 and $x^{l}$4 implement precision-weighting (Jiang et al., 2021). This formalism already contains the ingredients often associated with later HPC variants: hierarchical generative mappings, priors on latent states and weights, and reliability-weighted message passing.

Gradient descent on this objective yields the standard message-passing dynamics. With general nonlinearity $x^{l}$5 and Jacobian $x^{l}$6, the latent-state update is

$x^{l}$7

and the local synaptic rule is

$x^{l}$8

These updates are local in the sense emphasized by the cortical-function review: the presynaptic term is the latent state, the postsynaptic term is the prediction error, and learning operates on a slower timescale than inference (Jiang et al., 2021).

A complementary formal treatment appears in "A Theoretical Framework for Inference and Learning in Predictive Coding Networks" (Millidge et al., 2022). There, a predictive coding network (PCN) has activities $x^{l}$9, predictions $z^{l}$0, errors $z^{l}$1, and free energy

$z^{l}$2

During supervised training, $z^{l}$3 and $z^{l}$4 are clamped, and inference updates the hidden activities according to

$z^{l}$5

After convergence to an equilibrium $z^{l}$6, learning uses the local rule

$z^{l}$7

That paper characterizes prospective configuration as a generalized expectation-maximization procedure and proves convergence to critical points of the backpropagation loss under a sufficiently small learning rate and an initialization satisfying the stated energy-gradient bound. It also gives an exact target-propagation equivalence in the input-unconstrained invertible case, with

$z^{l}$8

showing that HPC interpolates between backpropagation-like and target-propagation-like regimes rather than being reducible to either one (Millidge et al., 2022).

3. Precision weighting, variational inference, and natural gradients

Precision is the principal extension that distinguishes uncertainty-aware HPC from fixed-variance error minimization. In the precision-learning formulation of (Ofner et al., 2021), the free energy is written as

$z^{l}$9

with a slight log-term inconsistency noted in the appendix of that work. Precision enters both the inference dynamics and the parameter updates, so layer-local learning rates are effectively scaled by $U^{l}$0. The same paper gives the precision update

$U^{l}$1

where $U^{l}$2 denotes the precision-weighted error. This makes precision learning a local inverse-variance estimation problem driven by prediction-error statistics (Ofner et al., 2021).

The same study makes the natural-gradient connection explicit. For activities,

$U^{l}$3

and for weights,

$U^{l}$4

Thus, the Fisher information with respect to activities equals the estimated precision at that layer, while the Fisher information with respect to weights factorizes into the layer precision and the variance of parent-layer activities. The paper argues that this gives precision-weighted HPC a distributed approximation to natural gradient descent, without global Fisher-matrix storage or inversion (Ofner et al., 2021).

"PredProp: Bidirectional Stochastic Optimization with Precision Weighted Predictive Coding" extends this idea by preconditioning both state and weight gradients with inverse covariance estimates. Its inference update is

$U^{l}$5

and its weight update for a single-layer decoder is

$U^{l}$6

PredProp explicitly states that hierarchical predictive coding layers are optimised individually using local errors, so the required precisions factorize over hierarchical layers; it then generalizes the same factorization within deep decoders embedded inside a PCN layer (Ofner et al., 2021).

Two recent formulations broaden the same theme. "Predictive Coding with Bayesian Priors via Proximal Gradients" recasts predictive coding as proximal-gradient descent on a regularized MAP objective and identifies the activation function with the proximal operator of the prior; in that framework, Gaussian priors yield linear shrinkage, Laplace priors yield soft-thresholding, and nonnegativity constraints yield ReLU (Bullo, 6 Jun 2026). "Closed-form predictive coding via hierarchical Gaussian filters" expresses deep predictive coding as a generalized hierarchical Gaussian filter with one-shot variational updates of posterior means and precisions, removing iterative relaxation and restoring precision-weighted message passing. On FashionMNIST, that paper states that the resulting networks approach backpropagation in epoch-level wall-clock cost while converging in fewer epochs, and outperform it on online, data efficiency, and concept-drift tasks (Baskakovs et al., 19 May 2026).

The variational interpretation runs throughout this literature. In the appendix of (Ofner et al., 2021), the free energy is written as a KL divergence,

$U^{l}$7

and under a Dirac-delta posterior reduces to the same precision-weighted error sum. This makes HPC simultaneously a machine-learning loss, a variational free-energy objective, and a deterministic ELBO surrogate (Ofner et al., 2021).

4. Cortical circuitry, spiking implementations, and empirical neurophysiology

The canonical cortical mapping assigns distinct computational roles to laminar populations. In the review of cortical predictive coding (Jiang et al., 2021), deep-layer (5/6) pyramidal neurons encode latent representations $U^{l}$8 and send predictions via feedback, whereas superficial-layer (2/3) pyramidal neurons compute and convey prediction errors via feedforward projections. Because errors can be positive or negative, the same review proposes two subclasses analogous to ON/OFF pathways. The cited empirical pattern is that predictable stimuli enhance alpha/beta in deep layers and suppress superficial activity, whereas unpredictable stimuli elicit gamma and spiking in superficial layers (Jiang et al., 2021).

The same review summarizes several classes of experimental support. In early visual cortex, endstopping and surround effects arise when higher areas predict center-surround statistics well enough to suppress error responses; removing feedback abolishes this suppression in Rao–Ballard simulations. In mismatch paradigms, superficial layer 2/3 neurons in rodent V1 show depolarizing or hyperpolarizing mismatch responses, while deep layer 5/6 neurons show integrated predictive activity and weaker mismatch selectivity. Similar mismatch and prediction-error phenomena are summarized for auditory cortex and primate inferotemporal cortex (Jiang et al., 2021).

"Dendritic predictive coding: A theory of cortical computation with spiking neurons" replaces explicit error-unit populations with local dendritic computations. In that model, basal compartments compute bottom-up errors through local E/I balance and lateral inhibition,

$U^{l}$9

while apical compartments compute top-down prediction errors,

$$x^{l} = f\!\big(U^{l} z^{l}\big) + n_{x}^{l}, \quad n_{x}^{l} \sim \mathcal{N}(0,\Sigma_{x}^{l}),$$0

Prediction-unit dynamics then become

$$x^{l} = f\!\big(U^{l} z^{l}\big) + n_{x}^{l}, \quad n_{x}^{l} \sim \mathcal{N}(0,\Sigma_{x}^{l}),$$1

The proposed interneuron mapping is specific: PV basket cells support tight local E/I balance and lateral competition, SST interneurons predominantly inhibit apical dendrites, and VIP interneurons implement disinhibitory gating. The paper argues that mismatch responses, extra-classical receptive-field effects, and sparse efficient spiking can emerge without distinct error-unit populations (Mikulasch et al., 2022).

A concrete two-area instantiation appears in "Predictive coding in area V4: dynamic shape discrimination under partial occlusion" (Choi et al., 2016). That model places V4 and PFC in a two-level predictive hierarchy, with initial V4 responses driven solely by bottom-up input at approximately $$x^{l} = f\!\big(U^{l} z^{l}\big) + n_{x}^{l}, \quad n_{x}^{l} \sim \mathcal{N}(0,\Sigma_{x}^{l}),$$2–$$x^{l} = f\!\big(U^{l} z^{l}\big) + n_{x}^{l}, \quad n_{x}^{l} \sim \mathcal{N}(0,\Sigma_{x}^{l}),$$3 ms, and delayed V4 responses at approximately $$x^{l} = f\!\big(U^{l} z^{l}\big) + n_{x}^{l}, \quad n_{x}^{l} \sim \mathcal{N}(0,\Sigma_{x}^{l}),$$4–$$x^{l} = f\!\big(U^{l} z^{l}\big) + n_{x}^{l}, \quad n_{x}^{l} \sim \mathcal{N}(0,\Sigma_{x}^{l}),$$5 ms incorporating PFC feedback. It reports that PFC responses are strongest for occluded stimuli and delayed responses in V4 are less sensitive to occlusion. The underlying quadratic objective is

$$x^{l} = f\!\big(U^{l} z^{l}\big) + n_{x}^{l}, \quad n_{x}^{l} \sim \mathcal{N}(0,\Sigma_{x}^{l}),$$6

which combines bottom-up V4 likelihood terms and top-down PFC predictive constraints (Choi et al., 2016).

5. Deep-learning, sparse, multimodal, and control-oriented implementations

In machine learning, HPC has been implemented both as a direct architectural principle and as a local optimization scheme inserted into deep networks. The survey (Hosseini et al., 2020) distinguishes predictive coding networks (PCNs), which augment CNN-like models with recurrent predictive-update cycles between adjacent layers, from PredNet, a hierarchical recurrent model for next-frame prediction. PredNet uses representation modules, error modules, and convolutional LSTMs, but the survey notes that its connectivity deviates from the Rao–Ballard protocol: higher-layer representations project directly to lower-layer representations rather than to error units, and lower-layer errors project upward to error units rather than to representations (Hosseini et al., 2020).

Sparse hierarchical variants make the top-down term explicit. "Effect of top-down connections in Hierarchical Sparse Coding" introduces 2-Layers Sparse Predictive Coding (2L-SPC), in which each layer minimizes a local reconstruction term, a top-down consistency term, and an $$x^{l} = f\!\big(U^{l} z^{l}\big) + n_{x}^{l}, \quad n_{x}^{l} \sim \mathcal{N}(0,\Sigma_{x}^{l}),$$7 penalty. Across four databases, that work reports that the overall prediction error generated by 2L-SPC is lower thanks to the feedback mechanism as it transfers prediction error between layers; the inference stage is faster to converge than in the Hierarchical Lasso model; and learning is also accelerated (Boutin et al., 2020). A more recent study, "Accelerating Hierarchical Sparse Predictive Coding with Hybrid Amortized Inference," keeps the hierarchical sparse energy fixed and compares ISTA, MFISTA, LISTA-style amortization, and a hybrid of amortized initialization with short corrective recurrence. For the two-layer default on Fashion-MNIST, the hybrid setting $$x^{l} = f\!\big(U^{l} z^{l}\big) + n_{x}^{l}, \quad n_{x}^{l} \sim \mathcal{N}(0,\Sigma_{x}^{l}),$$8 is reported with test loss $$x^{l} = f\!\big(U^{l} z^{l}\big) + n_{x}^{l}, \quad n_{x}^{l} \sim \mathcal{N}(0,\Sigma_{x}^{l}),$$9, reconstruction error $$z^{l} = f\!\big(U^{l+1} z^{l+1}\big) + n_{z}^{l}, \quad n_{z}^{l} \sim \mathcal{N}(0,\Sigma_{z}^{l}).$$0, and latency $$z^{l} = f\!\big(U^{l+1} z^{l+1}\big) + n_{z}^{l}, \quad n_{z}^{l} \sim \mathcal{N}(0,\Sigma_{z}^{l}).$$1 ms/sample, and the paper concludes that a shallow LISTA-style initializer plus short corrective recurrence improves over pure amortization while remaining much faster than long iterative inference (Fujita, 26 Jun 2026).

Hierarchical predictive coding has also been adapted to compression. "Deep Hierarchical Video Compression" defines a multiscale latent hierarchy $$z^{l} = f\!\big(U^{l+1} z^{l+1}\big) + n_{z}^{l}, \quad n_{z}^{l} \sim \mathcal{N}(0,\Sigma_{z}^{l}).$$2 with factorization

$$z^{l} = f\!\big(U^{l+1} z^{l+1}\big) + n_{z}^{l}, \quad n_{z}^{l} \sim \mathcal{N}(0,\Sigma_{z}^{l}).$$3

so that each scale combines same-scale temporal context and lower-scale spatial context. The reported complexity numbers on $$z^{l} = f\!\big(U^{l+1} z^{l+1}\big) + n_{z}^{l}, \quad n_{z}^{l} \sim \mathcal{N}(0,\Sigma_{z}^{l}).$$4p videos are: kMACs/pixel $$z^{l} = f\!\big(U^{l+1} z^{l+1}\big) + n_{z}^{l}, \quad n_{z}^{l} \sim \mathcal{N}(0,\Sigma_{z}^{l}).$$5 for DHVC versus $$z^{l} = f\!\big(U^{l+1} z^{l+1}\big) + n_{z}^{l}, \quad n_{z}^{l} \sim \mathcal{N}(0,\Sigma_{z}^{l}).$$6 for DCVC and $$z^{l} = f\!\big(U^{l+1} z^{l+1}\big) + n_{z}^{l}, \quad n_{z}^{l} \sim \mathcal{N}(0,\Sigma_{z}^{l}).$$7 for VCT; peak GPU memory $$z^{l} = f\!\big(U^{l+1} z^{l+1}\big) + n_{z}^{l}, \quad n_{z}^{l} \sim \mathcal{N}(0,\Sigma_{z}^{l}).$$8 versus $$z^{l} = f\!\big(U^{l+1} z^{l+1}\big) + n_{z}^{l}, \quad n_{z}^{l} \sim \mathcal{N}(0,\Sigma_{z}^{l}).$$9 and $$\varepsilon_{x}^{l} = x^{l} - f(U^{l} z^{l}), \quad \varepsilon_{z}^{l} = z^{l} - f(U^{l+1} z^{l+1}).$$0; encoding time $$\varepsilon_{x}^{l} = x^{l} - f(U^{l} z^{l}), \quad \varepsilon_{z}^{l} = z^{l} - f(U^{l+1} z^{l+1}).$$1 s versus $$\varepsilon_{x}^{l} = x^{l} - f(U^{l} z^{l}), \quad \varepsilon_{z}^{l} = z^{l} - f(U^{l+1} z^{l+1}).$$2 s and $$\varepsilon_{x}^{l} = x^{l} - f(U^{l} z^{l}), \quad \varepsilon_{z}^{l} = z^{l} - f(U^{l+1} z^{l+1}).$$3 s; and decoding time $$\varepsilon_{x}^{l} = x^{l} - f(U^{l} z^{l}), \quad \varepsilon_{z}^{l} = z^{l} - f(U^{l+1} z^{l+1}).$$4 s versus $$\varepsilon_{x}^{l} = x^{l} - f(U^{l} z^{l}), \quad \varepsilon_{z}^{l} = z^{l} - f(U^{l+1} z^{l+1}).$$5 s and $$\varepsilon_{x}^{l} = x^{l} - f(U^{l} z^{l}), \quad \varepsilon_{z}^{l} = z^{l} - f(U^{l+1} z^{l+1}).$$6 s, all on a single RTX 3090-24G. The same paper states that its solution is the first to enable progressive decoding (Lu et al., 2023).

Action-conditioned and temporally extended variants broaden HPC beyond passive perception. AFA-PredNet augments PredNet with an MLP that multiplicatively gates the generative state by an action vector,

$$\varepsilon_{x}^{l} = x^{l} - f(U^{l} z^{l}), \quad \varepsilon_{z}^{l} = z^{l} - f(U^{l+1} z^{l+1}).$$7

thereby making top-down predictions contingent on motor commands (Zhong et al., 2018). MTA-PredNet adds explicit multiple time scales, using

$$\varepsilon_{x}^{l} = x^{l} - f(U^{l} z^{l}), \quad \varepsilon_{z}^{l} = z^{l} - f(U^{l+1} z^{l+1}).$$8

with reported temporal parameters $$\varepsilon_{x}^{l} = x^{l} - f(U^{l} z^{l}), \quad \varepsilon_{z}^{l} = z^{l} - f(U^{l+1} z^{l+1}).$$9, $l$0, and $l$1, so higher levels evolve more slowly than lower levels (Zhong et al., 2018). "Differentiable Generalised Predictive Coding" further adds generalised coordinates of motion and automatic differentiation, jointly optimizing hierarchical and dynamical predictions on sequential data and showing that learning sampling distances in parallel can address meaningful locations in discretely sampled sequences (Ofner et al., 2021).

Hierarchical predictive coding has also been used as a framework for behavior and planning. "Habitual and Reflective Control in Hierarchical Predictive Coding" presents HPC as a single layered architecture that can implement both fast habitual and slower reflective actions by engaging different depths of the hierarchy. In the MNIST-digit1 task, the paper reports action accuracy of $l$2 and label accuracy of $l$3, supporting its claim that lower layers can support correct action selection even when higher-level labels have not settled (Kinghorn et al., 2021). "Active Predictive Coding" generalizes this into a hierarchical world-model framework with state networks, policy networks, and hypernetworks that generate lower-level parameters from higher-level embeddings. On vision tasks, the two-level APC model reports per-pixel reconstruction MSEs of $l$4 on MNIST, $l$5 on Fashion-MNIST, $l$6 on Omniglot test, and $l$7 on Omniglot transfer, while in planning it composes high-level options into lower-level action sequences through model-predictive control (Rao et al., 2022).

An unsupervised continual-learning variant appears in the Self-Taught Associative Memory architecture, where repeated modules combine online clustering with hierarchical predictive coding: higher levels send predicted centroids downward, lower levels return the mismatch between recalled and predicted prototypes, and cluster creation and merging implement novelty detection and graceful forgetting (Dovrolis, 2018).

6. Mathematical properties, limitations, and open questions

Formal analyses of HPC have moved beyond fixed-point existence and backpropagation correspondence toward macroscopic dynamical regimes. In the linear hierarchy studied in "Mathematical derivation of wave propagation properties in hierarchical neural networks with predictive coding feedback dynamics" (Faye et al., 2023), the scalar identity case yields the discrete update

$l$8

with amplification factor

$l$9

Near $\mu_l$0, the asymptotic wave speed is

$\mu_l$1

This gives an explicit direction criterion: forward propagation when $\mu_l$2, backward propagation when $\mu_l$3, and balanced regimes when the terms cancel. The same paper shows that transmission delays can produce oscillations at biologically plausible frequencies (Faye et al., 2023).

A nonlinear continuous-time hierarchy with predictive-coding feedback dynamics is analyzed in (Alamia et al., 14 May 2025). There, the scalar reduction is

$\mu_l$4

with $\mu_l$5. That work derives conditions for upward propagation, downward propagation, and propagation failure, and numerically demonstrates threshold behavior for constant inputs and finite flashes. It explicitly interprets low-$\mu_l$6 regimes as sensory-dominated, high-$\mu_l$7 regimes as prior-dominated, and intermediate regimes as balanced bidirectional propagation (Alamia et al., 14 May 2025).

Despite the breadth of formulations, several limitations recur. Precision propagation remains under-explored: (Ofner et al., 2021) states that the exact roles of bottom-up versus top-down precision propagation are still an open question. The theoretical analysis of PCNs in (Millidge et al., 2022) still depends on assumptions such as differentiability, sufficiently small learning rates, appropriate initialization, and stability of inference dynamics. The spiking account in (Mikulasch et al., 2022) notes that a complete multi-level solution to the weight-transport problem remains outstanding and that nonlinear dendritic computations and clustered-synapse learning require further development. In engineering settings, empirical scope is often narrow: (Ofner et al., 2021) emphasizes that broader tests on larger datasets and deeper hierarchies would strengthen claims; DHVC reports a noticeable performance gap on HEVC C/D/E due to downsampling at low input resolutions (Lu et al., 2023); AFA-PredNet does not learn a policy and leaves action selection to future work (Zhong et al., 2018); and hybrid sparse predictive coding reports increased variance as depth grows (Fujita, 26 Jun 2026).

Taken together, these results define HPC less as a single algorithm than as a family of closely related models organized around the same invariants: hierarchical generative structure, top-down predictions, bottom-up prediction errors, local learning, and—where uncertainty is modeled explicitly—precision-weighted inference. Across cortical theory, sparse coding, video compression, active perception, planning, and dynamical-systems analysis, the central claim remains constant: hierarchical inference can be implemented through recursive prediction-error minimization, while the precise form of the generative map, the precision model, and the message-passing schedule determines whether HPC behaves like variational inference, approximate natural-gradient descent, a spiking dendritic circuit, or a deep engineered architecture (Jiang et al., 2021, Ofner et al., 2021, Mikulasch et al., 2022).