Predictive Coding Graphs: Theory & Applications

• Predictive Coding Graphs are graph-based models that augment each node with a state, prediction, and local error to minimize a global energy objective.
• They extend traditional predictive coding to arbitrary directed graphs with skip, lateral, and recurrent connections, enabling flexible inference and learning.
• They bridge concepts from backpropagation, causal inference, and graph neural networks, demonstrating robust performance in graph representation learning and adversarial settings.

Searching arXiv for papers on predictive coding graphs and closely related formulations. Predictive Coding Graphs (PCGs) are predictive-coding models defined on graph-structured computation or dependency architectures rather than on strictly layered hierarchies. In this usage, a PCG augments each node with a value or state variable, a prediction generated from parent nodes, and a local prediction error, and performs inference and learning by minimizing a global energy or variational free-energy objective through local message passing (Salvatori et al., 2022, Millidge et al., 2020, Zwol, 6 Mar 2026). Across recent formulations, PCGs are used to extend predictive coding to arbitrary directed graph topologies, including skip connections, lateral connections, multi-parent nodes, and recurrent or cyclic structures, while preserving the characteristic predictive-coding separation between an inference phase on activities and a learning phase on parameters (Salvatori et al., 2022, Byiringiro et al., 2022, Zwol, 6 Mar 2026). The term “PCG” is also used in graph theory for pairwise compatibility graphs; that is a distinct literature with unrelated definitions and witness-tree criteria (Baiocchi et al., 2017, Calamoneri et al., 2022).

1. Definition and formalism

In the arbitrary-topology predictive-coding formulation, a PCG is a directed graph $G=(V,E)$ whose vertices carry state variables and prediction errors, and whose edges encode parametric dependencies used to generate local predictions (Salvatori et al., 2022, Millidge et al., 2020). A representative definition assigns to each vertex $i$ a value node $x_{i,t}$, a local prediction $\mu_{i,t}$, and a local prediction error $\epsilon_{i,t}=x_{i,t}-\mu_{i,t}$, with predictions computed from incoming neighbors through weighted nonlinear transformations (Salvatori et al., 2022). One explicit form is

$\mu_{i,t}=\sum_j \theta_{j,i} f(x_{j,t}),$

with energy

$\mathcal{E}_t=\frac12\sum_i (\epsilon_{i,t})^2,$

and local weight updates

$$\Delta\theta_{i,j}=\alpha\,\epsilon_{i,T} f(x_{j,T})$$

after inference has converged (Salvatori et al., 2022).

A closely related computational-graph formulation defines each node $\alpha$ by an activity $a_\alpha\in\mathbb{R}$, a prediction

$i$0

and a prediction error $i$1, yielding the energy

$i$2

under unit precisions (Zwol, 6 Mar 2026). This formulation explicitly positions PCGs as a generalization of predictive coding networks beyond layered hierarchies and proves that, with an appropriate hierarchical block mask, the PCG energy reduces to the standard hierarchical predictive-coding energy up to a constant (Zwol, 6 Mar 2026).

A third, variational formulation starts from a differentiable computation graph and augments every intermediate variable $i$3 with a prediction $i$4 and an error node $i$5, with a quadratic free energy

$i$6

under identity precisions (Millidge et al., 2020). In that setting, predictive coding acts on the augmented graph $i$7 and recovers reverse-mode differentiation through local error dynamics at convergence (Millidge et al., 2020).

These definitions share a common structure. First, predictions are generated along the directed edges of the graph. Second, discrepancies between current node states and their local predictions define nodewise errors. Third, the global objective is a sum of squared local errors or a precision-weighted variant of that sum. This suggests that “Predictive Coding Graph” is best understood not as a single architecture but as a family of graph-based predictive-coding instantiations linked by the same local error-minimization principle.

2. Inference dynamics and local learning

The central computational mechanism of a PCG is iterative inference on node activities while parameters are held fixed, followed by local parameter updates derived from the same energy (Byiringiro et al., 2022, Salvatori et al., 2022). In the arbitrary-graph formulation, state inference follows gradient descent on $i$8:

$i$9

where the first term reduces the local error at node $x_{i,t}$0, and the second propagates downstream error information through outgoing synapses (Salvatori et al., 2022). The corresponding local parameter update is

$x_{i,t}$1

a postsynaptic-error times presynaptic-activity rule (Salvatori et al., 2022).

In graph-representation-learning formulations, the same inference-before-learning schedule is emphasized more explicitly. Each node carries a value unit $x_{i,t}$2 and a prediction-error unit $x_{i,t}$3, with node predictions

$x_{i,t}$4

and global energy

$x_{i,t}$5

Activity updates then take the form

$x_{i,t}$6

for $x_{i,t}$7, followed by the local weight update

$x_{i,t}$8

once inference has stabilized (Byiringiro et al., 2022).

In differentiable generalized predictive coding, the objective is extended to precision-weighted hierarchical and dynamical errors. For latent states $x_{i,t}$9 across layers,

$\mu_{i,t}$0

and, in generalized-coordinate notation,

$\mu_{i,t}$1

with precision-weighted errors $\mu_{i,t}$2 and $\mu_{i,t}$3 (Ofner et al., 2021). State and parameter updates are obtained by automatic differentiation of the total precision-weighted objective (Ofner et al., 2021).

A recurrent theme across these variants is that PCGs replace a one-shot forward computation plus a separate nonlocal backward pass by iterative local relaxation. The factual commonality is the inference-before-learning schedule; a plausible implication is that many empirical and theoretical differences among PCG variants arise more from graph topology, precision modeling, and inference scheduling than from the basic local learning rule itself.

3. Relation to feedforward networks, backpropagation, and automatic differentiation

A major development in the PCG literature is the formal connection to reverse-mode automatic differentiation and standard feedforward neural networks (Millidge et al., 2020, Salvatori et al., 2021, Zwol, 6 Mar 2026). In the computation-graph perspective, predictive coding is defined directly on arbitrary differentiable DAGs by augmenting each variable with a prediction and an error node. With the free-energy objective

$\mu_{i,t}$4

the state dynamics are

$\mu_{i,t}$5

and the parameter dynamics are

$\mu_{i,t}$6

At fixed points, the error recursion coincides with the backpropagation recursion on the original computation graph (Millidge et al., 2020).

The stronger exactness result is given by the graph-theoretic generalization of Zero-Divergence Inference Learning. There, a differentiable scalar mapping is represented as a DAG, transformed into a levelled DAG by inserting identity nodes so that all directed paths between nodes have equal length, and then subjected to a level-synchronous predictive-coding schedule with $\mu_{i,t}$7 (Salvatori et al., 2021). Under zero-divergence initialization, output clamping, and level-synchronous parameter updates, the paper shows that the PCG update equals the reverse-mode gradient on arbitrary architectures representable as DAGs, including residual and transformer-style graphs (Salvatori et al., 2021). The relevant local activity update is

$\mu_{i,t}$8

with local parameter update

$\mu_{i,t}$9

and the theorem states that, under the stated construction, $\epsilon_{i,t}=x_{i,t}-\mu_{i,t}$0 (Salvatori et al., 2021).

The 2026 superset result shifts the emphasis from gradient equivalence to representational containment. A hierarchical predictive-coding network is equivalent to a feedforward neural network during testing because minimizing the hierarchical PCN energy enforces $\epsilon_{i,t}=x_{i,t}-\mu_{i,t}$1 layer by layer, yielding the standard feedforward equations $\epsilon_{i,t}=x_{i,t}-\mu_{i,t}$2 (Zwol, 6 Mar 2026). The same work proves that if the general PCG weight matrix is masked so that $\epsilon_{i,t}=x_{i,t}-\mu_{i,t}$3 unless $\epsilon_{i,t}=x_{i,t}-\mu_{i,t}$4, then the PCG energy equals the hierarchical PCN energy up to a constant, and the activity and learning argmins coincide (Zwol, 6 Mar 2026). This establishes PCGs as a mathematical superset of predictive coding networks and, during testing, of feedforward neural networks (Zwol, 6 Mar 2026).

This line of work clarifies a common misconception. PCGs are not merely heuristic or approximate graph analogues of neural networks. Under specific assumptions, they either asymptotically recover exact backpropagation on arbitrary computation graphs (Millidge et al., 2020), exactly implement reverse differentiation on levelled DAGs (Salvatori et al., 2021), or strictly contain the standard feedforward architecture class through an energy-based generalization (Zwol, 6 Mar 2026).

4. Generalized, dynamical, and modular PCGs

Beyond static graph inference, PCGs have been extended to generalized predictive coding with temporal dynamics, generalized coordinates, and flexible graph structure (Ofner et al., 2021). In this setting, the latent state at each layer is expanded into derivative orders,

$\epsilon_{i,t}=x_{i,t}-\mu_{i,t}$5

with shift operator $\epsilon_{i,t}=x_{i,t}-\mu_{i,t}$6 (Ofner et al., 2021). The total objective becomes a sum across layers and derivative orders,

$\epsilon_{i,t}=x_{i,t}-\mu_{i,t}$7

and the discrete-time dynamical pathway is written as

$\epsilon_{i,t}=x_{i,t}-\mu_{i,t}$8

with hierarchical, dynamical, and derivative errors

$\epsilon_{i,t}=x_{i,t}-\mu_{i,t}$9

combined as

$\mu_{i,t}=\sum_j \theta_{j,i} f(x_{j,t}),$0

These formulations allow hierarchical content predictions and temporal consistency constraints to act jointly on the same latent variables (Ofner et al., 2021).

The graph structure in generalized predictive coding is not restricted to strict hierarchies. The model explicitly allows directed acyclic graphs and directed cyclic graphs, with node types including observation nodes, latent nodes, parameter nodes, and precision nodes, and edge types including generative edges, temporal edges, and error/precision edges (Ofner et al., 2021). Stability constraints are correspondingly technical: precision matrices should be positive-definite, and damping, bounded $\mu_{i,t}=\sum_j \theta_{j,i} f(x_{j,t}),$1, and gradient clipping are used for recurrent or cyclic schedules (Ofner et al., 2021).

The same paper places PCGs in a modular language using canonical microcircuit metaphors and Markov blankets. In that account, generative top-down predictions correspond to $\mu_{i,t}=\sum_j \theta_{j,i} f(x_{j,t}),$2 edges, feedforward error signals correspond to $\mu_{i,t}=\sum_j \theta_{j,i} f(x_{j,t}),$3 nodes propagating upward, and blanket states define module interfaces such that internal states are conditionally independent of external states given the blanket (Ofner et al., 2021). This supports the interpretation of PCGs as compositional systems built from subgraphs with well-defined message-passing boundaries.

This suggests a broader reading of PCGs than the simpler “predictive coding on arbitrary graphs” slogan. A plausible implication is that the graph formalism is not only about arbitrary connectivity; it also provides a language for integrating hierarchy, dynamics, modularity, and time-scale adaptation within one predictive-coding objective.

5. Applications in machine learning and graph representation learning

A prominent application area is graph representation learning, where predictive coding is used as the message-passing rule inside graph neural architectures (Byiringiro et al., 2022). In this literature, Graph Predictive Coding Networks (GPCNs) instantiate PCGs with hierarchical layers mirroring popular GNNs such as GCN and GAT, while replacing the usual feedforward pass with iterative predictive-coding inference (Byiringiro et al., 2022). At each node, the model computes a prediction from neighbors, forms an error, relaxes activities by gradient descent on the energy, and only then updates weights locally (Byiringiro et al., 2022).

The reported empirical results show that these models are comparable to standard GNNs in raw predictive performance while improving calibration and robustness (Byiringiro et al., 2022). On transductive node classification, the paper reports: Cora, GCN $\mu_{i,t}=\sum_j \theta_{j,i} f(x_{j,t}),$4 and GPCN $\mu_{i,t}=\sum_j \theta_{j,i} f(x_{j,t}),$5; Citeseer, GCN $\mu_{i,t}=\sum_j \theta_{j,i} f(x_{j,t}),$6 and GPCN $\mu_{i,t}=\sum_j \theta_{j,i} f(x_{j,t}),$7; PubMed, GCN $\mu_{i,t}=\sum_j \theta_{j,i} f(x_{j,t}),$8 and GPCN $\mu_{i,t}=\sum_j \theta_{j,i} f(x_{j,t}),$9 (Byiringiro et al., 2022). On inductive F1, the paper reports: Cora, GCN $\mathcal{E}_t=\frac12\sum_i (\epsilon_{i,t})^2,$0 and GPCN $\mathcal{E}_t=\frac12\sum_i (\epsilon_{i,t})^2,$1; Citeseer, GCN $\mathcal{E}_t=\frac12\sum_i (\epsilon_{i,t})^2,$2 and GPCN $\mathcal{E}_t=\frac12\sum_i (\epsilon_{i,t})^2,$3; PubMed, GCN $\mathcal{E}_t=\frac12\sum_i (\epsilon_{i,t})^2,$4 and GPCN $\mathcal{E}_t=\frac12\sum_i (\epsilon_{i,t})^2,$5; PPI supervised, GCN $\mathcal{E}_t=\frac12\sum_i (\epsilon_{i,t})^2,$6 and GPCN $\mathcal{E}_t=\frac12\sum_i (\epsilon_{i,t})^2,$7; PPI unsupervised, GCN $\mathcal{E}_t=\frac12\sum_i (\epsilon_{i,t})^2,$8 and GPCN $\mathcal{E}_t=\frac12\sum_i (\epsilon_{i,t})^2,$9 (Byiringiro et al., 2022).

The same study reports lower calibration error and stronger resistance to adversarial attacks. For Nettack targeted evasion, the robustness metric $$\Delta\theta_{i,j}=\alpha\,\epsilon_{i,T} f(x_{j,T})$$0 is reported as $$\Delta\theta_{i,j}=\alpha\,\epsilon_{i,T} f(x_{j,T})$$1 for GPCN on Cora, compared to $$\Delta\theta_{i,j}=\alpha\,\epsilon_{i,T} f(x_{j,T})$$2 for GCN and $$\Delta\theta_{i,j}=\alpha\,\epsilon_{i,T} f(x_{j,T})$$3 for RGCN; $$\Delta\theta_{i,j}=\alpha\,\epsilon_{i,T} f(x_{j,T})$$4 for GPCN on Citeseer, compared to $$\Delta\theta_{i,j}=\alpha\,\epsilon_{i,T} f(x_{j,T})$$5 for GCN and $$\Delta\theta_{i,j}=\alpha\,\epsilon_{i,T} f(x_{j,T})$$6 for RGCN; and $$\Delta\theta_{i,j}=\alpha\,\epsilon_{i,T} f(x_{j,T})$$7 for GPCN on PubMed, compared to $$\Delta\theta_{i,j}=\alpha\,\epsilon_{i,T} f(x_{j,T})$$8 for GCN and $$\Delta\theta_{i,j}=\alpha\,\epsilon_{i,T} f(x_{j,T})$$9 for RGCN (Byiringiro et al., 2022). On global poisoning with Metattack at $\alpha$0, the reported results include Cora: GCN $\alpha$1, GPCN-GCN $\alpha$2, GAT $\alpha$3, GPCN-GAT $\alpha$4, RGCN $\alpha$5; Citeseer: GCN $\alpha$6, GPCN-GCN $\alpha$7, GAT $\alpha$8, GPCN-GAT $\alpha$9, RGCN $a_\alpha\in\mathbb{R}$0; PubMed: GCN $a_\alpha\in\mathbb{R}$1, GPCN-GCN $a_\alpha\in\mathbb{R}$2, GAT $a_\alpha\in\mathbb{R}$3, GPCN-GAT $a_\alpha\in\mathbb{R}$4, RGCN $a_\alpha\in\mathbb{R}$5 (Byiringiro et al., 2022).

Outside GNNs, arbitrary-topology PCGs have been applied to generation, reconstruction, denoising, classification, and associative memory within the same network by clamping or initializing different subsets of nodes (Salvatori et al., 2022). On fully connected PCGs for classification, the paper reports MNIST $a_\alpha\in\mathbb{R}$6, FashionMNIST $a_\alpha\in\mathbb{R}$7, and SVHN $a_\alpha\in\mathbb{R}$8 (Salvatori et al., 2022). For denoising on MNIST, Fréchet Inception Distance values are reported as $a_\alpha\in\mathbb{R}$9 for PCG at noise variance $i$00, compared to $i$01 for an autoencoder (Salvatori et al., 2022).

In generalized predictive coding, applications include perception and planning on sequential data, learning dynamical predictions, and learning sampling distances in parallel (Ofner et al., 2021). The paper states that a stride of $i$02 in the modulated sine-wave experiment yields a nearly stationary input at the lowest layer with an almost constant second derivative, whereas other strides such as $i$03 or $i$04 incur higher prediction error (Ofner et al., 2021). That claim is narrowly tied to the reported experiment; a plausible implication is that adaptive time-step selection can function as a structural degree of freedom in temporal PCGs.

6. Causal inference, interventions, and structure learning

PCGs have also been cast as structural causal models over arbitrary directed graphs (Salvatori et al., 2023). In that formulation, each node $i$05 has parents $i$06, predictive mean

$i$07

and Gaussian conditional

$i$08

with predictive-coding energy

$i$09

and local weight update

$i$10

after inference (Salvatori et al., 2023).

The key causal result is an intervention rule that does not require explicit graph mutilation. To compute the effect of $i$11, the procedure clamps the intervened node at all times, $i$12, and sets its local error to zero for all times, $i$13 (Salvatori et al., 2023). The paper states the resulting interventional expectation as

$i$14

arguing that zeroing the error prevents upstream correction through the intervened node and thus matches Pearl’s truncated-factorization semantics (Salvatori et al., 2023).

When the graph is unknown, the adjacency matrix can itself be learned by treating it as a continuous parameter in the predictive mean,

$i$15

and updating adjacency entries by gradient descent (Salvatori et al., 2023). To enforce sparsity and acyclicity, the paper adds priors including

$i$16

and the differentiable acyclicity term

$i$17

which is zero iff the graph is acyclic (Salvatori et al., 2023).

The empirical evaluation covers both causal inference and classification. On causal graphs such as collider, confounder, mediator, chain, fork, M-bias, and butterfly-bias motifs, the paper reports that PCGs achieve low errors across associational, interventional, and counterfactual queries and outperform MultiCVAE, CAREFL, and VACA on most metrics while using a very small number of parameters (Salvatori et al., 2023). One concrete example given is the butterfly graph, where PCGs report interventional MMD $i$18, MeanE $i$19, StdE $i$20, counterfactual MSE $i$21, and SSE $i$22, with only $i$23 parameters (Salvatori et al., 2023). On MNIST and FashionMNIST, the paper reports consistent improvements of approximately $i$24 test accuracy when test-time inference uses interventional queries that clamp inputs and set their errors to zero (Salvatori et al., 2023).

These results address another misconception. PCGs are sometimes described only as biologically motivated substitutes for backpropagation. The causal-modeling literature shows that the same local inference machinery can be used to answer associational, interventional, and counterfactual queries and to perform causal discovery on random graphs (Salvatori et al., 2023).

7. Scope, ambiguities, and limitations

The predictive-coding PCG literature is technically coherent but terminologically ambiguous. In graph theory, “PCG” usually denotes “Pairwise Compatibility Graph,” a tree-metric graph class defined by leaf-to-leaf distances lying inside an interval $i$25 (Baiocchi et al., 2017). In that literature, a graph $i$26 is a PCG if there exists an edge-weighted tree $i$27 and thresholds such that

$i$28

and recent work studies non-PCG graph classes, star-$i$29-PCGs, and $i$30-interval-PCGs (Baiocchi et al., 2017, Monti et al., 2022, Calamoneri et al., 2022). This meaning is entirely different from predictive coding graphs. Because both literatures use the same acronym, precise contextual qualification is necessary.

Within the predictive-coding sense, the main limitations are computational and dynamical rather than definitional. Iterative inference creates overhead relative to one-pass feedforward evaluation. In graph representation learning, PCG inference adds $i$31 inference steps per batch, with cost roughly $i$32 plus error-computation overhead, and the paper notes trade-offs between lower energy, calibration gains, and runtime (Byiringiro et al., 2022). In arbitrary-topology PCGs, per-sample training cost is $i$33 for inference plus $i$34 for one weight update, with inference iterations ranging from a handful in associative-memory settings to $i$35 in some generation and reconstruction experiments (Salvatori et al., 2022). The 2026 superset paper states that dense PCG testing is $i$36, and for a hierarchical PCG with comparable parameter count, testing can be a factor approximately $i$37 slower than one-pass FNN inference (Zwol, 6 Mar 2026).

Convergence and stability are recurrent concerns. The arbitrary-graph PCG paper reports robust empirical convergence provided the inference step size is not too large, weight decay is applied, and sufficient inference iterations are used (Salvatori et al., 2022). The generalized predictive-coding paper emphasizes positive-definite precisions, damping, bounded $i$38, gradient clipping, and truncated time horizons for BPTT through temporal edges (Ofner et al., 2021). The graph-representation-learning paper notes sensitivity to $i$39, $i$40, and learning rate, and advises damping for stability (Byiringiro et al., 2022). The causal PCG paper notes sensitivity to the weighting of sparsity and DAG priors, adjacency thresholds, and step sizes for state, weight, and adjacency updates (Salvatori et al., 2023).

Theoretical exactness also depends on specific assumptions. Asymptotic equivalence to backpropagation along arbitrary computation graphs in the 2020 work requires convergence to a fixed point and a fixed-prediction assumption during inference (Millidge et al., 2020). Exact reverse differentiation in the 2021 work requires a DAG representation, leveling by identity nodes, zero-divergence initialization, $i$41, and level-synchronous parameter updates (Salvatori et al., 2021). The superset result in 2026 establishes containment and test-time equivalence under a hierarchical block mask, but does not claim that arbitrary non-hierarchical PCGs inherit all feedforward-network properties under arbitrary training protocols (Zwol, 6 Mar 2026).

Taken together, these limitations do not weaken the core definition of PCGs. Rather, they mark the current frontier of the field: making arbitrary-topology predictive coding computationally efficient, stable under richer cyclic dynamics, scalable to larger graphs, and theoretically characterized beyond the special cases where exact equivalence to existing learning algorithms is already known (Ofner et al., 2021, Byiringiro et al., 2022, Zwol, 6 Mar 2026).