---
title: Probabilistic Program Graphs (PPGs)
url: https://www.emergentmind.com/topics/probabilistic-program-graphs-ppgs
type: topic
---

# Probabilistic Program Graphs (PPGs)

Probabilistic Program Graphs (PPGs) are graph-based representations of probabilistic computation in which nodes encode program points, checkpoints, sample sites, graph-interface terms, or dependence locations, and edges encode control flow, stochastic transitions, data/control dependence, or explicit reactive dependencies. The term is not used uniformly across the literature. In one line of work, PPGs are probabilistic control-flow graphs for imperative programs with random assignment and `observe` statements [1711.02256]. In another, they are automata-theoretic checkpoint graphs for universal probabilistic programs with arbitrary measures, score functions, and unbounded loops [2603.22463]. Other usages include static dependency graphs over user-labelled sample sites [2508.20922], dynamic reactive graphs for incremental MCMC [2606.30137], equationally specified interfaces for random graph models [2312.17127], and probabilistic program dependence graphs for fault localization [1201.3985]. The common denominator is a graph-structured semantic or analytic abstraction that exposes probabilistic dependencies more explicitly than the source program alone.

## 1. Terminological scope and main variants

The label “PPG” denotes several non-identical formalisms. The following usage classes are explicitly documented.

| Variant | Core graph objects | Primary role |
|---|---|---|
| Probabilistic control-flow graph | Program points, control-flow edges, assignments, `observe`, branches | Operational semantics for probabilistic imperative programs |
| Automata-theoretic PPG | Checkpoints, guarded transitions, parametric measures, node scores | Infinite-trace semantics, Feynman–Kac framing, SMC |
| Static sample-site PPG | Sample statements and conservative dependency edges | Static factorization and slicing |
| Reactive PPG | DAG nodes for `assume`, deterministic computation, `weight`, subgraphs | Incremental MCMC recomputation |
| Graph-interface PPG | Typed terms over `new` and `edge` | Random graph semantics via graphons |
| Probabilistic program dependence graph | PDG nodes with CPTs | Fault localization |

In the 2017 semantics paper, probabilistic program graphs are control-flow graphs for probabilistic imperative programs, with node labels such as deterministic assignment, random assignment, `Observe B`, `Skip`, and branching conditions [1711.02256]. In the 2026 universal-probabilistic-programming formulation, a PPG on $\bar{\mathbb{R}}^m$ is a tuple $G=(P,E,\mathrm{nil},\Gamma)$, where transitions carry predicates and parametric measures, and checkpoints carry score functions [2603.22463]. In the static-factorisation setting, the graph is built over user-labelled sample statements and conservative dependencies computed from a CFG [2508.20922]. In the reactive-graph setting, a single run yields a finite DAG whose nodes represent random variables, deterministic computations, factor nodes, and subgraphs [2606.30137].

This multiplicity is substantive rather than merely terminological. Some PPGs are semantic objects for entire programs, some are per-trace dynamic structures, and some are static approximations used to derive factorizations or support inference transformations. A plausible implication is that “Probabilistic Program Graph” functions as an umbrella term for graph-based intermediate representations of probabilistic programs rather than a single standardized formalism.

## 2. Control-flow semantics and adequacy for structured programs

In the control-flow interpretation, a pCFG is a directed graph with nodes $V$, special `Start` and `End` nodes, and labels for deterministic assignment $x := E$, random assignment $x \sim \psi$, `Observe B`, `Skip`, and branching on a boolean $B$ [1711.02256]. Stores are functions $s:\mathrm{Var}\to\mathbb{Z}$ over a finite variable set, and distributions are sub-probability distributions $D:F\to[0,\infty)$ with total mass at most $1$.

The semantics is given as a distribution transformer. For deterministic assignment,
\[
\llbracket x := e \rrbracket(D)(s') \;=\; \sum_{s \in F:\, s' = s[x \mapsto [e]s]} D(s).
\]
For state-independent random assignment,
\[
\llbracket x \sim \psi \rrbracket(D)(s') \;=\; \psi(s'(x)) \cdot \sum_{s \in F:\, s' = s[x \mapsto s'(x)]} D(s).
\]
For `observe`,
\[
\llbracket \operatorname{observe}\; b \rrbracket(D)(s) \;=\; \begin{cases}
D(s) & \text{if } [b]s,\\
0    & \text{otherwise.}
\end{cases}
\]
Selection, deterministic assignment, and random assignment are continuous, additive, multiplicative, and non-increasing; the selection operator additionally satisfies $\mathrm{Sel}_B(D)+\mathrm{Sel}_{\neg B}(D)=D$ [1711.02256].

Loops are handled through postdomination and a least fixed point over modification functions. If $v'$ postdominates $v$, and $\mathrm{1PPD}(v)$ is the first proper postdominator, semantic equations are defined by a continuous functional $H$ whose least fixed point $\omega=\mathrm{fix}(H)=\lim_{k\to\infty}H^k(0)$ gives the graph semantics. Intuitively, $\omega_k$ allows at most $k-1$ back-edges [1711.02256]. For a pCFG $G$ with `Start` node $v$ and `End` node $v'$, the graph transformer is
\[
T_G := \omega\{v\}\{v'\}.
\]

The same paper relates this operational semantics to a standard expectation-transformer semantics for structured statements `skip`, assignment, random assignment, `observe`, sequencing, conditionals, and `while` [1711.02256]. Program-level normalization for $P \equiv S;\mathrm{return}\;E$ is defined by
\[
\llbracket S; \text{return } E \rrbracket \;=\;
\frac{\llbracket S \rrbracket(\lambda s.\,[E]s)(\bot)}{\llbracket S \rrbracket(\lambda s.\,1)(\bot)}.
\]
The adequacy theorem states that if $D'=\omega\{v\}\{v'\}(D)$ and $F=\llbracket S \rrbracket(F')$, then
\[
\sum_{s \in F} F(s)\, D(s) \;=\; \sum_{s \in F} F'(s)\, D'(s),
\]
establishing that the pCFG semantics preserves denotationally computed expectations without loss of information [1711.02256].

A worked example in the same development uses $\psi_4$ uniform on $\{0,1,2,3\}$ and the program
\[
S_1 \equiv x := \psi_4;\; y := \psi_4;\; \mathrm{observe}(x+y\ge 5),\qquad
P_1 \equiv S_1;\mathrm{return}\;x.
\]
After the two random assignments, the joint distribution on $(x,y)$ is uniform on $\{0,\ldots,3\}\times\{0,\ldots,3\}$; after `observe`, only $(2,3)$, $(3,2)$, and $(3,3)$ remain, each with mass $1/16$, so the total surviving mass is $3/16$ and the normalized return value is $8/3$ [1711.02256].

## 3. Infinite-trace weighted semantics and the Feynman–Kac interpretation

The automata-theoretic formulation generalizes PPGs from discrete imperative pCFGs to universal probabilistic programs with arbitrary measures, score functions, and unbounded loops [2603.22463]. For fixed $m\ge 1$, a PPG on $\bar{\mathbb{R}}^m$ is a tuple $G=(P,E,\mathrm{nil},\Gamma)$ in which $P$ is a finite set of checkpoints, $E$ is a finite set of transitions $(S,\varphi,\zeta,S')$ with guard $\varphi:\bar{\mathbb{R}}^m\to\{0,1\}$ and parametric measure $\zeta$, $\mathrm{nil}$ is the distinguished terminated checkpoint with only outgoing transition $(\mathrm{nil},1,\mathrm{id},\mathrm{nil})$, and $\Gamma$ assigns a score function to each checkpoint [2603.22463]. Outgoing predicates at each checkpoint sum to $1$.

The operational kernel on global states $\Omega=\bar{\mathbb{R}}^{m+1}$ is
\[
(K(\omega))(A):=
\begin{cases}
\delta_\omega(A) & \text{if }\omega\notin \bar{\mathbb{R}}^m\times P,\\[3pt]
\sum_{(S,\varphi,\zeta,S')\in E_S} \varphi(v)\cdot \zeta(v)(A_{S'}) & \text{if }\omega=(v,S)\in \bar{\mathbb{R}}^m\times P,
\end{cases}
\]
where $A_{S'}=\{v\in\bar{\mathbb{R}}^m:(v,S')\in A\}$ [2603.22463]. This induces finite-trace measures $\mu_S^t=\delta_{(0,S)}\otimes K\otimes\cdots\otimes K$ and, by Ionescu–Tulcea, an infinite-trace law $\mu_S^\infty$ on the cylindrical $\sigma$-field of $\Omega^\infty$ [2603.22463].

Weights are checkpoint-local. Defining
\[
sc(v,S):=\Gamma(S)(v),
\qquad
w(\tilde\omega):=\prod_{j\ge 1} sc(\omega_j),
\qquad
w_t(\omega^t):=\prod_{j=1}^t sc(\omega_j),
\]
the semantics of a nonnegative measurable $f:\Omega^\infty\to\bar{\mathbb{R}}^+$ is
\[
[S]f := \mathbb{E}_{\mu_S^\infty}[f],
\qquad
[[S]]f := \frac{[S](f\cdot w)}{[S]w},
\]
provided $[S]w>0$ [2603.22463]. Here conditioning and soft evidence are handled by multiplicative checkpoint scores rather than statement-level pruning.

The finite approximation theorem introduces prefix-closed functionals and proves explicit lower and upper bounds in terms of fixed-length traces. If $f\le M$ and $[S]^t(1_{T^{\le t}}\cdot w_t)>0$, with
\[
\alpha_t:= \frac{[S]^t w_t}{[S]^t(1_{T^{\leq t}}\cdot w_t)} \ge 1,
\]
then
\[
\frac{[S]^t( f_t\cdot 1_{L^{\leq t}\cap T^{\leq t}}\cdot w_t )}{[S]^t w_t}
\;\le\; [[S]] f
\;\le\;
\frac{[S]^t( f_t\cdot 1_{L^{\leq t}\cap T^{\leq t}}\cdot w_t )}{[S]^t w_t}\,\alpha_t
+ M(\alpha_t-1).
\]
If termination is almost sure, both bounds converge to $[[S]]f$ as $t\to\infty$ [2603.22463].

The same work frames the semantics as a Feynman–Kac model with state space $X=\Omega$, initial law $\mu^1=\delta_{(0,S)}$, transition kernels $K_i=K$, and potentials $G_i=sc$ [2603.22463]. The global potential is then $w_t$, and the normalized path measure is
\[
\Phi(A) := \frac{\mathbb{E}_{\mu^t}[1_A\cdot G]}{\mathbb{E}_{\mu^t}[G]}.
\]
This reframing yields a direct consistency path for bootstrap particle filtering. Under bounded potentials and multinomial resampling,
\[
\hat{\phi}_t^N(h)=\sum_{j=1}^N \widehat{W}_t^{(j)} h(X_t^{(j)})
\;\xrightarrow{a.s.}\; \mathbb{E}_{\phi_t}[h]
\quad\text{as }N\to\infty
\]
for measurable nonnegative $h$ [2603.22463].

The vectorized Particle Filtering algorithm (VPF) stores particle values, node indices, and weights in arrays, computes masks $M_{s,\varphi}:=\varphi(V)\cdot(Z=s)$ per transition, and advances all particles in lock-step through the PPG [2603.22463]. The experiments summarized in the paper report accuracy comparable to state-of-the-art systems and significant speedups, especially at $N=10^6$, with runtime largely insensitive to conditioning frequency [2603.22463].

## 4. Static factorisation, dependency approximation, and slicing

A distinct PPG line treats the graph as a static approximation of dependency structure in programs with user-labelled sample statements and `while` loops [2508.20922]. A program state is a finite map $\sigma$ from variables to values, including a distinguished variable $p$ that accumulates density. Random variables are introduced only at sample statements
\[
x = \mathrm{sample}(E_0,f(E_1,\ldots,E_n)),
\]
where $E_0$ is a user-provided address and the trace $tr:\mathrm{Strings}\to V$ maps addresses to values [2508.20922]. Observations are realized by fixing $tr(\alpha)$ at addresses, so likelihood contributions are attached to the corresponding sample sites rather than expressed with a separate `observe` syntax.

The big-step sample rule is
\[
\frac{
\forall i \colon (E_i)=V_i\neq \text{null}\quad V_0\in \text{Strings}\quad V=tr(V_0)\neq \text{null}
}{
(\sigma,\,x = \text{sample}(E_0,f(E_1,\dots,E_n)))\Rightarrow [x\mapsto V,\,p\mapsto (\sigma(p))\cdot \text{pdf}_f(V;V_1,\dots,V_n)]
}.
\]
Programs are translated to CFGs with node types `START`, `END`, `Assign`, `Branch(E)`, and `Join`; CFG semantics are proved equivalent to operational semantics [2508.20922].

The PPG is then constructed by a static provenance analysis. Each source-level sample statement becomes a graph node. For a sample node $N_k$, the conservative dependency set is
\[
A_k = \mathrm{addresses}(N_k)\ \cup\ \bigcup_{i=0}^{n_k} \mathrm{prov}(N_k,E_i^k)\ \cup\ \bigcup_{N'\in BP(N_k)}\mathrm{prov}(N',\mathrm{condexp}(N')).
\]
Here $\mathrm{prov}(N,x)$ over-approximates addresses needed to determine the value of variable $x$ before executing $N$, using reaching definitions $RD(N,x)$ and branch parents $BP(N)$ [2508.20922]. Directed edges of the PPG go from $A_k\setminus \mathrm{addresses}(N_k)$ to $N_k$.

For loop-free programs with unique constant labels, this recovers the standard Bayesian-network factorization
\[
p(\mathbf{x})=\prod_i p(x_i\mid \mathrm{Pa}(x_i)).
\]
More generally, the program density factors as
\[
p(tr)=\prod_{k=1}^K p_k(tr),
\]
one factor per sample site, even when loops or dynamic labels induce an unbounded family of runtime random variables [2508.20922]. In the loop-free unique-address case, Theorem 2 gives
\[
p_k(tr)=b_k(tr)\,\text{pdf}_{f^k}\big(tr(\alpha_k);\ \hat E_1^k(tr),\dots,\hat E_{n_k}^k(tr)\big)\ +\ (1-b_k(tr))\,\mathbf{1}_{\text{null}(tr(\alpha_k))},
\]
where $b_k(tr)$ gates the factor by branch conditions [2508.20922]. With dynamic labels or non-unique labels, the factorization corresponds instead to a Markov network whose clique potentials are the per-site factors [2508.20922].

The paper further defines a slicing technique driven by the PPG. For a site of interest $N_k$, the slice retains only nodes on paths from $N_k$ to dependent sites and converts unrelated intervening sample nodes into read statements [2508.20922]. The key correctness theorem states that, for traces differing only at address $\alpha$, the full-program log-density difference equals the sliced-program log-density difference under appropriate matching conditions [2508.20922]. This locality enables three optimizations.

For variational inference, a localized score estimator is proved:
\[
\nabla_{\phi_\alpha}\mathbb{E}_{tr\sim Q_\phi}\big[\log p(tr)-\log Q_\phi(tr)\big]
=
\mathbb{E}_{tr\sim Q_\phi}\big[
\delta_{tr(\alpha)\ne\text{null}}\,
\nabla_{\phi_\alpha}\log q_\alpha(tr(\alpha)\mid \phi_\alpha)\,
\big(\log p_\alpha(tr)-\log q_\alpha(tr(\alpha)\mid \phi_\alpha)\big)
\big].
\]
For single-site Metropolis–Hastings, only the neighborhood $\mathcal{N}(k)=\mathcal{J}_k\cup\{k\}$ contributes to the acceptance ratio:
\[
\alpha=\min\left\{1,\prod_{j\in \mathcal{N}(k)} \frac{p_j(tr')}{p_j(tr)} \cdot \frac{Q_\alpha(tr\mid tr')}{Q_\alpha(tr'\mid tr)}\right\}.
\]
For SMC, slicing yields local incremental weights
\[
\log w_t^n=\log p_t(tr_n)-\log p_{t-1}(tr_n)=\log \sigma'_2(p)-\log \sigma'_1(p)
\]
on the sliced CFG [2508.20922].

The empirical summary reports single-site MH speedups of $1.2\times$–$10\times$, VI gradient-variance reductions up to four orders of magnitude, and SMC per-step speedups of $3\times$–$10\times$ on the considered benchmarks [2508.20922].

## 5. Dynamic reactive graphs and execution-local PPGs

Reactive graphs recast a probabilistic program run as a finite directed acyclic graph $G=(V,E)$ built in topological order during execution [2606.30137]. The graph records a particular execution trace. Recursion and probabilistic branching therefore do not appear as graph cycles in a run-level PPG; they are represented by nested subgraphs [2606.30137].

Node types are partitioned into random-variable nodes $V_r$, deterministic computation nodes $V_d$, factor or observation nodes $V_o$, and subgraph nodes $V_s$ [2606.30137]. Random-variable nodes are created by `assume`; each stores the current sample, a change-id, and an update function capable of redrawing the variable, reweighting it under changed distribution parents, and recording rollback actions. Deterministic nodes are created by `pure`, `map`, and `apply`; factor nodes are created by `weight`; and subgraph nodes are created by `sub` and composed with `join` to handle dynamic control flow and recursion [2606.30137].

A compiler translates a high-level universal probabilistic program into a graph-building program over an interface with operations `pure`, `map`, `apply`, `assume`, `weight`, `join`, and `sub` [2606.30137]. The applicative fragment $\{\mathrm{pure},\mathrm{map},\mathrm{apply},\mathrm{assume},\mathrm{weight}\}$ expresses Bayesian networks with fixed topology, whereas `join` and `sub` extend expressiveness to universal probabilistic programming with dynamic topology [2606.30137].

During an MCMC proposal, the system selects a subset $S\subseteq V_r$ of random-variable nodes for redraw, sets a new proposal id, and runs node update functions in topological order [2606.30137]. Deterministic nodes recompute only if a parent changed. Weight nodes recompute their local contribution to the log joint. An `assume` node has three cases: direct redraw if selected, prior reweighting if its distribution parent changed but the stored sample remains in support, or redraw under support invalidation [2606.30137]. For subgraph nodes, changed inputs may trigger subgraph reconstruction. Acceptance uses
\[
\alpha = \min\Big(1, \exp(\Delta \log p + \Delta \log q)\Big)
\]
with transactional rollback by replaying a reset stack on rejection [2606.30137].

The log joint is treated as a local factorization
\[
\log p(\mathbf{x},\mathbf{y}) = \sum_i \log f_i(\mathbf{x}_i,\mathbf{y}_i),
\]
and only changed factors contribute
\[
\Delta \log p = \sum_{i \in \mathcal{A}} \Big[\log f_i(\mathbf{x}'_i,\mathbf{y}_i)-\log f_i(\mathbf{x}_i,\mathbf{y}_i)\Big].
\]
This is the formal basis for localized recomputation [2606.30137].

The paper explicitly connects the construction to functional reactive programming. Random-variable nodes act both as sources of samples and as sinks for their input distributions; change propagation follows edge-local data dependencies; and topological update order realizes a reactive dataflow machine [2606.30137]. The stated advantages are localized recomputation, a clean separation of deterministic and probabilistic computation, and support for universal PPLs via subgraphs, while stated limitations include node overhead, theoretically unsatisfying aspects of the automatic translation, and the absence of quantitative performance benchmarks in the present report [2606.30137].

## 6. Related extensions: graph-generating interfaces, dependence models, and recursive regular graphs

One extension uses the phrase PPG for typed graph-generating program interfaces rather than control-flow or dependency graphs [2312.17127]. The language has a type of vertices and two operations,
\[
\mathrm{new}:\mathrm{unit}\to \mathrm{vertex},
\qquad
\mathrm{edge}:\mathrm{vertex}\times \mathrm{vertex}\to \mathrm{bool},
\]
and programs such as
\[
\vdash\ t_n \equiv \mathrm{let}\ x_1\,\mathrm{new()}\ \cdots\ \mathrm{let}\ x_n\,\mathrm{new()}\ \big( \mathrm{edge}(x_i,x_j) \big)_{1\le i,j\le n}: \mathrm{bool}^{(n^2)}
\]
denote distributions over finite adjacency matrices [2312.17127]. In this setting, PPGs are equationally specified program interfaces interpreted in distributive Markov categories. The central theorem states that every Bernoulli-based equational theory with deterministic symmetric `edge` yields an exchangeable, consistent, and local random graph model and therefore a unique graphon up to cut-distance equivalence, and conversely every graphon arises from such a theory [2312.17127]. This use of “PPG” is semantically orthogonal to the control-flow and inference-oriented usages.

Another extension is the Probabilistic Program Dependence Graph (PPDG), a PDG augmented with learned statistical dependences between node states [1201.3985]. A PPDG has structure $G=(V,E)$, discrete node-state variables $X_v$, and CPTs $P(X_v\mid X_{\mathrm{Pa}(v)})$ learned from instrumented traces [1201.3985]. Count-based estimates are
\[
\hat{P}(X_i=s)=\frac{N_i(s)}{\sum_{s'\in S_i}N_i(s')},
\qquad
\hat{P}(X_i=s\mid \mathrm{Pa}(X_i)=t)=\frac{N_i(s,t)}{N_i(t)}.
\]
Fault localization is then performed by `RankCP`, which assigns suspiciousness by the minimum conditional probability seen for each node along a failing trace [1201.3985]. The reported comparison includes the example that, for the Hill Cipher program, SBI identified $25$ faults while PPDG identified $28$ faults [1201.3985]. This is a diagnostic rather than semantic use of a probabilistic program graph.

A third neighboring line models recursive probabilistic programs by probabilistic regular graphs generated from deterministic graph grammars with probabilistic terminal labels [1011.0222]. These graphs induce infinite-state Markov chains, support qualitative PCTL model checking, and admit approximate quantitative verification via finite polynomial fixed-point systems over summary variables $D$ and $W$ [1011.0222]. Exact quantitative PCTL model checking is undecidable, while qualitative thresholds $0$ and $1$ remain decidable [1011.0222]. This is not presented under the PPG label in the paper itself, but the supplied account explicitly relates probabilistic regular graphs to PPGs for recursive probabilistic programs.

Taken together, these variants establish that PPGs are not a single formal object but a family of graph-centric semantic, inferential, and analytic representations. The persistent technical themes are explicit factorization, control/dependence exposure, and the use of graph structure to obtain compositional semantics, correctness theorems, or local computation. This suggests that future unification, if pursued, would have to distinguish at least four axes already present in the literature: static versus dynamic graphs, whole-program versus per-trace representations, operational versus denotational roles, and exact versus conservative dependency structure.

Source: https://www.emergentmind.com/topics/probabilistic-program-graphs-ppgs