Probabilistic Generating Circuits (PGCs)

• PGCs are computational frameworks that represent multivariate probability distributions as generating polynomials with tractable marginalization for binary variables.
• They generalize and subsume models like probabilistic circuits, sum-product networks, and determinantal point processes by permitting negative coefficients to enhance expressiveness.
• PGCs use efficient two-pass evaluation strategies for inference and can be hybridized with neural components, successfully applied to density estimation and graph generation tasks.

Probabilistic Generating Circuits (PGCs) are a computational framework for modeling multivariate probability distributions that exploits the encoding power of generating polynomials. The PGC formalism unifies, generalizes, and in significant respects subsumes established tractable probabilistic models such as probabilistic circuits (PCs), sum-product networks (SPNs), and determinantal point processes (DPPs). By encoding distributions into multiaffine polynomials whose coefficients represent probabilities, PGCs enable efficient exact inference for a wide class of distributions, notably supporting tractable marginalization and conditioning in polynomial time for binary variables. For real-world density estimation and sequence modeling tasks, PGC-based learners have demonstrated practical competitiveness with state-of-the-art tractable probabilistic models. Theoretical developments reveal that the full expressive power of PGCs arises not from the polynomial representation itself, but from the relaxation that permits negative coefficients—thereby strictly extending the expressive (and tractable) frontier defined by monotone PC models.

1. Mathematical Definition and Circuit Semantics

A PGC over $n$ binary random variables $X_1, \ldots, X_n$ is a directed acyclic computational graph (arithmetic circuit) whose leaf nodes are either indeterminates $z_i$ or real-valued constants, and whose internal nodes are either sum ($\oplus$) or product ($\otimes$) gates. The output at the root node is a multiaffine polynomial, the probability generating polynomial:

$$g(z_1, \ldots, z_n) = \sum_{x \in \{0,1\}^n} p(x) \prod_{i: x_i=1} z_i$$

Here $p(x)$ is the joint probability mass function. The circuit is typically required to produce a generating polynomial with nonnegative coefficients summing to one, ensuring that $g$ encodes a valid probability distribution (Zhang et al., 2021, Agarwal et al., 2024, Broadrick et al., 2024).

PGC node types and their semantics:

Node Type	Operation	Constraint
Leaf	$z_i$ or $w\in\mathbb{R}$	N/A
Sum ($X_1, \ldots, X_n$0)	$X_1, \ldots, X_n$1, $X_1, \ldots, X_n$2 (or $X_1, \ldots, X_n$3 in generalized PGCs)	Children are polynomials; sum of weights = 1
Product ($X_1, \ldots, X_n$4)	$X_1, \ldots, X_n$5	N/A

PGCs are built by recursive composition, supporting mixture, factorization, and hierarchical substitution operations. For instance, substituting $X_1, \ldots, X_n$6 into another generating polynomial constructs hierarchically composed models that cannot be realized efficiently in classical PCs or DPPs alone (Zhang et al., 2021).

2. Inference and Computational Properties

The core tractability property of PGCs for binary variables is that marginal, conditional, and evidential queries reduce to polynomial evaluation after suitable variable substitution. Specifically, to extract the marginal probability $X_1, \ldots, X_n$7 for disjoint $X_1, \ldots, X_n$8, one evaluates

$X_1, \ldots, X_n$9

where the operation $z_i$0 extracts the coefficient of $z_i$1 (Zhang et al., 2021). In practice, exact inference is implemented via two-pass evaluation: bottom-up to compute node values and, if required, top-down to propagate contexts for marginals. For a circuit of $z_i$2 edges and $z_i$3 variables, total inference (including extraction of degree-$z_i$4 coefficients) operates in $z_i$5 time using FFT-based multiplication (Zhang et al., 2021).

In the general PC framework, smoothness (all children of sum-nodes share the same scope) and decomposability (children of product-nodes have disjoint scopes) suffice to guarantee tractable inference, namely $z_i$6 per query (Sidheekh et al., 2024). Determinism adds tractable MAP-inference (Sidheekh et al., 2024).

For categorical variables, tractability is lost when $z_i$7: PGC-marginalization becomes #P-hard through reduction to permanent computation (Agarwal et al., 2024, Broadrick et al., 2024).

3. Expressive Power, Connection to Other Models, and Formal Separations

PGCs strictly subsume several major classes of tractable probabilistic models. Given decomposable, smooth PCs (sum-product networks), one can construct an equivalent PGC by relabeling literals $z_i$8. Thus, every such PC can be simulated by a PGC of equal or smaller size (Zhang et al., 2021).

Moreover, PGCs can efficiently represent DPPs, which are parameterized via the determinant of matrix polynomial entries:

$z_i$9

where $\oplus$0 is a positive semidefinite kernel. A determinantal PGC construction yields $\oplus$1 network size for $\oplus$2-variable DPPs (Zhang et al., 2021). PGCs also encompass strongly Rayleigh measures (distributions with real stable generating polynomials), such as uniform (weighted) spanning-tree models, by encoding the combinatorial structure (e.g., Laplacian minors) as generating polynomials (Zhang et al., 2021).

A formal separation exists: There exist distributions (certain DPPs) admitting polynomial-size PGCs but only exponential-size decomposable PCs (Zhang et al., 2021). Further, PGCs with hierarchical compositions ($\oplus$3) cannot be efficiently emulated by PCs with tractable DPP leaves, strictly increasing representational power (Zhang et al., 2021).

Recent work demonstrates that the expressiveness of PGCs, when generalized to permit negative coefficients, is not inherently due to the polynomial representation per se, but rather to the relaxation of monotonicity (nonnegativity). In this sense, PCs with negative weights (nonmonotone PCs) are polynomially equivalent to PGCs in the binary case and strictly subsume them in categorical settings by allowing tractable marginalization through set-multilinear constraints (Agarwal et al., 2024).

4. Extensions: Probabilistic Graph Circuits and Sparse PGCs

Probabilistic Graph Circuits (“graph PGCs”) generalize the PGC paradigm to structured domains such as arbitrary attributed graphs. Here, circuits operate over subgraph scopes and encode joint distributions over both node/edge attributes and variable cardinality (Papež et al., 15 Mar 2025, Rektoris et al., 11 Aug 2025). Structural properties (smoothness of sum units over graph scopes and decomposability of product units into disjoint subgraphs) ensure tractable inference, supporting polynomial-time computation of partition functions, marginals, conditionals, and expectations over graph-valued random variables.

A core technical challenge for graph PGCs is enforcing permutation invariance. This is achieved either via inherent circuit-level invariance, explicit symmetrization (summing over all node permutations, which is intractable for large graphs), or via canonical sorting schemes (e.g., breadth-first ordering), which yield efficient but approximate likelihoods (Papež et al., 15 Mar 2025).

For large sparse graphs, Sparse PGCs (SPGCs) define the representation over node/edge lists and ensure exact inference with $\oplus$4 complexity for graphs of $\oplus$5 nodes and $\oplus$6 edges, circumventing the $\oplus$7 complexity in matrix-based representations while preserving the analytic tractability and competitive generative performance (Rektoris et al., 11 Aug 2025).

5. Learning, Algorithmic Techniques, and Neural Hybridization

Parameter learning in PGCs typically follows maximum-likelihood estimation, leveraging the tractable evaluation of log-densities and gradients enabled by circuit structure. Closed-form updates are available for deterministic PGC topologies:

$\oplus$8

Stochastic gradient descent and expectation-maximization algorithms are also widely used, with structure learning approaches ranging from heuristic split-and-cluster to randomized deep architectures (RAT-PCs, Einsum Networks) (Sidheekh et al., 2024). Extensions to Bayesian structure scoring and signomial programs for optimization are established (Sidheekh et al., 2024).

PGCs admit hybridization with neural components. Approaches include replacing sum-node weights with learned neural networks (e.g., HyperSPNs, conditional gating), combining PGCs with variational autoencoders (latent variable distillation, mixtures at leaves), or integrating invertible transform (flow) modules at internal nodes (Sum-Product-Transform Nets), provided circuit decomposability is preserved for tractable inference (Sidheekh et al., 2024).

6. Limitations, Hardness, and Open Problems

A primary structural limitation of PGCs is the loss of tractability for categorical variables of arity $\oplus$9, where even single marginal inference is #P-hard, stemming from reductions to computing the permanent—a notorious computational bottleneck (Agarwal et al., 2024, Broadrick et al., 2024). For $\otimes$0-ary variables with $\otimes$1, lower complexity regimes may exist, but general classes of tractable categorical PGCs remain an open problem (Broadrick et al., 2024).

The key distinction in expressive frontier is that for binary variables, PGCs, nonmonotone PCs, network, likelihood, and Fourier polynomial circuit representations are all polynomially interconvertible with the same tractable inference properties (Broadrick et al., 2024). For general nonmonotone PCs constrained to compute set-multilinear polynomials and valid probability distributions, tractable inference is restored for arbitrary arities, but deciding positivity is #P-hard (Agarwal et al., 2024).

Outstanding questions include structural characterization of maximally tractable PGC fragments, tight expressivity separations versus other tractable models, and extensions to broader domains (continuous, hybrid, or relational variables) (Broadrick et al., 2024, Sidheekh et al., 2024).

7. Empirical Results and Applications

Simple, hand-crafted PGC architectures (e.g., mixtures of determinantal PGCs) have demonstrated competitive or superior performance relative to established tractable models (DPPs, mixture of trees, Strudel, einsum networks) on a variety of binary density-estimation benchmarks, including UCI-style datasets and retail basket data (Zhang et al., 2021). In the graph domain, PGCs and their sparse variants achieve comparable validity, uniqueness, and chemical diversity metrics in molecular generation tasks (QM9, ZINC250k) relative to intractable deep graph generative models, while supporting principled exact inference and conditional sampling (Papež et al., 15 Mar 2025, Rektoris et al., 11 Aug 2025).

PGCs therefore enable a unified, expressive, and tractable probabilistic modeling toolkit with rigorous analytical underpinning, tractable inference guarantees, and practical performance across both classical tabular and modern structured domains.

---

Key References:

• "Probabilistic Generating Circuits" (Zhang et al., 2021)
• "Probabilistic Generating Circuits -- Demystified" (Agarwal et al., 2024)
• "Polynomial Semantics of Tractable Probabilistic Circuits" (Broadrick et al., 2024)
• "Building Expressive and Tractable Probabilistic Generative Models: A Review" (Sidheekh et al., 2024)
• "Probabilistic Graph Circuits: Deep Generative Models for Tractable Probabilistic Inference over Graphs" (Papež et al., 15 Mar 2025)
• "Sparse Probabilistic Graph Circuits" (Rektoris et al., 11 Aug 2025)