Autoencoding Probabilistic Circuits (APCs)

• Autoencoding Probabilistic Circuits (APCs) are a framework where a smooth, decomposable probabilistic circuit models a joint distribution over observed variables and explicit embedding variables.
• They leverage exact conditional inference for encoding and use a neural decoder for reconstruction, ensuring robust handling of missing data.
• Empirical studies show that APCs outperform traditional methods on both image and tabular datasets, offering superior reconstruction quality and embedding robustness.

Searching arXiv for the specified APC and closely related probabilistic-circuit papers to ground the article in current literature. arXiv search query: (Braun et al., 6 Jul 2025) Autoencoding Probabilistic Circuits (APCs) are a framework for tractable representation learning in which a smooth and decomposable probabilistic circuit (PC) models a joint distribution over observed variables $X$ and explicit embedding variables $Z$, written $p_C(X,Z)$. In this formulation, embeddings are not post-hoc features extracted from circuit activations, but explicit random variables in the probabilistic model itself. A latent code is obtained by exact conditional inference, typically $z \sim p_C(Z \mid X=x)$, and reconstruction is performed by a separate neural decoder $g_\theta : \mathcal Z \to \mathcal X$. APCs therefore combine exact probabilistic conditioning and marginalization in the encoder with neural reconstruction in a hybrid, end-to-end trainable architecture (Braun et al., 6 Jul 2025).

1. Definition and conceptual position

APCs were introduced to address representation learning with PCs, a setting in which prior approaches relied on external neural embeddings or activation-based encodings rather than explicit probabilistic embedding variables. The central modeling object is a PC encoder $C$ over $X \cup Z$, so that the learned representation is defined through the conditional posterior of $Z$ under the joint model: $p_C(Z \mid X=x) = \frac{p_C(X=x, Z)}{p_C(X=x)}.$ Because the encoder is a smooth and decomposable PC, the numerator and denominator are tractable, and the same applies under arbitrary partial evidence (Braun et al., 6 Jul 2025).

A key distinction in APCs is between internal PC latent variables $H$, associated with sum nodes through the latent-variable interpretation of PCs, and explicit embedding variables $Z$0, introduced as dedicated random variables modeled by input units in the circuit. The paper emphasizes that APC embeddings are not merely extracted activations or internal mixture assignments. They are actual random variables in the modeled joint distribution, and unlike the internal sum-node variables $Z$1, which are discrete, the explicit embeddings $Z$2 can be continuous. In the reported experiments, they are modeled with Gaussian input units (Braun et al., 6 Jul 2025).

This positions APCs between earlier PC representation methods and neural autoencoders. Relative to standard PCs, APCs extend the modeled domain from $Z$3 to $Z$4. Relative to autoencoders and variational autoencoders (VAEs), APCs do not learn a separate approximate encoder $Z$5; instead, encoding is exact conditional inference in a tractable joint model. Relative to prior PC embeddings such as activation-based and latent-state/MPE-based encodings, APCs make the embedding variables first-class probabilistic objects rather than structural readouts (Braun et al., 6 Jul 2025).

2. Probabilistic semantics and tractable inference

The tractability of APCs is inherited from the smooth and decomposable PC encoder. Smoothness requires that every sum node’s children have the same scope, while decomposability requires that every product node’s children have disjoint scopes. Under these conditions, APCs retain exact evidence computation, arbitrary marginals, conditionals, moments, ancestral sampling, conditional sampling of embeddings, and partial-evidence queries in time linear in circuit size, $Z$6 (Braun et al., 6 Jul 2025).

For fully observed input $Z$7, APC encoding is defined by conditional inference: $Z$8 For partial evidence $Z$9 on observed variables $p_C(X,Z)$0, APCs compute

$p_C(X,Z)$1

with missing variables marginalized exactly: $p_C(X,Z)$2 This exact conditioning under partial evidence is one of the defining properties of the framework. The appendix formulation further specifies that when an input variable is missing, the forward pass returns $p_C(X,Z)$3 at that input unit, corresponding to integrating it out (Braun et al., 6 Jul 2025).

The encoder posterior is obtained through a two-pass procedure. A forward pass computes the evidence $p_C(X,Z)$4 or $p_C(X,Z)$5 and caches local likelihood values. A subsequent backward sampling pass traverses the conditioned circuit. At a sum node with weights $p_C(X,Z)$6 and child likelihoods $p_C(X,Z)$7, the conditioned weights are reweighted as

$p_C(X,Z)$8

Sampling with these reweighted weights yields a sample from the conditioned circuit. The same mechanism applies under missingness, because unobserved inputs are marginalized during the forward pass (Braun et al., 6 Jul 2025).

The paper also states that APCs support

$p_C(X,Z)$9

as well as partial-evidence variants such as

$z \sim p_C(Z \mid X=x)$0

It further notes that embeddings can be obtained by sampling or most probable explanation (MPE) from the conditional state distribution, although the experiments emphasize sampling (Braun et al., 6 Jul 2025).

3. Architecture and optimization

The APC architecture is a hybrid design consisting of a PC encoder and a neural decoder. The encoder is a smooth and decomposable PC over $z \sim p_C(Z \mid X=x)$1. The decoder is a neural network

$z \sim p_C(Z \mid X=x)$2

that reconstructs $z \sim p_C(Z \mid X=x)$3 from $z \sim p_C(Z \mid X=x)$4. The paper emphasizes the complementarity of these components: the PC provides principled, tractable encoding and missing-data handling, while the neural decoder provides expressive nonlinear reconstruction (Braun et al., 6 Jul 2025).

For tabular data, the implementation uses EinsumNetworks with a RAT structure. Data variables and embedding variables are both represented at the input layer, and embedding inputs are inserted alongside data inputs. For image data, the paper uses a convolutional-style layerwise PC with input units for pixels, alternating product and sum layers, product layers that combine neighboring scopes and reduce spatial resolution by a factor of 2, and sum layers that behave somewhat like channel mixing. In the image APCs, each embedding input is randomly coupled with a random data variable unit via a local product node. The paper describes these design choices as simple heuristics rather than principled structure-learning solutions (Braun et al., 6 Jul 2025).

The embedding variables $z \sim p_C(Z \mid X=x)$5 are explicit random variables with their own domains and parametric input units in the PC. In all reported experiments, $z \sim p_C(Z \mid X=x)$6 are modeled with Gaussian input units. For images, the embedding dimensionality is $z \sim p_C(Z \mid X=x)$7 for MNIST and Fashion-MNIST and $z \sim p_C(Z \mid X=x)$8 for the other image datasets; tabular dimensions vary with dataset size (Braun et al., 6 Jul 2025).

Training uses a hybrid, multi-objective loss: $z \sim p_C(Z \mid X=x)$9 The reconstruction term is

$g_\theta : \mathcal Z \to \mathcal X$0

and the paper also writes $g_\theta : \mathcal Z \to \mathcal X$1, making the autoencoding interpretation explicit. Because a sampled PC trace induces a tree in which input units become conditionally independent given $g_\theta : \mathcal Z \to \mathcal X$2 and the internal latent choices $g_\theta : \mathcal Z \to \mathcal X$3, the embedding variables satisfy

$g_\theta : \mathcal Z \to \mathcal X$4

This supports coordinate-wise regularization against a prior $g_\theta : \mathcal Z \to \mathcal X$5: $g_\theta : \mathcal Z \to \mathcal X$6 The third term encourages the encoder to fit the joint model: $g_\theta : \mathcal Z \to \mathcal X$7 In the experiments,

$g_\theta : \mathcal Z \to \mathcal X$8

The paper characterizes this neither as pure maximum likelihood nor as standard variational learning, but as a hybrid reconstruction-plus-probabilistic-regularization objective (Braun et al., 6 Jul 2025).

A technical obstacle is that conditional sampling in PCs is discrete because of sum-node choices. APCs address this with differentiable sampling for PCs. The implementation replaces Gumbel-Softmax with SIMPLE, using a straight-through construction of the form

$g_\theta : \mathcal Z \to \mathcal X$9

where $C$0 is the sampled one-hot vector. Thus the forward pass is discrete, while the backward pass uses gradients of $C$1. This makes joint optimization of encoder and decoder possible (Braun et al., 6 Jul 2025).

4. Relation to prior PC representation methods and neural autoencoders

APCs were introduced partly in response to limitations of previous representation-learning approaches with PCs. Activation-based PC embeddings use activations of selected circuit units as continuous representations, while latent-state or MPE-based embeddings infer assignments to internal discrete latent variables associated with sum nodes. The APC paper identifies several limitations in these methods: no explicit embedding variables are modeled, the representation is often tied to circuit structure, modern PCs can contain a very large number of internal latent variables, earlier PC autoencoding schemes such as SPAE are largely post-hoc, and earlier approaches are not end-to-end trainable with modern neural decoders (Braun et al., 6 Jul 2025).

This yields two common clarifications. First, APC embeddings are not synonymous with internal sum-node latent states. The paper explicitly notes that CAT embeddings can be seen as a special case if one treats internal sum indicators as the latent representation, but APCs are more general because embedding variables can be inserted arbitrarily and use arbitrary distributions. Second, APCs are not simply standard PCs with a decoder attached. What distinguishes them is the explicit joint modeling of data and embeddings through $C$2, together with tractable conditional inference in that joint model (Braun et al., 6 Jul 2025).

The comparison with classical autoencoders and VAEs is likewise structural rather than merely empirical. Standard autoencoders require complete input or ad hoc imputation. VAEs learn a neural approximation $C$3 to the posterior and typically sample from a simple prior such as $C$4. APCs instead compute exact tractable conditionals from the joint PC encoder and can sample from the exact learned embedding marginal

$C$5

The paper argues that, unlike VAEs, APCs do not suffer from a mismatch between an approximate posterior and a simple sampling prior because they can sample from the exact learned marginal over embeddings (Braun et al., 6 Jul 2025).

A further misconception addressed by the reported experiments is that APC robustness under missingness is attributable primarily to the neural decoder. The paper stresses that APC, VAE, and MIWAE use the same neural decoder in the reconstruction experiments, so the gains are attributed to the encoder rather than decoder capacity (Braun et al., 6 Jul 2025).

5. Empirical behavior and reported capabilities

The empirical evaluation covers image datasets—MNIST, Fashion-MNIST, SVHN, CIFAR-10, Flowers, LSUN, CelebA, and Tiny-ImageNet—and the 20 DEBD binary tabular datasets. Baselines include SPAE, VAE, MIWAE, and missForest. The reported tasks include reconstruction quality under complete and missing inputs, downstream embedding quality via logistic regression, embedding-space visualization, latent-space generation, robustness under MCAR and MAR corruption, data-free knowledge distillation from a pretrained VAE teacher, and a preliminary study of out-of-distribution detection using embedding likelihoods (Braun et al., 6 Jul 2025).

The headline empirical claim is that APCs outperform existing PC-based autoencoding methods in reconstruction quality and produce embeddings competitive with, while showing superior robustness in handling missing data compared to neural autoencoders. Under MCAR corruption from $C$6 to $C$7 missing pixels, APCs maintain lower reconstruction MSE and higher SSIM than VAE, MIWAE, SPAE, and often missForest in the more challenging regimes. Average MSE values reported over corruption levels include MNIST: APC $C$8, MIWAE $C$9, missForest $X \cup Z$0, SPAE $X \cup Z$1, VAE $X \cup Z$2; CIFAR: APC $X \cup Z$3, MIWAE $X \cup Z$4, VAE $X \cup Z$5, missForest $X \cup Z$6, SPAE $X \cup Z$7; and CelebA: APC $X \cup Z$8, MIWAE $X \cup Z$9, VAE $Z$0, SPAE $Z$1. On tabular data, APCs achieve the best reconstruction on 18 of 20 DEBD datasets (Braun et al., 6 Jul 2025).

The robustness results are especially prominent under structured MAR corruption, including missing bands, missing halves, border-to-center, and center-to-border patterns. APCs consistently rank first across nearly all such corruption patterns, and the paper attributes this to exact marginalization in the PC encoder: missing values are left unobserved rather than imputed heuristically (Braun et al., 6 Jul 2025).

Embedding quality is evaluated by freezing the learned embeddings and fitting a logistic regression classifier. The paper reports that with complete data, neural autoencoders may sometimes obtain slightly higher downstream accuracy, but as corruption increases, APC embeddings remain linearly separable and useful, whereas VAE and MIWAE embeddings degrade rapidly toward random guessing. The accompanying t-SNE plots are described as showing that APC clusters remain distinct even at $Z$2 corruption on MNIST while neural embeddings collapse into an unstructured cloud (Braun et al., 6 Jul 2025).

The paper also reports that APCs can sample exact latent codes from $Z$3 and decode them, yielding samples that are more visually plausible than directly sampling a vanilla PC, which often exhibits circuit artifacts. In data-free knowledge distillation experiments, APC students match pretrained VAE teachers reasonably well under full evidence and surpass them under missing-data corruption in both reconstruction and downstream embedding quality. The out-of-distribution detection results are presented as preliminary: for APCs trained on MNIST, embedding likelihoods separate in-distribution MNIST from most OOD datasets, with only minor overlap with Fashion-MNIST (Braun et al., 6 Jul 2025).

6. Broader probabilistic-circuit context

APCs are most naturally understood as a specialized representation-learning architecture built on tractable PCs rather than as an isolated model family. Their encoder assumptions—smoothness and decomposability—place them directly in the same tractable PC lineage as Einsum Networks, which provide a scalable implementation of smooth and decomposable PCs through vectorized sum-product computation and dense tensor contractions. That line of work is relevant because APCs use EinsumNetworks with a RAT structure for tabular data, and more generally because efficient PC execution is a prerequisite for repeated conditional inference during training and encoding (Peharz et al., 2020).

Two adjacent lines of research further illuminate the APC design space. Probabilistic Integral Circuits (PICs) extend PCs with integral units representing continuous latent variables, allowing hierarchical continuous mixtures and yielding quadrature-induced PCs (QPCs) whose latent states are interpreted as ordered discretization points rather than arbitrary categories (Gala et al., 2023). APCs do not use PICs, and the APC paper does not formulate an integral-circuit encoder. Nonetheless, this suggests a broader connection between tractable PC inference and continuous latent-variable modeling, especially because PICs employ decoder-like conditional parameterizations $Z$4 via small neural networks (Gala et al., 2023).

Positive unital circuits (PUnCs) extend probabilistic circuits from nonnegative scalar computations to positive semidefinite matrix-valued computations, using POVM leaves, unital quantum operations, and probabilities of the form

$Z$5

That framework is not autoencoding-specific and the paper explicitly states that PUnCs are not APCs. A plausible implication is that PUnCs enlarge the space of tractable circuit parameterizations that APC-like architectures could potentially use in the future, while preserving normalization and tractable marginalization (Martires, 2 Jun 2025).

Within this broader context, APCs contribute a specific answer to a narrower question: how to turn a tractable PC from a density estimator into a representation learner. Their central move is to model explicit embedding variables inside the circuit, obtain those embeddings by exact conditional inference, and pair that encoder with a neural decoder. The APC paper also leaves several directions open, including more principled circuit structure choice, more principled or hierarchical placement of embedding variables, broader exploration of the weighting between reconstruction, KL, and NLL terms, scaling methods analogous to those used in modern VAEs, and further study of knowledge distillation and out-of-distribution detection (Braun et al., 6 Jul 2025).