PolyGraph Discrepancy (PGD) Metric

• The paper introduces PGD as a classifier-based evaluation metric that estimates a variational lower bound on the Jensen–Shannon divergence between descriptor distributions.
• PGD replaces kernel-based MMD with probabilistic discrimination, yielding scores on a bounded [0,1] scale that simplify cross-descriptor comparison.
• The method enforces strict train/test separation and cross-validation to select the most informative descriptor, ensuring robust benchmarking of graph generative models.

Searching arXiv for the cited paper and closely related evaluation work to ground the article. PolyGraph Discrepancy (PGD) is a classifier-based evaluation metric for graph generative models designed for the standard setting in which a sample of reference graphs from a target distribution is compared with a sample of generated graphs from a model. It was introduced to address limitations of Maximum Mean Discrepancy (MMD)-based evaluation, especially the absence of an absolute scale, sensitivity to kernel and descriptor parametrization, poor comparability across descriptors, and substantial instability on small graph benchmarks. PGD replaces kernel distance estimation with probabilistic discrimination: graphs are featurized by a descriptor, a probabilistic binary classifier is trained to distinguish reference from generated samples, the held-out classifier log-likelihood is interpreted as a variational lower bound on the Jensen–Shannon (JS) divergence, and the reported score is the corresponding distance-like quantity in the unit interval $[0,1]$, where lower values indicate greater similarity between generated and reference graph distributions (Krimmel et al., 7 Oct 2025).

1. Motivation and problem setting

The method is defined for graph generation benchmarks in which one has two samples: reference or real graphs drawn from a target distribution, and generated graphs drawn from a model. The evaluation task is to quantify how different the two graph distributions are. In common practice, this is done by computing MMD between descriptor distributions, using features such as degree histograms, clustering coefficients, Laplacian spectra, orbit counts, or GNN embeddings. The paper argues that this regime is useful for ranking models under a fixed setup, but that it has several structural weaknesses (Krimmel et al., 7 Oct 2025).

First, MMD lacks absolute interpretability. A value such as $0.03$ or $0.3$ has no intrinsic meaning across kernels or descriptors, unlike a bounded probability distance. Second, MMD is highly sensitive to external design choices, especially the kernel and its parameters; the paper emphasizes that rescaling features can rescale MMD, and that prior work had already shown sensitivity to kernel parametrization. Third, because MMD values are descriptor- and kernel-dependent, numerical values obtained from degree features and orbit features are not directly comparable, so one cannot infer from the raw scale which descriptor is more informative. Fourth, on small-sample graph benchmarks with only tens of graphs, the paper reports substantial bias and variance in standard MMD estimates.

PGD is proposed as a remedy to these issues. Its central interpretive claim is simple: if a classifier can easily distinguish descriptor vectors of real and generated graphs, then the two distributions are discrepant under that descriptor; if discrimination is difficult, they are similar. This yields descriptor-wise scores on a common scale and supports descriptor comparison within a single benchmarking framework. The associated software is released as the PolyGraph framework by BorgwardtLab at https://github.com/BorgwardtLab/polygraph-benchmark (Krimmel et al., 7 Oct 2025).

2. Variational formulation and descriptor-wise score

Let $P$ and $Q$ denote the reference and generated graph distributions over a graph space $\mathcal X$. The paper gives the variational representation of the Jensen–Shannon divergence as

$$\operatorname{JSD}(P\Vert Q) = \sup_{D:\mathcal X\to[0,1]} \left[ \frac12 \mathbb E_{x\sim P}\log_2 D(x) + \frac12 \mathbb E_{x\sim Q}\log_2(1-D(x)) + 1 \right].$$

Under equal class priors, the objective is the binary data log-likelihood of a discriminator $D$, up to the additive constant $+1$. The paper states explicitly that the log-likelihood of any classifier provides a valid lower bound on the JS divergence, and that the bound is tightened by fitting a classifier via maximum-likelihood methods. It also reviews the more general $f$-divergence variational bound from Nguyen et al., with the tightness condition that the functional class be expressive enough to contain a subderivative of $0.03$0 at the density ratio $0.03$1 (Krimmel et al., 7 Oct 2025).

PGD does not operate on raw graphs directly. A descriptor

$0.03$2

is first applied, and the classifier is trained on descriptor vectors. Consequently, the variational object is a lower bound on the JS divergence between descriptor distributions. The reported metric is then the square root of the nonnegative clipped held-out lower bound: $0.03$3 In the paper’s pseudocode this appears as

$0.03$4

This square-root step matters because the variational quantity is a divergence estimate, whereas the reported score is a distance-like quantity. Because the formulation uses $0.03$5, JS divergence lies in $0.03$6, and the resulting JS distance also lies in $0.03$7. The normalization is therefore intrinsic rather than imposed by ad hoc rescaling. Under this interpretation, a value near $0.03$8 means that real and generated graphs are hard to distinguish under the chosen descriptor and classifier, whereas a value near $0.03$9 indicates near-maximal separability in descriptor space.

3. Estimation pipeline and descriptor selection

The estimation procedure has two levels: single-descriptor discrepancy estimation and multi-descriptor selection. For generic graph datasets, the descriptor set used in the paper includes degree histogram, clustering coefficient histogram, Laplacian spectrum histogram, 4-node orbit counts, 5-node orbit counts, and GIN embeddings from a randomly initialized GIN. The appendix specifies 100 bins in $0.3$0 for clustering histograms, 200 bins in $0.3$1 for Laplacian spectrum histograms, and orbit counts computed with ORCA. For molecular graphs, the descriptor family is expanded with Morgan fingerprints, ChemNet embeddings, MolCLR embeddings, RDKit topological descriptors, and Lipinski/physicochemical descriptors (Krimmel et al., 7 Oct 2025).

To avoid overestimating discrepancy through memorization, the reference and generated datasets are each split into disjoint fit and test halves. The pseudocode uses alternating indices—reference[0::2], generated[0::2] for fitting and reference[1::2], generated[1::2] for testing—but the methodological requirement is simply disjoint fit and held-out test subsets of equal size. The discriminator is required to be probabilistic, efficient, and hyperparameter-free. The paper selects TabPFN as the default classifier and studies logistic regression as an alternative; TabPFN generally produces tighter and less noisy lower bounds.

Descriptor selection is performed only on fit data. For each descriptor $0.3$2, the method carries out 4-fold stratified cross-validation on the fit set, using three folds for training and one for validation, and averages the validation PGD estimate across folds. The final descriptor is then chosen as

$0.3$3

A classifier is retrained on the full fit split using $0.3$4, evaluated on the held-out test split, and the resulting held-out lower bound is transformed into the final PGD score.

This selection rule is the paper’s central aggregation mechanism. Descriptors are not concatenated in the main method. Instead, each descriptor yields its own lower bound, and the largest cross-validated lower bound is taken as the maximally tight lower bound available from the considered descriptor family. The appendix notes that this may underuse complementary information across descriptors and suggests that future work could combine features before classification, but that is not part of the core method.

4. Summary metric, theoretical properties, and interpretation

The final PolyGraph Discrepancy is the held-out score of the descriptor selected by maximal cross-validated discrepancy: $0.3$5 Conceptually, this is the maximum descriptor-wise lower bound available from the chosen descriptor set. The justification is theoretical rather than heuristic: each descriptor-specific classifier yields a lower bound on JS distance after featurization, and the largest such lower bound is the tightest one available from that descriptor family (Krimmel et al., 7 Oct 2025).

Several properties are emphasized. The first is boundedness on $0.3$6, which gives the score an absolute scale lacking in MMD. The second is comparability across descriptors, because all descriptor-wise quantities estimate the same bounded divergence family. The third is the lower-bound interpretation: the method estimates discrepancy between descriptor distributions, and only indirectly the discrepancy between full graph distributions. The appendix states this limitation explicitly: PGD operates on hand-crafted descriptors rather than raw graphs, so it yields a lower bound on divergence between descriptor distributions, which is itself a lower bound on divergence between graph distributions.

A further dependence is on classifier expressiveness. If the discriminator family cannot approximate the optimal classifier, the lower bound may be loose. This is not merely a theoretical caveat; the paper’s ablation shows that TabPFN systematically yields higher PGD values than logistic regression, which is interpreted as providing tighter lower bounds. The paper does not prove formal finite-sample consistency theorems for the full estimation-and-selection pipeline, but it reports empirical monotonicity with perturbation magnitude and training progress.

The work also places PGD in a broader discriminator-based perspective by deriving

$0.3$7

which frames MMD as an RKHS-constrained classification functional. The contrast, then, is not that MMD is unrelated to discrimination, but that PGD uses probabilistic log-likelihood and a JS-based objective with fixed scale and direct probabilistic semantics.

5. Empirical behavior and benchmarking

The empirical study covers instability analysis, synthetic perturbations, model-quality tracking, representative-model benchmarking, classifier ablations, and sample-size ablations. Before presenting PGD as a benchmark metric, the paper shows that standard MMD estimates exhibit high bias and variance on common small procedural graph benchmarks, especially around sample sizes of $0.3$8–$0.3$9 graphs. This motivates larger datasets, unbiased estimation, and explicit uncertainty reporting (Krimmel et al., 7 Oct 2025).

In synthetic perturbation experiments on Proteins, Citeseer ego nets, Planar, SBM, and Lobster, the perturbed operations include edge deletion, edge addition, edge rewiring, dataset-level mixing with Erdős–Rényi graphs, and a new edge-swapping perturbation that preserves degrees. One half of each dataset is kept fixed as reference while the other half is perturbed. PGD correlates strongly with perturbation magnitude, on par with or better than MMD. The paper also reports a characteristic saturation effect: for very large perturbations, PGD can approach $P$0, so correlations are evaluated in the non-saturated regime. A notable example is edge swapping, where degree-based and GIN-based MMD may miss the perturbation, while PGD remains informative by selecting a more discriminative descriptor.

For model-quality tracking, the paper studies DiGress on Planar-L, SBM-L, and Lobster-L, using increasing numbers of denoising steps and training epochs as quality proxies. PGD decreases as generation quality improves, correlates much more strongly with validity than MMD metrics, and tracks training progression more smoothly, whereas MMD can be erratic or even negatively correlated. The reported negative Pearson correlations between PGD and validity are described as very high, for example around $P$1 on Planar-L in denoising and training experiments.

In benchmark comparisons across AutoGraph, DiGress, GRAN, and ESGG on Planar-L, Lobster-L, SBM-L, Proteins, and the molecular datasets GuacaMol and MOSES, AutoGraph and DiGress usually obtain the best PGD values. PGD often aligns with validity and VUN rankings, but it also exposes cases in which validity alone is misleading. The Proteins dataset yields high PGD scores overall, which the paper interprets as suggesting greater modeling difficulty. Descriptor-wise subscores are also informative: orbit counts are often the most discriminative descriptor, but no single descriptor dominates universally.

The study of uncertainty repeats evaluation on 10 half-dataset subsamples without replacement. In subsampling experiments, PGD becomes markedly more stable beyond sample sizes of roughly $P$2. Timing experiments show that PGD is slower than a single MMD computation because it requires descriptor extraction, classifier training, and cross-validation across descriptors, but the paper describes it as still practical.

6. Interpretation, caveats, and relation to other discrepancy notions

PGD is primarily an evaluation metric, not merely a classifier two-sample test. The distinction drawn in the paper is that classifier two-sample tests often use classification accuracy as the statistic, whereas PGD uses probabilistic log-likelihood because that quantity directly yields a variational lower bound on JS divergence. The method is therefore framed as discrepancy estimation for graph generation rather than hypothesis testing in the narrow sense (Krimmel et al., 7 Oct 2025).

Several caveats are explicit. Strict train/test separation is essential: without held-out evaluation and fit-only descriptor selection, a powerful classifier could memorize samples and spuriously inflate the discrepancy estimate. Descriptor quality also remains a genuine bottleneck. Weak descriptors can understate true graph-distribution discrepancy even if the classifier is strong. Conversely, weak classifiers can make the lower bound loose even when descriptors are informative. The score is intrinsically better behaved than MMD with respect to arbitrary feature scaling, but it is still representation-dependent.

Descriptor-wise PGD values are intended to be interpretable structurally. A low degree-histogram PGD together with a high orbit-count PGD indicates matching low-order degree statistics but mismatched higher-order local motifs. The paper gives a concrete example of a clustering-coefficient PGD of $P$3 on Lobster-L, where clustering coefficients are structurally uninformative. This is one reason the proposed summary metric uses max-reduction rather than averaging: averaging across weak descriptors can conceal meaningful failures.

The term “discrepancy” in PGD should also be distinguished from other graph-theoretic and numerical-analysis usages. In graph theory, “positive discrepancy,” “graded discrepancy,” and “relative discrepancy” refer to induced-subgraph, prefix-ordering, or alignment-overlap notions such as $P$4, not classifier-based divergence estimation for graph generation (Räty et al., 2023, Chen et al., 27 May 2025, Luong-Le et al., 29 Jun 2025). In reduced-order modeling, “PGD” commonly denotes Proper Generalized Decomposition rather than PolyGraph Discrepancy (Ruel et al., 1 Jul 2026). These are terminologically adjacent but conceptually distinct.

In formal summary, PGD consists of five steps: select a descriptor $P$5; train a probabilistic binary classifier $P$6 to distinguish reference from generated descriptor vectors; estimate the held-out lower bound

$P$7

report

$P$8

and finally select the descriptor with maximal cross-validated discrepancy and evaluate it on a disjoint held-out test split. Within that design, PGD functions as a descriptor-aware, bounded, classifier-based estimate of graph-distribution discrepancy with a lower-bound interpretation in terms of Jensen–Shannon distance.