---
title: Score-Difference Proxy in Model Evaluation
url: https://www.emergentmind.com/topics/score-difference-proxy
type: topic
---

# Score-Difference Proxy in Model Evaluation

Searching arXiv for papers relevant to “score-difference proxy” and closely related uses of the term.
A **score-difference proxy** is a surrogate quantity designed to preserve the information carried by **differences in scores**, rather than to reproduce exact scores pointwise. Across several research areas, the term refers to constructions that replace an expensive, latent, noisy, or otherwise impractical target score with a more tractable proxy whose main purpose is to retain the ordering, local variation, or inferential content of score differences. In Bayesian network structure learning, the term is used explicitly for a learned approximation to the Bayesian network structure score that is accurate enough for **search** over neighboring graphs [1206.6452]. In generative modeling, it denotes transport fields or kernel mean-shift quantities that act as proxies for the score difference $\nabla \log p - \nabla \log q$ between data and model distributions [2304.12906; 2603.07514]. In sequential forecast evaluation, forecast comparison, and related inferential settings, the relevant observable object is often a score-difference process that serves as a proxy for a latent comparative target [2110.00115; 2109.02432]. The unifying theme is operational: the proxy is constructed so that the downstream procedure—search, optimization, testing, or comparison—remains effective even when the original score is unavailable or too costly to use directly.

## 1. Bayesian network structure learning formulation

The most direct use of the term arises in "Smoothness and Structure Learning by Proxy" [1206.6452]. There, the score-difference proxy is an approximation to the Bayesian network structure score specifically intended to be accurate enough for **search**, rather than for reproducing exact score values. The setting is Bayesian network structure learning, where every candidate graph must be scored against the full data set and the search space of graphs is super-exponential in the number of variables [1206.6452].

The exact score under study is the Bayesian Dirichlet equivalent score, or **BDe**, written as
$$
sc(G \mid D) = \prod_{i=1}^{n} \prod_{j \in C(x_i)} \frac{\Gamma(M_{ij})}{\Gamma(M_{ij}+N_{ij})} \prod_{k \in V_i} \frac{\Gamma(d_{ijk}+N_{ijk})}{\Gamma(d_{ijk})},
$$
where $G$ is the graph, $D$ is the data set, $N_{ijk}$ counts how often $x_i=k$ with parent configuration $j$, $N_{ij}=\sum_k N_{ijk}$, and $d_{ijk}$ are the pseudo-count hyperparameters [1206.6452]. The analysis is carried out on the logarithm of this score:
$$
\log sc(G \mid D) = \sum_i \sum_{j \in C_i} \left( \log \Gamma(M_{ij}) - \log \Gamma(M_{ij}+N_{ij}) + \sum_{k \in V_i} \left[\log \Gamma(d_{ijk}+N_{ijk})-\log \Gamma(d_{ijk})\right] \right).
$$

The learned proxy is denoted $\hat{sc}(G)$, but the central claim is that what matters for local graph search is fidelity to **local score differences** between neighboring graphs in the metagraph, namely graphs that differ by one edge addition or deletion [1206.6452]. The proxy is built with **Gaussian Process regression** in simple kriging form,
$$
\hat{y} = K(g,\hat{g})K(g,g)^{-1}y,
$$
with kernel
$$
k(G_1,G_2) = \exp\!\left(\sum_{e \in G_1 \cap G_2} w_e\right),
$$
where the weights $w_e$ are learned by maximizing marginal likelihood [1206.6452].

This construction separates the **search process** from the **original data**. A sample of candidate graphs is scored exactly once, the GP regressor is trained on those scored graphs, and the learned proxy is then used in place of the exact score during greedy search [1206.6452]. The practical significance is computational: the proxy reduces dependence on data size during search, avoids rescoring from raw data at each move, and can smooth out local irregularities so greedy search can escape shallow local maxima [1206.6452].

## 2. Smoothness justification and metagraph geometry

The main theoretical justification in [1206.6452] is a smoothness result for the BDe score over the **metagraph** topology, where vertices are directed graphs and edges connect graphs that differ by exactly one edge. The paper proves that the BDe score is **Lipschitz smooth** over this topology and that the score change caused by adding or deleting one edge is only logarithmic in the number of data points $N$ [1206.6452].

For graphs $G$ and $G'$ that differ by one added edge, the difference in log-score can be reduced to terms involving
$$
y(a,b)=\log \Gamma(a+b)-\log \Gamma(a)-\log \Gamma(b),
$$
which is essentially $-\log B(a,b)$ [1206.6452]. Using Stirling’s approximation, the paper derives
$$
y(a,a)= (2\log 2)\,a + \mathcal{O}(\log a),
$$
and from the extrema analysis concludes that the maximum and minimum possible score jumps from a single edge edit scale as $\mathcal{O}(\log N)$ [1206.6452]. The paper’s summary is that, with respect to the addition/deletion topology, the BDe score is Lipschitz smooth with constant $K=\mathcal{O}(\log N)$ [1206.6452].

In this formulation, the score-difference proxy is not merely a heuristic replacement for an expensive objective. It is justified by a bound showing that the true score landscape does not change arbitrarily fast under the neighborhood relation used by local search [1206.6452]. This suggests that a regression model defined on graph similarity can preserve the local comparative information needed by search even if its pointwise score predictions are imperfect.

The experimental results in [1206.6452] align with that view. On ADULT1, ADULT2, ADULT3, CENSUS-INCOME, TIC2000, and MUSK, the proxy-based greedy search was typically faster and in all but one case found networks that were comparable to or better than exact-search networks [1206.6452]. The paper reports, for example, exact versus proxy search times of $91.32$ versus $22.70 \pm 1.36$ seconds on ADULT1, $149.75$ versus $34.90 \pm 2.27$ on ADULT2, and $45.2$ versus $23.24 \pm 0.83$ on TIC2000 [1206.6452]. The most dramatic case was MUSK, where the proxy smoothed over local structure and found a much better-scoring network than exact greedy search, while CENSUS-INCOME was the one weaker case for which the authors state they do not know what property of that data set caused the poor behavior [1206.6452].

## 3. Score-difference proxies in generative modeling

A distinct but conceptually related usage appears in implicit generative modeling and score-based generation. In "The Score-Difference Flow for Implicit Generative Modeling" [2304.12906], the fundamental object is the score difference
$$
\nabla_x \log p(x) - \nabla_x \log q_t(x),
$$
viewed as a deterministic transport direction for moving a current or source distribution $q_t$ toward a target distribution $p$ [2304.12906]. The associated probability-flow ODE is
$$
\frac{d x}{d t} = \frac{\sigma(t)^2}{2} \left[ \nabla_x \log p(x) - \nabla_x \log q_t(x) \right].
$$

The paper shows that the score-difference direction is the one that most rapidly reduces KL divergence under the stated infinitesimal transport setting, and the corresponding decrease is proportional to the Fisher divergence [2304.12906]. Because the true scores are often unavailable or ill-defined in the ambient space, the method applies the flow to **proxy distributions** formed by Gaussian smoothing:
$$
\tilde{p}(x) = p(x;\sigma) = \int p(y)\,\mathcal{N}(x;y,\sigma^2 I)\,dy,\qquad
\tilde{q}(x) = q(x;\sigma) = \int q(y)\,\mathcal{N}(x;y,\sigma^2 I)\,dy.
$$
The key claim is that $\mathbb{D}_{\mathrm{KL}}(\tilde{q}\|\tilde{p}) = 0$ if and only if $q=p$, so aligning the proxies is equivalent to aligning the original distributions [2304.12906].

For Gaussian smoothing, the score-difference between the proxies can be written as
$$
\nabla_x \log p(x;\sigma) - \nabla_x \log q(x;\sigma)
= \frac{1}{\sigma^2} \left[ \frac{\mathbb{E}_{y\sim p}[K_{\sigma}(x,y)\, y]}{\mathbb{E}_{y\sim p}[K_{\sigma}(x,y)]} - \frac{\mathbb{E}_{y\sim q}[K_{\sigma}(x,y)\, y]}{\mathbb{E}_{y\sim q}[K_{\sigma}(x,y)]} \right],
$$
which the paper interprets as the difference between optimal denoisers for the target and source [2304.12906]. In this sense, a score-difference proxy is a practically estimable quantity that retains the transport content of the true score gap.

A related perspective is developed in "A Unified View of Drifting and Score-Based Models" [2603.07514]. There, the drifting discrepancy field
$$
\Delta_{p,q}(x) := \eta\big(\mathbf V_{p,k}(x)-\mathbf V_{q,k}(x)\big)
$$
is interpreted as a proxy for the score difference between data and model distributions, or more precisely between their **kernel-smoothed** versions [2603.07514]. For Gaussian kernels, the correspondence is exact:
$$
\mathbf V_{\pi,k_\tau}(x)=\tau^2\,\mathbf s_{\pi,\tau}(x), \qquad
\Delta_{p,q}(x)=\eta\tau^2\big(\mathbf s_{p,\tau}(x)-\mathbf s_{q,\tau}(x)\big).
$$
For general radial kernels, the paper derives an exact decomposition into a preconditioned score term plus a residual, and for the Laplace kernel it proves error bounds showing that drifting remains an accurate proxy for score matching in low-temperature and high-dimensional regimes [2603.07514].

These two papers use the same phrase differently from [1206.6452], but the structural analogy is close. In each case, the proxy is not required to recover an exact latent object everywhere. It is required to preserve the differential signal that drives an optimization or transport procedure.

## 4. Finite-difference proxies for score and Hessian terms

In score matching, another form of score-difference proxy appears as a local finite-difference replacement for derivatives. "Efficient Learning of Generative Models via Finite-Difference Score Matching" [2007.03317] rewrites score matching objectives in directional-derivative form and approximates those directional derivatives using only forward function evaluations.

The paper starts from the Hyvärinen score matching loss
$$
\mathcal{J}_{\text{SM}(\theta)}=\mathbb{E}_{p_{\textup{data}}(x)}\left[\text{tr}(\nabla_{x}^{2}\log p_{\theta}(x))+\frac{1}{2}\|\nabla_{x}\log p_{\theta}(x)\|_{2}^{2}\right]
$$
up to a constant [2007.03317]. It then notes that the relevant first- and second-order terms are directional derivatives and provides finite-difference identities such as
$$
v^{\top}\nabla_{x}\mathcal{L}_{\theta}(x)=\frac{1}{2}\mathcal{L}_{\theta}(x+v)-\frac{1}{2}\mathcal{L}_{\theta}(x-v)+o(\epsilon^2),
$$
and
$$
v^{\top}\nabla^{2}_{x}\mathcal{L}_{\theta}(x)v=\mathcal{L}_{\theta}(x+v)+\mathcal{L}_{\theta}(x-v)-2\mathcal{L}_{\theta}(x)+o(\epsilon^3),
$$
for $\|v\|_2=\epsilon$ [2007.03317].

These expressions are explicitly described as the fundamental score-difference proxy expressions: the first-order directional score is approximated by a difference of neighboring log-densities, and the second-order Hessian quadratic form by a centered second difference [2007.03317]. The resulting objectives FD-SSM, FD-DSM, and FD-SSMVR preserve the structure of the original score-matching formulations while avoiding nested differentiation [2007.03317].

The computational significance is that nested backpropagation through $\nabla_x$ and Hessian-vector products is replaced with independent evaluations of $\log p_\theta$ or $s_\theta$ at perturbed points [2007.03317]. Empirically, the paper reports comparable model quality together with lower runtime and memory use, including MNIST deep EBM timings of $673$ ms for SSM versus $230$ ms for FD-SSM and $282$ ms for DSM versus $162$ ms for FD-DSM [2007.03317]. This is another instance in which the proxy is useful because the downstream objective depends on local differential structure more than on direct access to exact derivatives.

## 5. Sequential and comparative forecast evaluation

In forecast comparison, the relevant quantity is frequently a score difference rather than an absolute score, and the observable score difference may itself function as a proxy for a latent evaluative target. "Comparing Sequential Forecasters" [2110.00115] defines the time-varying average forecast score differential
$$
\Delta_t := \frac{1}{t}\sum_{i=1}^t E_{i-1}\{ S(p_i, y_i) - S(q_i, y_i) \},
$$
with observable empirical version
$$
\hat\Delta_t := \frac{1}{t}\sum_{i=1}^t \{ S(p_i, y_i) - S(q_i, y_i) \}.
$$
The paper identifies $\hat\Delta_t$ as an observable proxy for the latent target $\Delta_t$ and studies the martingale error
$$
S_t := t(\hat\Delta_t - \Delta_t)=\sum_{i=1}^t (\hat\delta_i-\delta_i),
$$
which underlies confidence sequences, e-processes, and p-processes for anytime-valid sequential comparison [2110.00115].

Here the proxy role is inferential rather than computational. The score-difference trajectory $\hat\Delta_t$ is the quantity available online, while the object of interest is the running average of conditional expected score gaps [2110.00115]. The paper’s confidence sequences are of the form
$$
C_t = \hat\Delta_t \pm \frac{u(\hat V_t)}{t},
$$
and the empirical Bernstein version uses
$$
\hat V_t = \sum_{i=1}^t (\hat\delta_i - \gamma_i)^2,
$$
often with $\gamma_i=\hat\Delta_{i-1}$, to obtain variance adaptivity [2110.00115].

"Using Proxies to Improve Forecast Evaluation" [2109.02432] treats forecast comparison from a different angle. There, the central object is the **loss difference**
$$
\Delta L(Y)=L(x_1,Y)-L(x_2,Y),
$$
and the paper studies replacing $Y$ with an observed proxy $\tilde Y$ that preserves the relevant conditional moment [2109.02432]. For moment targets with strictly consistent scoring functions of the form
$$
L(x,y)=\phi(y)-\phi(x)-\phi'(x)\big(h(y)-x\big),
$$
the loss difference depends on $y$ only through $h(y)$ [2109.02432]. If the proxy satisfies $\mathbb E[h(Y)] = \mathbb E[\tilde Y]$, then the expected loss difference is unchanged, while the variance can be smaller if $\operatorname{Var}(\tilde Y)\le \operatorname{Var}(h(Y))$ [2109.02432]. The inferential consequence is increased power in Diebold-Mariano-type tests because the numerator is preserved while the variance of loss differences is reduced [2109.02432].

These two forecast-evaluation papers illustrate two distinct score-difference-proxy mechanisms. In one, the observable empirical score differential proxies a latent conditional expectation process [2110.00115]. In the other, a proxy outcome preserves expected loss differences while reducing their variance [2109.02432]. In both cases, the central quantity is comparative: the object of interest is a difference in scores or losses, and the proxy is judged by how well it preserves that comparative object.

## 6. Broader proxy interpretations and recurring design principles

Several additional papers broaden the conceptual range of score-difference proxies, although they use domain-specific terminology. In "Model Consistency as a Cheap yet Predictive Proxy for LLM Elo Scores" [2509.23510], the proxy is not an explicit score difference but a derived statistic intended to track a latent comparative score. The paper defines contest-level win probability
$$
p_{ij} = \frac{\text{Wins}_{ij} + 0.5 \cdot \text{Ties}_{ij}}{n_{ij}},
$$
forms the weighted average Bernoulli variance
$$
\overline{\text{Var}}(m_\text{judge}, M) = \frac{\sum_{ij \in M} n_{ij}\, p_{ij}(1-p_{ij})}{\sum_{ij \in M} n_{ij}},
$$
and rescales it to a bounded **Consistency** score
$$
\text{Consistency}(m_\text{judge}, M) = 1 - 4 \cdot \overline{\text{Var}}(m_\text{judge}, M).
$$
This score has a reported Pearson correlation of $0.91$ with human-produced LMSYS Chatbot Arena Elo across $24$ judge models [2509.23510]. A plausible implication is that the paper exemplifies the same design logic: an inexpensive observable is used because it preserves enough comparative structure to stand in for a much costlier target metric.

In "Estimate Level Adjustment For Inference With Proxies Under Random Distribution Shifts" [2605.06484], the operative quantity is the discrepancy
$$
d_k := \widehat{\theta}_k^\ast - \widehat{\theta}_k
$$
between proxy-based and primary estimates in historical domains. The paper models the latent bias $\phi_k := \theta_k^\ast-\theta_k$ as a random effect and uses the historical discrepancies $d_k$ as noisy observations of that bias [2605.06484]. The method-of-moments estimators
$$
\widehat{\rho} = \frac{1}{K-1}\sum_{k=1}^{K-1} \left(\widehat{\theta}_k^\ast-\widehat{\theta}_k\right),
$$
and
$$
\widehat{\gamma}^2 = \frac{1}{K-1}\sum_{k=1}^{K-1}(d_k-\overline{d})^2 - \frac{1}{K-1}\sum_{k=1}^{K-1}\widetilde{\sigma}_k^2
$$
are then used to adjust the target-domain interval [2605.06484]. This is again a discrepancy-preserving proxy construction, now at the level of domain-wise estimates rather than scores on models or graphs.

Across these formulations, several recurrent principles are visible.

| Principle | Description | Examples |
|---|---|---|
| Preserve differences, not levels | The proxy is evaluated by whether it preserves local or expected score differences | Bayesian network search [1206.6452]; finite-difference score matching [2007.03317] |
| Decouple expensive objects from downstream procedures | The proxy replaces repeated access to data, derivatives, or human labels | GP score proxy [1206.6452]; FD score matching [2007.03317]; LLM Elo proxy [2509.23510] |
| Use smoothness or variance structure for justification | Validity comes from Lipschitz bounds, asymptotics, or variance reduction | BDe smoothness [1206.6452]; Gaussian identity in drifting [2603.07514]; forecast proxy variance reduction [2109.02432] |

This suggests that “score-difference proxy” is less a single method than a methodological pattern: retain the comparative signal that controls the downstream task, while replacing the original scoring mechanism with a cheaper or more stable surrogate.

## 7. Misconceptions, limitations, and scope

A common misconception is that a score-difference proxy must accurately reproduce the original score numerically. The Bayesian network paper states the opposite: the proxy is meant to be accurate enough for **search**, not necessarily for reproducing exact score values [1206.6452]. What matters is fidelity to local score differences between neighboring graphs, because those differences determine the trajectory of greedy structure search [1206.6452].

A related misconception is that score-difference proxies are purely heuristic. In the Bayesian network setting, the use of the proxy is justified by a proved Lipschitz smoothness property of the BDe score over the edge-edit metagraph [1206.6452]. In drifting models, the Gaussian-kernel case yields an exact identity between mean shift and score difference on kernel-smoothed distributions, while the Laplace-kernel case comes with explicit approximation regimes [2603.07514]. In finite-difference score matching, the proxy constructions are asymptotically consistent under differentiability assumptions, with objective-gradient alignment guarantees for sufficiently small $\epsilon$ [2007.03317].

The limitations are equally domain-specific. In [1206.6452], the proxy was less effective on CENSUS-INCOME, and the authors state that they do not know what property of that data set caused the poor behavior. In [2509.23510], the consistency proxy separates broad quality tiers well but has weak within-cluster correlation among top models. In [2605.06484], proxy-based intervals can remain miscalibrated when the number of historical domains is small unless domain-bootstrap uncertainty is propagated. In forecast evaluation, proxy replacement requires that the proxy preserve the relevant conditional moment structure; otherwise expected loss differences need not be maintained [2109.02432].

Taken together, the literature treats score-difference proxies as justified only insofar as they preserve the comparative object actually used by the task. That comparative object may be a local graph-score increment, a score gap between model and data distributions, a directional derivative of a log-density, a loss difference between forecasts, or an estimate discrepancy across domains [1206.6452; 2603.07514; 2007.03317; 2109.02432; 2605.06484]. The term therefore names a family of surrogate constructions unified by a precise operational criterion: preserve the score differences that drive search, optimization, or inference, even when exact scores are inaccessible or too costly to compute.

Source: https://www.emergentmind.com/topics/score-difference-proxy