---
title: Model Proxies in ML and Inference
url: https://www.emergentmind.com/topics/model-proxies
type: topic
---

# Model Proxies in ML and Inference

A model proxy is a surrogate statistical or computational construct that approximates a target function, class, property, or latent variable, often for reasons of computational efficiency, tractability, privacy, or interpretability. The concept pervades machine learning, causal inference, federated optimization, fairness auditing, and scientific modeling, with instantiations that range from learned prototype vectors in metric learning to noisy classifiers for sensitive attribute inference. Model proxies can take the form of simple regression models, neural network predictors, clustering prototypes, or index-specific surrogates, and frequently serve as stand-ins for more expensive, complex, inaccessible, or latent processes.

## 1. Proxy-Based Deep Metric Learning

In deep metric learning, proxies are compact, learnable class representatives in the embedding space, designed to replace expensive pairwise or triplet computations with single comparisons to class prototypes. For example, ProxyNCA++ [2004.01113] associates each class $c$ with a unique proxy vector $p_c \in \mathbb{R}^D$, learned as a parameter and constrained post-optimization to the unit hypersphere. During training, embeddings $x_i$ are “pulled” towards their class proxy and “pushed” from others:

\[
L_{\mathrm{ProxyNCA++}}(x_i) = -\log \frac{\exp\left(-\|x_i - p_{c_i}\|^2 / T\right)}{\sum_{a=1}^K \exp\left(-\|x_i - p_{a}\|^2 / T\right)}
\]
with low temperature $T$ producing peaked assignment probabilities. Fast-moving proxies employ an elevated learning rate to match the evolving embedding space, and pooling choices (GMP vs. GAP) further impact alignment and discriminativity. ProxyNCA++ demonstrates a $22.9$ percentage-point improvement in Recall@1 across four retrieval datasets, with state-of-the-art class separation and scalability [2004.01113]. Similarly, Soft Orthogonal Proxies [2306.13055] introduce a regularization term favoring orthogonal proxy placement:

\[
R_{SO} = \|P^\top P - I_r\|_F^2
\]
minimizing redundancy and enhancing inter-class discrimination.

## 2. Proxies for Fast Approximate Set Selection

In large-scale retrieval and filtering tasks, proxy models are cheap classifiers or regressors that subsample or score records for more costly downstream evaluation by an oracle (human or high-end DNN). Importance-sampling frameworks such as SUPG [2004.00827] employ these proxies as scoring functions $A(x) \in [0,1]$, targeting hard guarantees for recall or precision:

\[
\Pr[\mathrm{Recall}(\mathrm{res}) \geq \tau_r] \geq 1 - \delta
\]
with variance-minimizing sample weights $w(x) \propto \sqrt{A(x)}$. This approach yields up to $30\times$ reduction in labeling costs or result sizes compared to uniform sampling, and provides asymptotic confidence intervals for query returns. Rigorous calibration and conditional independence assumptions are critical, especially when proxies are noisy, adversarial, or correlated [2004.00827].

## 3. Proxy Use and Detection in Model Fairness and Discrimination

A “proxy variable” or proxy model in fairness auditing refers to an input, intermediate variable, or component correlated with a protected attribute and influential in the model’s predictions. In linear regression [1810.07155], proxy detection is formalized as identifying $P = \sum \alpha_i \beta_i X_i$ with both high association $Asc(P,Z)$ and influence $Infl_f(P)$:

\[
Asc(P, Z) = \frac{\mathrm{Cov}(P, Z)^2}{\mathrm{Var}(P)\mathrm{Var}(Z)}, \quad Infl_f(P) = \frac{\mathrm{Var}(P)}{\mathrm{Var}(f(X))}
\]
Proxy detection is formulated as a second-order cone program (SOCP) with constraints on association and influence, extendable to “business necessity” legally exempt features. Proxies are crucial in explaining, mitigating, or regulating discriminatory behavior, and empirical studies confirm their prevalence and tractability [1810.07155]. In fairness estimation with inaccessible sensitive attributes, weak proxies (with merely better-than-random accuracy) suffice, provided they are informative and conditionally independent. Calibrated procedures with three proxies enable provably accurate fairness estimation, even as proxy quality degrades [2210.03175].

## 4. Model Proxies in Causal Inference and Identification

Proxies function as critical identification tools in causal inference under unmeasured confounding. Proxy controls [1810.00283] are observed variables informative about latent confounders $W$, partitioned into outcome-aligned ($V$) and treatment-aligned ($Z$) proxies. Identification leverages conditional independence and completeness assumptions, employing operator equations (Fredholm) and nonparametric bridge functions to recover average potential outcomes. In panels, time-lagged outcomes and future treatments serve as proxies for unobserved heterogeneity. Proximal frameworks, further extended to the adaptive selection of valid among many potentially invalid proxies [2507.19623], solve for causal effects via penalized regression and the median trick, attaining root-$n$ consistency and robust confidence intervals.

## 5. Model Proxies for Efficient ML Inference and Query Acceleration

Proxy models routinely accelerate pipelines in database-integrated ML systems, acting as lightweight filters preceding costly UDFs. Probabilistic predicates (PP) [2201.00309] and correlative proxy optimizers (CORE) build lightweight classifiers for each predicate, online or offline, and optimize ordering and accuracy assignments for minimal expected cost:

\[
C_\mathrm{CORE} = \sum_{i=1}^n s_i\,c_{\hat o_i} + s_i(1 - r_i)\,c_{F_i}
\]
Allowing for predicate correlations in proxy construction dramatically improves throughput (up to 80\%) over baseline and prior independence-based approaches, confirmed via branch-and-bound search and empirical study. Strategic injection of proxies in query rewrite rules ensures global accuracy targets and optimal filtering.

## 6. Proxy Models in Federated Learning and Large-Scale Adaptation

Proxies are increasingly integral in resource-constrained federated learning, privacy preservation, and decentralization. In ProxyFL [2111.11343], each client maintains a private model and a public proxy model, the latter trained under differential privacy via DP-SGD and mutual learning objectives:

\[
L_k^{\mathrm{proxy}}(\phi_k; \theta_k) = (1-\beta) L_{CE}(h_k) + \beta L_{KL}(h_k \| f_k)
\]
Only proxies are exchanged via gossip protocols, enabling heterogeneity in private models and per-client $(\varepsilon, \delta)$-DP guarantees. In FedPromo [2508.03356], server-side knowledge distillation first aligns a small CNN proxy to a large foundation model, then federated client classifiers train atop the frozen proxy, with novel regularization (inactive class preservation, class de-biasing) to combat non-IID data and semantic drift. Aggregation and plugging classifier heads back onto the foundation model yield domain-adapted, privacy-preserving models with minimal client computation.

## 7. Proxy Strategies in Language Models and Knowledge Mining

Recent innovations extend proxies to adapt and steer large language models (LLMs) with black-box weights via proxy-tuning [2401.08565]: the predictions of a small, tuned LM adjust the outputs of a large, untuned model at inference,

\[
p_t^{\mathrm{final}}(w) = p_t^{\mathrm{large, base}}(w) + \lambda \big(p_t^{\mathrm{small, tuned}}(w) - p_t^{\mathrm{small, base}}(w)\big)
\]
This closes most of the behavioral/tuning gap at a fraction of computation, and enables task/domain/temporal adaptation even for proprietary models. In knowledge mining (Falconer [2510.01427]), LLMs act as planners and annotators, producing minimal, task-unified proxy models for sequence labeling and classification (get_label, get_span) that achieve near-LLM accuracy at up to $20\times$ speed-up and $90\%$ cost reduction.

## 8. Proxies for Model Explanation, Topic Evaluation, and Scientific Reconstruction

ExplainReduce [2502.10311] summarizes local explanations (e.g., LIME/SHAP) by greedily selecting a minimal proxy set that maximizes coverage or minimizes loss over the full data:

\[
S^* = \arg\max_{|S| \leq k} C(S, \varepsilon) \quad \text{or} \quad \arg\min_{|S| \leq k} F(S)
\]
This produces a global surrogate explanation with interpretable clusters matching the closed-box behavior. ProxAnn [2507.00828] validates the use of LLM-based proxies for human-aligned topic modeling evaluation, where proxies are prompted to generate labels, fit ratings, and rankings, demonstrating indistinguishable statistical agreement with crowdworkers in both fit and representativeness tasks.

In hierarchical scientific modeling (hydroclimate reconstruction [2202.09383]), multiple proxies (e.g., tree rings, speleothems) are hierarchically fused via Bayesian inference to reconstruct latent climate indices over time, employing AR(1) stochastic process priors and sparse calibration.

## References

- ProxyNCA++ [2004.01113]
- Approximate Selection with Guarantees [2004.00827]
- Discriminatory Proxies in Linear Regression [1810.07155]
- Proxy Controls for Causal Panel Data [1810.00283]
- Deep Metric Learning with Soft Orthogonal Proxies [2306.13055]
- CORE: Correlative Proxy Models in ML Queries [2201.00309]
- Bayesian Hydroclimate Reconstruction with Proxies [2202.09383]
- ProxAnn: Topic Model Evaluation via LLM Proxies [2507.00828]
- ExplainReduce: Proxy-Based Model Summarization [2502.10311]
- ProxyFL: Federated Proxy Model Sharing [2111.11343]
- FedPromo: Federated Lightweight Proxy Models [2508.03356]
- EMQ: Evolving Training-Free Proxies for Quantization [2307.10554]
- Proxy-Based Fairness Evaluation [2210.03175]
- Location Tests with Noisy Proxies [2507.19696]
- Adaptive Proximal Causal Inference [2507.19623]
- Tuning Language Models by Proxy [2401.08565]
- Falconer: Knowledge Mining with Proxy Agents [2510.01427]

Source: https://www.emergentmind.com/topics/model-proxies