Proxy-Tuning: Surrogate-Guided Adaptation

• Proxy-tuning is an approach that uses surrogate models and metrics to adapt large, resource-constrained machine learning systems efficiently.
• It encompasses diverse techniques such as logit adjustment, proxy data synthesis, federated proxy tuning, and external metric feedback.
• Empirical benchmarks show that proxy-tuning can significantly close performance gaps in language and image models while reducing computational overhead.

Proxy-tuning is an approach for adapting machine learning models—especially large-scale neural models under resource or access constraints—by strategically leveraging smaller or more tractable “proxy” models or metrics to guide training, steering, or evaluation of the target system. The paradigm encompasses a family of techniques, all united by the principle of employing a surrogate (i.e., a proxy) in place of direct data, supervision, or intractable fine-tuning of the target model. This entry surveys proxy-tuning’s core methodologies, theoretical foundations, algorithmic variants, empirical results, and practical considerations across major use cases including black-box LLM adaptation, multimodal generation, optimization proxies, federated learning, and data selection.

1. Foundational Principles and Conceptual Scope

Proxy-tuning leverages a lightweight or tractable “proxy” in scenarios where direct, full-scale, or black-box model adaptation is inaccessible or computationally prohibitive. Proxies may be auxiliary models (smaller LLMs, diffusion models), scoring metrics, or data-driven surrogates. The essential motif is to steer the adaptation or output of a target system using the output, guidance, or supervision provided by these proxies, in place of expensive or even impossible direct optimization. Key venues include black-box LLMs whose weights are not modifiable, multimodal transformer image generators where data is scarce, and safety-critical optimization pipelines demanding fast, reliable surrogates.

Proxy-tuning is inherently flexible: it can be implemented as a decoding-time adjustment (aligning a large model’s predictions toward those of a fine-tuned small proxy), as a data-selection or ablation tool in data curation, as a means for federated adaptation under privacy constraints, or as a metric-guided process that constructs synthetic training targets from principle-specific external feedback.

2. Principal Methodologies Across Domains

Proxy-tuning has manifested as a set of formally distinct but conceptually aligned methodologies:

• Decoding-Time Logit Adjustment (“Logit Proxy-Tuning”): A small, accessible model is tuned to the target task, and then its difference in output logits (post-finetuning vs. its untuned version) is added to the pre-softmax logits of the large, unmodifiable foundation model at every inference step. This is most prominent in language modeling (e.g., LLAMA2-70B proxy-tuned via 7B proxies) (Liu et al., 2024).
• Proxy Data Supervision: A generative model (often a diffusion model, e.g., DreamBooth) is adapted for a specific subject or concept and then used to synthesize a large proxy dataset. This synthesized set is then employed to train a separate, usually autoregressive (AR) model better suited for generalization or compositionality (Wu et al., 13 Mar 2025).
• External Proxy Metric Feedback: External, automatic, and principle-specific metrics act as proxies for human evaluation. Candidate outputs are iteratively refined—as in self-refinement of LLM generation—until passing all metric-based constraints. The fine-tuned model is subsequently updated using only those outputs that pass the proxy evaluator (Ramji et al., 2024).
• Federated Proxy Tuning (FedPT): In federated settings, resource-constrained devices only tune and exchange proxy models (small LMs). The server then combines their updates with a large black-box foundation model by logit-level proxy-tuning, maintaining privacy and compute efficiency (Gao et al., 2024).
• Data Curation via Proxy: Small proxy models are trained on candidate data recipes with a “tiny” learning rate. Datasets are then ranked by validation loss under the proxy so as to reliably predict which would yield optimal performance in full-scale downstream training (Wang et al., 30 Dec 2025).
• Surrogate Modeling for Black-Box Adaptation: When query access to a foundation model is limited (e.g., via API), a surrogate (e.g., Gaussian Process) is trained on a selected, informative subset of API responses. This surrogate then guides proxy-tuning of a smaller model offline, yielding adaptation with drastically fewer queries (Xie et al., 13 Nov 2025).
• Proxy-Tuning in Black-Box Adversarial Transfer: Proxy models are locally fine-tuned to reproduce the loss surface of a private black-box model in the vicinity of adversarial queries, boosting transferability of attacks (Shah et al., 2023).

3. Formalization and Algorithmic Recipes

While domain-specific realizations differ, the core algorithms generally follow the sequence:

1. Proxy Model/Metric Adaptation: Train or fine-tune the proxy (e.g., small LM, diffusion model, or external metric selector) on available supervision, often using parameter-efficient techniques like LoRA or low-rank adapters.
2. Proxy Supervision or Signal Extraction: Obtain proxy outputs—either as logits, generated data, or metric feedback—on a designed set of inputs (real, synthesized, or selected via diversity heuristics).
3. Induced Adaptation of Target Model or Data:
   • For logit adjustment: The target model’s outputs are modified at inference by shifting its logits according to the proxy’s (e.g., $$s_\mathrm{proxy}(y|x) = s_L(y|x) + \alpha\Delta(y|x)$$, with $\Delta(y|x)$ the logit difference between proxy expert and anti-expert at $x$) (Liu et al., 2024, He et al., 2024).
   • For data-driven AR models: The AR student is trained on data synthesized by the proxy generator or diffusion supervisor (Wu et al., 13 Mar 2025).
   • For federated or resource-limited cases: Adapt proxy models are aggregated and their learned offset is applied to the black-box target at the central server (Gao et al., 2024).
   • For metric-driven refinement: Candidate outputs are iteratively refined to pass external metric thresholds, then used as synthetic fine-tuning targets for the next round (Ramji et al., 2024).
4. Consistency Enforcement (CPT): Recent work addresses the mismatch between training on vanilla proxy models and testing with logit-adjusted proxy-tuned predictions. “Consistent Proxy Tuning” incorporates the logit adjustment into the training objective itself, ensuring that the supervised targets match test-time inference (He et al., 2024).

Algorithmic pseudocode in the literature reflects these structures, e.g.:

for t in range(T): sL = L.logits(prefix) # Large, frozen s0 = S0.logits(prefix) # Proxy, untuned s1 = S1.logits(prefix) # Proxy, tuned s_proxy = sL + alpha * (s1 - s0) next_token = greedy_or_sample(s_proxy) prefix.append(next_token)

and

loss = cross_entropy(z_s + alpha * (z_l^p - z_s^p), y)

as in (He et al., 2024).

4. Theoretical Justification, Performance, and Generalization

Proxy-tuning approaches are supported by both theoretical and empirical analyses:

• Weak-to-Strong Generalization: Particularly in multimodal AR models, proxy-tuning yields “student” AR models exceeding the “teacher” diffusion supervisor in subject fidelity and prompt adherence—a phenomenon attributed to the AR’s semantic priors filtering out proxy idiosyncrasies (Wu et al., 13 Mar 2025).
• Preservation of Dataset Rankings: Proxy-tuning at small learning rates in proxy data ablation experiments provably preserves the ordering of achievable losses across candidate datasets under broad conditions (random-feature models, infinite-width kernels, SGD with sufficiently small $\eta$), ensuring reliable data selection (Wang et al., 30 Dec 2025).
• Approximation Quality: Decoding-time logit adjustment is formally analogous to first-order Taylor expansion of full fine-tuning, with the proxy’s logit difference acting as a surrogate for the gradient-induced shift one would achieve by directly tuning the large model (Liu et al., 2024).
• Consistency Bounds (CPT): Embedding the logit offset into the training objective ensures that the train and test predictions are aligned, removing the approximation error due to the proxy–test-time ensemble mismatch (He et al., 2024).
• Uniform Proxy Error Bounds in Optimization: Sobolev proxy training aligns not just values but local sensitivities, yielding quadratic improvement in uniform error as a function of training set covering radius under Lipschitz continuity, greatly benefiting optimization proxies (Rosemberg et al., 16 May 2025).

5. Empirical Benchmarks and Applications

Proxy-tuning has demonstrated robust performance across architectures and application domains:

Application	Core Metric/Setup	Proxy-Tuning Outcome	Reference
LLM adaptation	AlpacaFarm, TruthfulQA	Closes 88–91% of gap to full-tuned chat LLaMA2-13B/70B	(Liu et al., 2024)
Code adaptation	Pass@10, CodeLlama	LLaMA2-13B: 33.7% → 65.7% (proxy); LLaMA2-70B: 62.0% → 70.7%	(Liu et al., 2024)
Multimodal AR T2I	CLIP-I, DINO, CLIP-T	Proxy-tuned AR student outperforms diffusion teacher	(Wu et al., 13 Mar 2025)
Black-box fine-tuning	11 NLP tasks, accuracy	LLM: 55.92% (pretrain) → 86.85% (GP-proxy-tuned)	(Xie et al., 13 Nov 2025)
Data curation	Pile validation loss	Standard proxy η: Spearman ρ < 0.75; tiny-η: ρ > 0.92	(Wang et al., 30 Dec 2025)
Fed learning LMs	ROUGE-L, comm./VRAM	FedPT: comm./VRAM reduced 30–45%; accuracy near full fine-tune	(Gao et al., 2024)

In addition, proxy-tuning underpins methods in adversarial attack transferability on LLMs (LoFT), fast optimization in large-scale operations (Sobolev proxies), and black-box few-shot adaptation.

6. Limitations, Constraints, and Extensions

Notable limitations and points of empirical sensitivity for proxy-tuning include:

• Inference Overhead: Logit-level proxy tuning often requires simultaneous forward passes through multiple models at inference, with $2–3$x runtime and increased memory use unless optimized via parallelism (Liu et al., 2024, He et al., 2024).
• Hyperparameter Tuning: Empirical performance is sensitive to the proxy weight ($\alpha$), learning rate ($\eta$), and, in surrogate-based approaches, the number and selection of queried instances (Wang et al., 30 Dec 2025, Xie et al., 13 Nov 2025).
• Proxy Quality Dependence: The final system’s accuracy is capped by the proxy’s ability to match or filter essential structure; poor proxies may limit transfer (Shah et al., 2023).
• Train–Test Consistency: The original logit-shift proxy-tuning is mismatched at train/test; CPT explicitly addresses this (He et al., 2024).
• Scaling and Surrogate Limits: GP or Bayesian surrogate models for API-efficient proxy-tuning scale cubically with the subset size, imposing hard upper bounds on data budget (Xie et al., 13 Nov 2025).
• Extension to Other Modalities: While extensively validated in language and image domains, adaptation to video, audio, or fine-grained structured outputs is the subject of ongoing work (Wu et al., 13 Mar 2025).

7. Directions for Theory and Practice

Active research targets several promising directions:

• Automated Proxy Data Sizing: Employing active learning or uncertainty-based selection to optimally allocate proxy supervision (Wu et al., 13 Mar 2025).
• Learnable and Adaptive Proxy Combination: Per-class or per-decision adaptive weighting of proxy offsets ($\alpha$), as well as stacking multiple proxies for continual learning (He et al., 2024).
• Stronger Surrogate Models: Replacing GPs with scalable uncertainty-aware neural surrogates to extend to higher dimensions (Xie et al., 13 Nov 2025).
• Co-Tuning and Distillation Loops: Federated proxy-tuning architectures with iterative knowledge distillation (Gao et al., 2024).
• Theoretical Generalization Bounds: Formal characterization of train–test gap under proxy-mismatch and logit adjustment (He et al., 2024).
• Optimization-Driven Proxies: Applying directional-derivative (e.g., Sobolev) proxy supervision to improve surrogate quality for fast, reliable deployment in constrained settings (Rosemberg et al., 16 May 2025).

Proxy-tuning represents a canonical framework for practical machine learning model adaptation in the face of resource, access, or supervision constraints, with demonstrated robustness, extensibility, and rapidly broadening application scope.