Reference-Based Training Methods

• Reference-based training procedures use auxiliary signals (e.g., high-resolution images, sketches, or stronger model outputs) to guide learning, transfer structure, and verify outcomes.
• They encompass various learning paradigms—supervised, self-supervised, reinforcement, and training-free methods—applicable in tasks such as image restoration, segmentation, and language model alignment.
• Empirical studies report significant gains in metrics like PSNR, SSIM, BLEU, and evaluation accuracy, though challenges remain in reference selection and integration.

Reference-based training procedure denotes a family of methods in which learning, adaptation, or quality estimation is organized around an auxiliary reference signal rather than an isolated input–target pair. In the cited literature, the reference may be a high-resolution image, a sketch, a support set with masks, a same-speaker bona fide recording, a hidden clean image used only offline, a stronger model’s output, a guiding policy, or direct numerical simulation (DNS) trajectories. The procedure may be fully supervised, self-supervised, preference-optimized, reinforcement-learned, or training-free; the unifying property is that the reference constrains correspondence, transfers structure or style, adjusts sample weights, or supplies a proxy notion of correctness (Aslahishahri et al., 2023, Pan et al., 22 Apr 2025).

1. Reference as correspondence anchor, teacher, and verifier

A first axis of variation concerns the operational role of the reference. In reference-based super-resolution, the reference is a high-resolution image that contributes textures and correspondences to a low-resolution input. In few-shot instance segmentation, the reference is a small support set whose masks define the target category. In line-art colorization and sketch extraction, the reference provides color, brushstroke, or contour priors. In speaker anti-spoofing, a same-speaker bona fide utterance is paired with the test utterance during training. In LLM alignment, the reference may be a stronger model’s response or a previously optimized policy. In reference-guided flow control, the reference is DNS data available only over an early-time window (Han et al., 2023, Staněk et al., 9 Jun 2026, Shi et al., 18 Feb 2026, Ivagnes et al., 2 Mar 2026).

A second axis concerns when the reference is visible. Some procedures are explicitly reference-conditioned at inference, as in visual restoration and style transfer. Others use the reference only offline to define supervision. The hidden-reference artifact predictor learns a mapping $$\mathcal{A} \mapsto \mathcal{A} \ominus \mathcal{B}$$ from the distorted image alone, even though the target is computed from a hidden clean reference image. The paper’s training set construction uses a 50/50 mixture of natural patches and distorted patches, with balancing over error magnitudes to avoid collapse to trivial zero predictions (Bemana et al., 2018). This establishes an important distinction: reference-based training does not imply full-reference deployment.

A third axis is whether the reference is restrictive or informative. Standard DPO uses a fixed reference model initialized from the same SFT model as the policy, whereas Pre-DPO argues that the reference functions as a data weight adjuster and should instead encode “foresight” about the optimized policy state. Reference-guided LLM evaluation makes the same point from another direction: a reference output is useful only when the judge is explicitly instructed how to use it, as in RefEval or RefMatch, rather than treated as inert context (Pan et al., 22 Apr 2025, Shi et al., 18 Feb 2026).

2. End-to-end visual reference learning

In image restoration and synthesis, reference-based training procedures are often implemented as explicit dual-stream or multi-stream architectures. DARTS is a two-stream transformer for reference-based super-resolution in which the low-resolution stream and the high-resolution reference stream are processed separately and coupled by self-attention, cross-attention, and a gating attention mechanism. Queries for self-attention come from the low-resolution stream, cross-attention uses low-resolution queries with reference keys and values, and the two are blended per head as

$$A^h = (1-\sigma(\lambda_h))\cdot \text{self-attn} + \sigma(\lambda_h)\cdot \text{cross-attn}.$$

The model is trained end-to-end in a single stage with $L_1$ reconstruction loss, VGG19 relu5-1 perceptual loss, and adversarial loss based on the StyleSwin discriminator, hinge loss, $R_1$, bCR, and a wavelet discriminator. On SUN80 it reports a PSNR/SSIM of 29.83 / .809 (Aslahishahri et al., 2023).

Multi-reference super-resolution extends the same logic to variable-sized reference sets. LMR provides 112,142 groups of $300\times300$ training images, each with 1 target and 5 references of high-, medium-, and low-similarity levels. MRefSR aligns each reference, fuses them with a Multi-Reference Attention Module whose softmax is normalized over the reference dimension, and refines the result with a Spatial Aware Filtering Module. The method is trained first with reconstruction loss only, then fine-tuned with reconstruction, perceptual, and adversarial losses, using $\lambda_{rec}=1$, $\lambda_{per}=10^{-4}$, and $\lambda_{adv}=10^{-6}$. On the LMR test set, Ours-rec reports 31.81 dB / 0.895 and Ours-rec-LPF 31.98 dB / 0.898, compared with 30.64 dB / 0.869 for C²-Matching-LMR (Zhang et al., 2023).

Few-shot instance segmentation uses the reference differently: the support set is not merely a style exemplar but a category-defining memory. Reference Twice (RefT) uses support information twice. First, support masks are pooled into dynamic class centers that reweight query features. Second, support object queries are selected by mask IoU and used as instance-level references for cross-attention with query object queries:

$$\boldsymbol{q}_Q^{Enhanced}=\operatorname{softmax}\left(\boldsymbol{q}_Q \operatorname{TopK}(\boldsymbol{q}_S)^{\mathrm{T}}\right)\operatorname{TopK}(\boldsymbol{q}_S).$$

Built on Mask2Former, RefT replaces proposal-based support-query matching with feature-level and query-level transformer interactions and adds a class-enhanced base knowledge distillation loss for incremental FSIS (Han et al., 2023).

Reference-based line-art colorization exposes a complementary issue: reference coupling can destabilize optimization. The Stop-Gradient Attention (SGA) procedure observes that the query and key branches of dot-product attention often have negative cosine similarity with the total gradient, whereas skip and value branches are usually aligned. SGA therefore computes the attention map in a no-gradient region, preserving forward correspondence but stopping backward flow through the conflict-prone attention-map construction. The method reports improvements in FID of up to 27.21% and in SSIM of up to 25.67% (Li et al., 2022).

3. Reference-guided objectives in LLM training

In LLM preference optimization, the reference is an explicit component of the objective. Pre-DPO derives the DPO gradient as

$$\nabla_\theta \mathcal{L}_{\text{DPO}} = - \beta \,\mathbb{E}\!\left[\lambda \cdot \nabla_\theta \log \frac{\pi_\theta(y^+ \mid x)}{\pi_\theta(y^- \mid x)} \right],$$

where $$A^h = (1-\sigma(\lambda_h))\cdot \text{self-attn} + \sigma(\lambda_h)\cdot \text{cross-attn}.$$0 depends on both policy and reference likelihood ratios. The paper argues that the reference model is therefore a sample weight controller. Its procedure is two-stage: run DPO or SimPO once from $$A^h = (1-\sigma(\lambda_h))\cdot \text{self-attn} + \sigma(\lambda_h)\cdot \text{cross-attn}.$$1, set the optimized model as $$A^h = (1-\sigma(\lambda_h))\cdot \text{self-attn} + \sigma(\lambda_h)\cdot \text{cross-attn}.$$2, then re-optimize with DPO using $$A^h = (1-\sigma(\lambda_h))\cdot \text{self-attn} + \sigma(\lambda_h)\cdot \text{cross-attn}.$$3 as reference. Reported average gains are +2.5 points in AlpacaEval 2 length-controlled win rate, +2.8 points in AlpacaEval 2 win rate, and +2.6 points in Arena-Hard v0.1 win rate (Pan et al., 22 Apr 2025).

Reference-guided evaluation in non-verifiable alignment makes the reference output itself a soft verifier. RefEval tells the judge to treat the reference as a successful instruction-following exemplar; RefMatch asks which candidate is more similar to the reference in instruction-following pattern. Across 11 open-source judges and 5 human-annotated datasets, RefEval reports 79.1% evaluation accuracy, compared with 73.7% for Ref-Free (Ours), 74.8% for HREF-Ref, and 77.7% for RefMatch. The improved judges are then used to generate pairwise preferences for DPO self-improvement after an SFT distillation stage on DeepSeek-V3 references. The resulting Llama-3-8B-Instruct system reaches 73.1% on AlpacaEval and 58.7% on Arena-Hard, while Qwen2.5-7B reaches 70.0% and 74.1%, respectively (Shi et al., 18 Feb 2026).

Reference-based language training can also broaden the target space rather than judge outputs. Simulated Multiple Reference Training (SMRT) replaces one-hot supervision with a paraphraser distribution over plausible next tokens:

$$A^h = (1-\sigma(\lambda_h))\cdot \text{self-attn} + \sigma(\lambda_h)\cdot \text{cross-attn}.$$4

Training alternates SMRT and standard NLL with probability $$A^h = (1-\sigma(\lambda_h))\cdot \text{self-attn} + \sigma(\lambda_h)\cdot \text{cross-attn}.$$5, and paraphrase tokens are sampled from the top 100 paraphraser candidates. Reported gains range from 1.2 to 7.0 BLEU, and the method is described as complementary to back-translation (Khayrallah et al., 2020).

4. Training-free and latent-space reference procedures

Several recent systems retain the reference mechanism while eliminating task-specific training. MixSA formulates sketch extraction as latent editing rather than supervised image-to-image translation. A colored image $$A^h = (1-\sigma(\lambda_h))\cdot \text{self-attn} + \sigma(\lambda_h)\cdot \text{cross-attn}.$$6 and a reference sketch $$A^h = (1-\sigma(\lambda_h))\cdot \text{self-attn} + \sigma(\lambda_h)\cdot \text{cross-attn}.$$7 are inverted by DDIM, an initial contour sketch $$A^h = (1-\sigma(\lambda_h))\cdot \text{self-attn} + \sigma(\lambda_h)\cdot \text{cross-attn}.$$8 anchors structure, and the U-Net decoder replaces sketch self-attention keys and values with those from the reference sketch:

$$A^h = (1-\sigma(\lambda_h))\cdot \text{self-attn} + \sigma(\lambda_h)\cdot \text{cross-attn}.$$9

The mixed query $L_1$0 interpolates color content, contour structure, and reference guidance through the DCT module. Feature injection is concentrated in late decoder layers, typically within the 10th–11th layers, because those layers govern local stroke textures and nuances. On FS2K, the full variant “Initial + MSA + DCT + RCD” reports LPIPS 0.4309, PSNR 28.28, and FID 183.37 (Yang et al., 1 Jan 2025).

Training-free reference-based instance segmentation uses frozen foundation models rather than learned support-query adapters. The method based on DINOv2 and SAM2-L constructs a category memory bank from masked reference features, aggregates instance-wise and class-wise prototypes, matches target mask descriptors to reference prototypes, and refines scores with semantic-aware soft merging. It reports 36.8% nAP on COCO FSOD, 71.2% nAP50 on PASCAL VOC Few-Shot, and 22.4% nAP on Cross-Domain FSOD (Espinosa et al., 3 Jul 2025).

Reference Trustable Decoding (RTD) transfers the same idea to LLM inference. A datastore $L_1$1 is built from task examples, the current hidden state retrieves the top-$L_1$2 nearest references by Euclidean distance, and their labels induce a reference distribution $L_1$3 that is fused with the model distribution $L_1$4:

$L_1$5

RTD is explicitly training-free and gradient-free; it adapts a frozen base model at decoding time rather than via fine-tuning (Shi et al., 2024).

The hidden-reference artifact predictor occupies an intermediate position. It is trained with full-reference targets such as MSE, SSIM, or VGG16 layer-5 distance, yet the deployed predictor sees only the distorted image. The reference is therefore indispensable to supervision but absent from inference, which broadens the meaning of “reference-based procedure” beyond explicit runtime conditioning (Bemana et al., 2018).

5. Optimization design, stability, and reference attenuation

Reference-based procedures frequently change the optimization landscape rather than only the forward computation. Pre-DPO makes this explicit by interpreting the reference model as a data weight adjuster. SGA makes an analogous claim for attention training: internal branches of a reference-coupling module can carry mutually conflicting gradients, and suppressing the offending branches can improve convergence (Pan et al., 22 Apr 2025, Li et al., 2022).

RAT demonstrates a more counterintuitive outcome. During training, the detector receives both a test utterance and a same-speaker bona fide reference utterance through a Reference-Informed Block built from an MLP branch and a cross-attention branch. Yet the optimization process rapidly attenuates the reference contribution. The branch-energy ratio

$L_1$6

is about 0.05 in the first epoch and later drops to about 0.02, while replacing the reference with energy-matched noise changes the logit margin by less than 5% by the final epoch. The procedure is therefore reference-augmented at training time but largely reference-invariant at inference time (Staněk et al., 9 Jun 2026).

Reference-guided Image Synthesis Assessment (RISA) addresses a different instability: coarse supervision. Training data are synthesized from intermediate RIS checkpoints and weakly labeled by iteration count. Because those labels are too coarse, RISA adds pixel-wise interpolation,

$L_1$7

multiple binary classifiers in place of a direct regressor, an unsupervised contrastive loss based on correct and incorrect references, and a supremum loss that assigns style-preserving augmentations of real images the maximum score. The procedure shows how references can stabilize evaluation models even without human annotations (Guo et al., 2021).

Reference-guided reinforcement learning for Evolve–Filter regularization gives the same theme a numerical-analysis formulation. In the DD-RL-EF setting, the reward is built from the relative $L_1$8 velocity error against filtered DNS data over a limited early-time window, and a DQN learns a time-dependent discrete filter radius $L_1$9. This is a reference-guided policy-training procedure even though deployment occurs beyond the training interval and without reference access (Ivagnes et al., 2 Mar 2026).

6. Empirical behavior, applications, and limitations

Across domains, reference-based procedures consistently report gains when the reference is informative and the coupling mechanism is explicit. DARTS attributes competitive reference-based SR performance to attention alone, without multi-stage correspondence learning or distillation, and reports state-of-the-art on SUN80 at 29.83 / .809 PSNR/SSIM (Aslahishahri et al., 2023). MixSA reports that, among 50 participants, its outputs were preferred 71.68% of the time over Ref2Sketch, Semi-ref2sketch, and StyleID, and its ablations show that removing the contour anchor, MSA, or reconstruction color distribution correction degrades LPIPS, PSNR, and FID (Yang et al., 1 Jan 2025). Pre-DPO reports consistent gains over both DPO and SimPO on AlpacaEval 2.0 and Arena-Hard v0.1, including +3.0 LC, +4.0 WR, and +4.1 Arena-Hard WR on Llama3.2-3B-Instruct relative to DPO (Pan et al., 22 Apr 2025). RAT reaches 2.57% EER and 0.074 minDCF on ASVspoof 5 with a single detector, and notably does so even when the reference recording is replaced by a zero vector at inference (Staněk et al., 9 Jun 2026).

The same literature also shows that reference quality and reference handling remain central bottlenecks. MixSA struggles with very faint or low-contrast strokes and can become too tied to the initial contour map, making fluid or minimal brush styles difficult to realize (Yang et al., 1 Jan 2025). Training-free instance segmentation notes confusion between semantically similar categories, missed small objects, under-detection in crowded scenes, and sensitivity to the selected reference set; the paper identifies optimal reference selection as an open problem (Espinosa et al., 3 Jul 2025). Reference-guided LLM judges improve most when references come from frontier models or human-edited “Oracle” outputs, although even weaker references still help (Shi et al., 18 Feb 2026). Multi-reference super-resolution argues that merely stitching references into one large image is memory-inefficient and fails to model relationships among references, which is why explicit multi-reference interaction is required (Zhang et al., 2023).

A recurring misconception is that “reference-based” implies permanent dependence on a reference channel at deployment. The hidden-reference artifact predictor, RAT, and reference-guided RL all contradict that assumption in different ways: the reference may disappear after supervision is learned, after invariance is induced, or after policy training on an early-time window (Bemana et al., 2018, Staněk et al., 9 Jun 2026, Ivagnes et al., 2 Mar 2026). A plausible implication is that reference-based training procedures are best understood not as a single architecture class, but as a design principle: the reference supplies privileged structure during optimization or adaptation, and the main research question is how much of that privileged information should remain explicit at inference.