RefSTAR: Reference Selection, Transfer & Reconstruction

• The paper introduces RefSTAR as a novel, three-stage framework that decomposes blind facial restoration into explicit reference selection, feature transfer via Dual-Stream Cross-Attention, and mask-aware reconstruction.
• RefSTAR is defined by its use of a dedicated RefSel module for refining reference masks, integrated with advanced loss functions and optimization strategies to ensure identity preservation and high perceptual quality.
• Extensive experiments on benchmarks like Celeb-Ref-Test and RealRef60 demonstrate state-of-the-art performance, validating its applicability across diverse domains such as medical imaging and astronomical calibration.

Reference Selection, Transfer, and Reconstruction, abbreviated RefSTAR, denotes a blind facial image restoration method that organizes reference-guided restoration into three coupled operations: selecting usable regions from a high-quality reference face, transferring reference features into a restoration backbone, and reconstructing an output that remains faithful both to the degraded input’s latent clean image and to compatible textures in the reference (Yin et al., 14 Jul 2025). In the exact nomenclature of the literature, the acronym belongs to the facial restoration framework of Yin et al.; however, closely related selection–transfer–reconstruction pipelines also appear in medical volume colorization, MRI reconstruction, and Euclid redshift-distribution calibration, even when the term itself is absent or only used descriptively (Devkota et al., 2022, Guo et al., 2021, Kang et al., 5 Jan 2026).

1. Conceptual scope and problem setting

In the facial restoration formulation, the degraded input is a low-quality facial image $I_{\mathrm{LQ}\in\mathbb R^{H\times W\times 3}}$, produced by an unknown blind degradation operator $D(\cdot)$ acting on a clean ground-truth face $I$. The auxiliary input is a high-quality reference image $I^{\mathrm{Ref}\in\mathbb R^{H\times W\times 3}}$ of the same subject, potentially differing in pose, expression, or lighting. The target output is a restored image $I_{\mathrm{Out}\in\mathbb R^{H\times W\times 3}}$ that approximates $I$, suppresses artifacts introduced by $D$, and incorporates personalized textures from $I^{\mathrm{Ref}}$ where the reference and ground truth are consistent. The objective is written as a restoration function $$f_{\mathrm{rest}}:(I_{\mathrm{LQ}},I^{\mathrm{Ref}})\mapsto I_{\mathrm{Out}}$$ with the constraints that $I_{\mathrm{Out}}\approx I$ and that $D(\cdot)$0 preserves identity and details from $D(\cdot)$1 in compatible regions (Yin et al., 14 Jul 2025).

A central point in this formulation is that high-quality references are not uniformly beneficial. Existing reference-guided restoration methods are described as struggling with identity preservation problems because of improper feature introduction on detailed textures. RefSTAR addresses that issue by decomposing reference use into explicit subproblems: determining which reference regions are actually consistent, ensuring that transferred reference features are not ignored by the backbone, and verifying through reconstruction that the selected reference content is present in the final image rather than being overwhelmed by conventional fidelity losses (Yin et al., 14 Jul 2025).

A common misconception is to treat RefSTAR as a generic label for any reference-based restoration procedure. The literature presented here does not support that usage. The 2022 medical volume-rendering paper does not use the name in its text, and the MRI and Euclid works instantiate structurally similar reference-guided pipelines under different methodological vocabularies, such as a Texture Transformer Module or a deep-to-wide transfer function rather than RefSTAR proper (Devkota et al., 2022, Guo et al., 2021, Kang et al., 5 Jan 2026).

2. Reference selection through RefSel and mask supervision

The reference-selection component of RefSTAR is the RefSel module, whose purpose is to predict a binary mask $D(\cdot)$2 indicating which pixels or facial regions in the reference image should be used for texture transfer. The training data for this module are organized as RefSel-HQ. The construction begins from approximately $D(\cdot)$3 pairs of high-quality facial images $D(\cdot)$4 drawn from CelebRef-HQ and other sources. For a held-out subset of 800 pairs, binary segmentation masks are manually annotated so that only regions whose textures in $D(\cdot)$5 and $D(\cdot)$6 are truly consistent are retained; the examples given include mouth-open versus mouth-closed, eye-open versus eye-closed, conflicting freckles, and occluding glasses. A U-Net is trained on these 800 pairs to predict consistency masks, its predictions are run on the remaining 9,200 pairs, and those outputs are manually filtered or refined, yielding 10,000 triplets $D(\cdot)$7 with high-quality ground-truth consistency masks (Yin et al., 14 Jul 2025).

To generate degraded training inputs, each clean $D(\cdot)$8 is processed by the Real-ESRGAN pipeline, including motion and defocus blur, to produce $D(\cdot)$9. The paper further states that when degradation severity exceeds a threshold and $I$0 becomes extremely blurred, $I$1 is replaced by a full-face mask in order to avoid spurious mask errors. The RefSel network itself is a U-Net,

$I$2

which takes the two images concatenated in the channel dimension and outputs per-pixel probabilities $I$3. Binarization is performed by $I$4. Training uses OHEM cross-entropy,

$I$5

where $I$6 is the ground-truth mask and $I$7 contains the hardest pixels, selected by a threshold $I$8 on individual cross-entropy loss values so that optimization concentrates on boundary and conflict regions (Yin et al., 14 Jul 2025).

This design makes the selection step explicit rather than implicit. A plausible implication is that RefSTAR treats reference compatibility as a structured prediction problem in its own right rather than as a side effect of feature matching.

3. Feature transfer and the avoidance of trivial cross-attention

RefSTAR’s transfer stage is motivated by a failure mode in vanilla cross-attention. Let $I$9 denote the query, key, and value tensors of the degraded input stream, and let $I^{\mathrm{Ref}\in\mathbb R^{H\times W\times 3}}$0 denote the key and value tensors of the reference stream. A straightforward fusion strategy is

$I^{\mathrm{Ref}\in\mathbb R^{H\times W\times 3}}$1

with $I^{\mathrm{Ref}\in\mathbb R^{H\times W\times 3}}$2 denoting softmax. The paper argues that this arrangement admits a trivial solution because $I^{\mathrm{Ref}\in\mathbb R^{H\times W\times 3}}$3 can be driven to a matrix of $I^{\mathrm{Ref}\in\mathbb R^{H\times W\times 3}}$4 logits, which causes the model to ignore $I^{\mathrm{Ref}\in\mathbb R^{H\times W\times 3}}$5 altogether (Yin et al., 14 Jul 2025).

To prevent that collapse, the method introduces Dual-Stream Cross-Attention (DSCA). Instead of one joint attention over concatenated keys and values, the model computes parallel self-stream and reference-stream terms and sums them: $I^{\mathrm{Ref}\in\mathbb R^{H\times W\times 3}}$6 This DSCA layer is inserted into each transformer block of the one-step diffusion backbone built upon Arc2Face. The stated purpose is to force both streams to contribute, so that even when $I^{\mathrm{Ref}\in\mathbb R^{H\times W\times 3}}$7 is initially weak, the $I^{\mathrm{Ref}\in\mathbb R^{H\times W\times 3}}$8 branch still contributes nontrivially and the network can learn a meaningful alignment (Yin et al., 14 Jul 2025).

Within RefSTAR, therefore, “transfer” is not merely the existence of an attention pathway from reference to target. It is a constrained fusion mechanism designed to make reference information unavoidable during optimization.

4. Reconstruction, cycle consistency, and optimization

RefSTAR’s reconstruction stage is organized around the claim that supervision of $I^{\mathrm{Ref}\in\mathbb R^{H\times W\times 3}}$9 against the ground truth $I_{\mathrm{Out}\in\mathbb R^{H\times W\times 3}}$0 is insufficient to guarantee that textures from $I_{\mathrm{Out}\in\mathbb R^{H\times W\times 3}}$1 actually appear in the output. The method therefore adds a reference image reconstruction mechanism with a mask-compatible cycle. The cycle proceeds in three steps: first, the model restores $I_{\mathrm{Out}\in\mathbb R^{H\times W\times 3}}$2; second, the reference image $I_{\mathrm{Out}\in\mathbb R^{H\times W\times 3}}$3 is degraded by the same operator $I_{\mathrm{Out}\in\mathbb R^{H\times W\times 3}}$4 to produce $I_{\mathrm{Out}\in\mathbb R^{H\times W\times 3}}$5, and $I_{\mathrm{Out}\in\mathbb R^{H\times W\times 3}}$6 is then used as the reference to restore $I_{\mathrm{Out}\in\mathbb R^{H\times W\times 3}}$7; third, $I_{\mathrm{Out}\in\mathbb R^{H\times W\times 3}}$8 is constrained to match the original $I_{\mathrm{Out}\in\mathbb R^{H\times W\times 3}}$9 only in mask-selected regions $I$0 (Yin et al., 14 Jul 2025).

The reconstruction loss on the final output is

$I$1

where the constituent terms are pixel-wise, perceptual, ArcFace identity, and GAN losses. The mask-compatible cycle term is

$I$2

with $I$3 described as a mixture of $I$4, identity, and perceptual terms. The full objective is

$I$5

and the example weights reported are $I$6. The RefSel module is trained separately or jointly under $I$7, while the restoration model is trained end-to-end, phase-wise for the diffusion backbone (Yin et al., 14 Jul 2025).

The reconstruction stage thus serves a dual role: it is both the image synthesis component and the mechanism by which reference transfer is audited during training.

5. Benchmarks, ablations, and observed behavior

On the synthetic Celeb-Ref-Test benchmark at $I$8, RefSTAR is evaluated with PSNR, LPIPS, FID, ID-GT, ID-Ref, and MUSIQ; on the real-world RealRef60 benchmark, where no ground truth is available for PSNR or LPIPS, evaluation uses FID, ID-Ref, and MUSIQ. The reported results are state-of-the-art and are presented as evidence of better identity preservation ability and reference feature transfer quality (Yin et al., 14 Jul 2025).

Benchmark	RefSTAR	Next comparison
Celeb-Ref-Test	PSNR 24.69, LPIPS 0.335, FID 21.01, ID-GT 82.76%, ID-Ref 64.53%, MUSIQ 73.91	RefLDM: LPIPS 0.387, FID 22.90, ID-Ref 53.31
RealRef60	FID 155.18, ID-Ref 56.44, MUSIQ 72.75	RefLDM: FID 155.73, ID-Ref 46.70
RefSel accuracy	Average mask-prediction accuracy 0.88 across 5 conflict scenarios	—

The ablation studies identify each stage as functionally important. Removing RefSel by using an all-ones mask reduces ID-GT from 82.76 to 81.86; using an all-zeros mask lowers ID-GT to 77.70. Removing DSCA leads to “no effective reference infusion,” increasing FID from 21.01 to 27.84 and reducing ID-GT to 74.78. Removing the cycle loss causes texture bleed and lowers ID-Ref from 64.53 to 60.70. Taken together, these results are interpreted in the paper as confirmation that explicit consistency-region selection, forced feature infusion, and mask-aware cycle consistency each contribute critically to high-fidelity, identity-preserving, reference-guided blind face restoration (Yin et al., 14 Jul 2025).

6. Related selection–transfer–reconstruction pipelines beyond facial restoration

The broader methodological pattern of selecting references, transferring structured information, and reconstructing a target appears in several distinct research areas. In reference-based color transfer for medical volume rendering, Devkota et al. begin with a stack of monochrome CT or MRI slices and a database of candidate full-color cryosection images $I$9. Reference selection is performed by ranking candidates using cosine similarity in the 4,096-dimensional fc6 feature space of gray-VGG-19. Color transfer then uses Deep Image Analogy with PatchMatch-based nearest-neighbor fields $D$0 and $D$1, followed by Fast Global Smoother guidance to remove unwanted texture and geometry transfer while preserving chroma: $D$2 The resulting $D$3 slices are stacked and volume-rendered, with voxel opacity set by normalized luminance $D$4 and color set by $D$5, so that no manual transfer-function editing is required. That paper reports purely qualitative evaluation and explicitly states that it did not report PSNR, SSIM, or CIEDE2000 metrics. It also notes that most of the correspondence machinery is borrowed from Liao et al.’s Deep Image Analogy rather than being introduced as a new loss or architecture (Devkota et al., 2022).

In accelerated MRI reconstruction, the Texture Transformer Module of “Reference-based Magnetic Resonance Image Reconstruction Using Texture Transformer” operationalizes a comparable logic. The under-sampled image $D$6, an under-sampled reference $D$7, and its fully sampled counterpart $D$8 are mapped by a four-block convolutional feature extractor into $D$9, $I^{\mathrm{Ref}}$0, and $I^{\mathrm{Ref}}$1. Query and key maps are unfolded into non-overlapping $I^{\mathrm{Ref}}$2 patches; normalized inner products $I^{\mathrm{Ref}}$3 define hard attention via $I^{\mathrm{Ref}}$4 and soft attention via $I^{\mathrm{Ref}}$5. The transferred patches are stitched back and fused as

$I^{\mathrm{Ref}}$6

The module is trained end-to-end with a pixel-wise $I^{\mathrm{Ref}}$7 loss and improves multiple backbones on IXI $I^{\mathrm{Ref}}$8-weighted brain MRI at $I^{\mathrm{Ref}}$9 acceleration: U-net gains $$f_{\mathrm{rest}}:(I_{\mathrm{LQ}},I^{\mathrm{Ref}})\mapsto I_{\mathrm{Out}}$$0 dB PSNR and $$f_{\mathrm{rest}}:(I_{\mathrm{LQ}},I^{\mathrm{Ref}})\mapsto I_{\mathrm{Out}}$$1 SSIM, KIKI gains $$f_{\mathrm{rest}}:(I_{\mathrm{LQ}},I^{\mathrm{Ref}})\mapsto I_{\mathrm{Out}}$$2 dB and $$f_{\mathrm{rest}}:(I_{\mathrm{LQ}},I^{\mathrm{Ref}})\mapsto I_{\mathrm{Out}}$$3 SSIM, D5C5 gains $$f_{\mathrm{rest}}:(I_{\mathrm{LQ}},I^{\mathrm{Ref}})\mapsto I_{\mathrm{Out}}$$4 dB and $$f_{\mathrm{rest}}:(I_{\mathrm{LQ}},I^{\mathrm{Ref}})\mapsto I_{\mathrm{Out}}$$5 SSIM, and PC-RNN gains $$f_{\mathrm{rest}}:(I_{\mathrm{LQ}},I^{\mathrm{Ref}})\mapsto I_{\mathrm{Out}}$$6 dB and $$f_{\mathrm{rest}}:(I_{\mathrm{LQ}},I^{\mathrm{Ref}})\mapsto I_{\mathrm{Out}}$$7 SSIM (Guo et al., 2021).

A structurally analogous pattern is also present in Euclid redshift-distribution calibration. Deep Euclid Auxiliary Fields with spectroscopic redshifts are selected under explicit photometric and shear-quality cuts. A deep-to-wide transfer function then degrades deep photometry into wide-like photometry, either independently per band or jointly through Multi-Passband Transfer, where one wide-sample neighbour is drawn with probability proportional to a multi-band likelihood $$f_{\mathrm{rest}}:(I_{\mathrm{LQ}},I^{\mathrm{Ref}})\mapsto I_{\mathrm{Out}}$$8, its uncertainty vector is applied to the deep object, and degraded fluxes are sampled accordingly. A $$f_{\mathrm{rest}}:(I_{\mathrm{LQ}},I^{\mathrm{Ref}})\mapsto I_{\mathrm{Out}}$$9-d tree with $I_{\mathrm{Out}}\approx I$0 nearest neighbours accelerates the procedure. Reconstruction is then carried out through a $I_{\mathrm{Out}}\approx I$1 self-organising map trained on flux-ratio features, with the wide-sample redshift distribution recovered by

$I_{\mathrm{Out}}\approx I$2

The paper reports that Multi-Passband Transfer outperforms Balrog in Wasserstein-distance comparisons and that Scenario G is the only calibration scenario with greater than 30% success in every tomographic bin for meeting the Euclid weak-lensing mean-redshift requirement $I_{\mathrm{Out}}\approx I$3 (Kang et al., 5 Jan 2026).

Taken together, these works suggest that RefSTAR is best understood at two levels. In the narrow sense, it is the specific blind facial image restoration architecture of Yin et al. In a broader methodological sense, it names a recurring research strategy in which reference data are first filtered or ranked, then selectively transferred through attention, correspondence, or probabilistic matching, and finally converted into a reconstructed target representation whose fidelity depends on the quality of both the reference selection and the transfer mechanism.