Reference Image Survey (RIS): Methods & Applications

• RIS is a term encompassing multiple methods where the reference may serve as a linguistic cue, style exemplar, or photometric baseline in diverse applications.
• In referring image segmentation, techniques like HARIS use human-like attention and parameter-efficient fine-tuning to enhance mask accuracy and filter spurious associations.
• In difference imaging and radar, RIS methods tackle PSF matching and synthetic aperture challenges, emphasizing the importance of tailored reference selection strategies.

RIS is an overloaded acronym in current arXiv literature rather than a single technical doctrine. In computer vision, it commonly denotes Referring Image Segmentation, where a free-form linguistic expression specifies the image region to be segmented. In generative modeling, closely related work studies Reference-guided Image Synthesis, where a reference image supplies the target style and motivates single-image quality assessment. In astronomical Difference Image Analysis, the central technical problem is the selection of a reference image against which target images are PSF-matched and subtracted. A separate radar-imaging literature uses RIS to mean Reconfigurable Intelligent Surface, which is terminologically distinct but also participates in image formation (Zhang et al., 2024, Guo et al., 2021, Huckvale et al., 2014, Ilgac et al., 2024).

1. Scope of the acronym and the role of the reference

Across these literatures, the reference acts as an anchor for correspondence, evaluation, or subtraction, but its operational meaning differs. In Referring Image Segmentation, the conditioning signal is linguistic rather than another image. In Reference-guided Image Synthesis Assessment, the reference is an exemplar whose style must be matched by a generated image. In Difference Image Analysis, the reference is the photometric and geometric baseline used in every subtraction. In radar, the acronym is unrelated to reference images and instead denotes a reconfigurable reflecting aperture.

Usage	Conditioning or anchor	Primary output
Referring Image Segmentation	image $I$ and language expression $T$	binary mask $M$
Reference-guided Image Synthesis Assessment	reference image $I_r$ and generated image $I_g$	scalar score $s(I_r,I_g)\in[0,1]$
Difference Image Analysis	reference image $R$ in a time series	difference images and calibrated lightcurves
Reconfigurable Intelligent Surface imaging	RIS phase settings and scene reflectivity	reconstructed radar image

This division matters because the same abbreviation can obscure sharply different mathematical objects. A common misconception is that RIS always refers to a segmentation task; the supplied arXiv record shows at least three reference-centered usages and one unrelated radar usage.

2. Referring Image Segmentation

Referring Image Segmentation takes as input an image $I\in\mathbb{R}^{H\times W\times 3}$ and a free-form language expression $T$, and returns a binary mask $M\in\{0,1\}^{H\times W}$ that segments exactly the region referred to by $T$0 (Zhang et al., 2024). The HARIS formulation begins from the observation that conventional RIS systems perform one-shot cross-attention between all visual tokens and all word tokens in a bottom-up manner. Given visual features $T$1 and linguistic features $T$2, the standard pairing computes

$T$3

The stated failure mode is that a word such as “black” can be paired with every black-colored region in the image, producing spurious associations.

HARIS addresses this with a Human-Like Attention (HLA) block and a parameter-efficient fine-tuning (PEFT) framework. The visual side uses a frozen image encoder, such as a SAM-backbone, producing multi-scale features $T$4, each flattened to $T$5. The linguistic side uses a frozen CLIP text encoder to produce per-word embeddings $T$6 and a global sentence embedding $T$7. Inside each HLA block, the first round of bidirectional cross-attention computes

$T$8

A feedback signal is then generated by re-evaluating fused vision tokens against the global sentence embedding: $T$9 The feedback is re-injected into the visual tokens,

$M$0

and a second-round language-weighted attention yields the final language-aware visual tokens

$M$1

The paper describes this as a top-down refinement that centers the network on the referent and discards irrelevant image-text pairs.

After HLA processing at each scale, HARIS performs hierarchical fusion according to

$M$2

where CBA is $M$3. The mask decoder concatenates a learnable query token $M$4 with $M$5, uses them as queries to a standard Transformer Decoder with $M$6, and produces $M$7. Two upsampling blocks of Conv+BN+Upsample then multiply with $M$8 to generate the final mask $M$9.

The PEFT design freezes the vision and text encoders and trains only the HLA blocks, fusion convolutions, query token $I_r$0, and mask decoder. The reported rationale is that pre-trained parameters $I_r$1 remain unchanged while a small additional subset $I_r$2 is learned, preserving zero-shot capabilities. Training minimizes

$I_r$3

where $I_r$4 is focal loss and $I_r$5 is Dice loss. The implementation details specify Adam with initial learning rate $I_r$6, batch size $I_r$7, 4× V100 GPUs, decay by $I_r$8 at epoch 30, total training length 50 epochs, maximum sentence length 17 for RefCOCO/RefCOCO+ and 25 for G-Ref, and standard random flip and resize inherited from the SAM backbone.

Quantitatively, HARIS reports $I_r$9 IoU on RefCOCO val/testA/testB, $I_g$0 on RefCOCO+ val/testA/testB, and $I_g$1 on G-Ref val U/test U/val G. On PhraseCut zero-shot transfer, the reported IoU is $I_g$2 when trained on RefCOCO, $I_g$3 when trained on RefCOCO+, and $I_g$4 when trained on G-Ref, compared with CRIS at $I_g$5 and LAVT at $I_g$6. Ablation further attributes the main gains to the HLA design: removing the feedback branch reduces IoU to $I_g$7, removing the vision-weighted branch to $I_g$8, and removing the language-weighted branch to $I_g$9. The hierarchical structure and the mask decoder also contribute materially, with the full HARIS model reaching $s(I_r,I_g)\in[0,1]$0 IoU on RefCOCO val, versus $s(I_r,I_g)\in[0,1]$1 without hierarchical structure and $s(I_r,I_g)\in[0,1]$2 with a simplified mask decoder. These results position the feedback-enhanced language-weighted pathway as the dominant component.

3. Reference-guided Image Synthesis assessment

Reference-guided Image Synthesis differs from segmentation in that the model renders a source image in the style of another reference image, and the central evaluation problem is often the quality of a single generated sample rather than the fidelity of a generated distribution (Guo et al., 2021). The RISA framework was proposed precisely because distribution-level GAN metrics such as Inception Score, Fréchet Inception Distance, Kernel MMD, and Wasserstein distance cannot determine whether one synthesized image correctly mimics the style of its reference.

RISA receives a reference image $s(I_r,I_g)\in[0,1]$3 and a generated image $s(I_r,I_g)\in[0,1]$4, and outputs a scalar score

$s(I_r,I_g)\in[0,1]$5

intended to reflect both texture fidelity and style-consistency relative to $s(I_r,I_g)\in[0,1]$6. Its architecture has two components. A style encoder $s(I_r,I_g)\in[0,1]$7 maps each image to a style code $s(I_r,I_g)\in[0,1]$8. A set of $s(I_r,I_g)\in[0,1]$9 binary classifiers $R$0 receives the absolute style difference

$R$1

and predicts whether the latent quality exceeds thresholds $R$2. The final score is

$R$3

The weak supervision is derived from intermediate generator checkpoints. The reported empirical observation is that image quality improves, on average, with GAN training iterations. RISA therefore assigns coarse pseudo-labels from the iteration count $R$4, stretched only up to $R$5 because the maximum score $R$6 is reserved for real images introduced via style-preserving augmentations. Because this supervision is too coarse, the method performs pixel-wise interpolation between a low-quality image from the elbow-stage checkpoint and a high-quality image from the final checkpoint: $R$7

$R$8

This creates intermediate triplets whose local textures gradually improve while content and global style remain anchored.

Instead of direct regression, RISA uses $R$9 binary decisions with targets

$I\in\mathbb{R}^{H\times W\times 3}$0

optimized by aggregate binary cross-entropy

$I\in\mathbb{R}^{H\times W\times 3}$1

To make the score style-aware, the method introduces an unsupervised contrastive objective. Two style-preserving augmentations of the reference, $I\in\mathbb{R}^{H\times W\times 3}$2 and $I\in\mathbb{R}^{H\times W\times 3}$3, are paired with the same generated image, and the positive term is

$I\in\mathbb{R}^{H\times W\times 3}$4

For mismatched pairs, a negative reference $I\in\mathbb{R}^{H\times W\times 3}$5 is sampled and the loss

$I\in\mathbb{R}^{H\times W\times 3}$6

is applied with $I\in\mathbb{R}^{H\times W\times 3}$7. A supremum term additionally requires two augmentations of the same real image to receive the maximum score: $I\in\mathbb{R}^{H\times W\times 3}$8 The full training objective is

$I\in\mathbb{R}^{H\times W\times 3}$9

with $T$0.

The reported setup spans Yosemite, CelebA-HQ, AFHQ, LSUN Church, and LSUN Bedroom at $T$1, with training data constructed from DRIT, MSGAN, StarGAN v2, and Swap Autoencoder checkpoints. For each snapshot, 1,000 examples are generated; interpolation between the last two snapshots adds 1,000 examples for each $T$2, yielding 16,000 training triplets per dataset. Training uses one NVIDIA RTX 2080Ti, Adam with $T$3, $T$4, learning rate $T$5, weight decay $T$6, batch size 4, for 100 epochs.

In forced-choice human preference tests, RISA achieves roughly $T$7–$T$8 agreement in intra-model settings, with chance at $T$9, and exceeds the best competing metrics such as SIFID or LPIPS, reported at approximately $M\in\{0,1\}^{H\times W}$0–$M\in\{0,1\}^{H\times W}$1. In cross-model settings it still leads by $M\in\{0,1\}^{H\times W}$2–$M\in\{0,1\}^{H\times W}$3. The ablation study attributes about $M\in\{0,1\}^{H\times W}$4 improvement to pixel-wise interpolation over using only stable-stage snapshots, and a further $M\in\{0,1\}^{H\times W}$5–$M\in\{0,1\}^{H\times W}$6 each to contrastive and supremum losses. The stated limitation is that RISA focuses on style fidelity and largely ignores content alignment, justified by the observation that content is already well preserved early in RIS training.

4. Reference-image selection in Difference Image Analysis

In astronomical Difference Image Analysis, the reference image is not a conditioning exemplar but the common photometric baseline for an entire time series (Huckvale et al., 2014). A reference image $M\in\{0,1\}^{H\times W}$7 or stacked reference is selected, all target images $M\in\{0,1\}^{H\times W}$8 are aligned to its pixel grid, and a kernel $M\in\{0,1\}^{H\times W}$9 is estimated so that one image matches the PSF of the other before subtraction. Errors in $T$00’s signal-to-noise and in PSF matching propagate directly into lightcurve quality, which makes automated reference-image selection a practical necessity in crowded-field photometry.

The standard formulation is Reference-Image Convolution (RIC): $T$01 with $T$02 obtained by minimizing the squared residuals between $T$03 and $T$04. The reversed formulation is Target-Image Convolution (TIC): $T$05 so that target images are convolved to a worse-seeing reference. Difference-image quality is summarized by the variance-weighted reduced $T$06 over stamp regions, with ISIS returning both $T$07 and $T$08. Good subtractions satisfy $T$09 and small $T$10.

The empirical comparison uses OGLE-III optical data and VVV near-infrared data. OGLE-III has $T$11pixel on a 2048×4096 SITe CCD, 24 epochs, and seeing $T$12–$T$13 pixels. VVV has $T$14pixel on a 2048×2048 VIRGO array, 27 epochs, and seeing $T$15–$T$16 pixels. For OGLE under RIC, $T$17 increases sharply when $T$18, the deconvolution regime, and is minimized for $T$19, the convolution regime. For VVV under RIC, the dependence on seeing difference is weak and much noisier, indicating difficulty when either image is under-sampled. Background flux is reported to have negligible impact on $T$20 in both surveys.

The paper evaluates two figure-of-merit strategies. A Universal Figure-of-Merit (UFoM) combines PSF simulation heuristics—signal-to-noise $T$21, centroid error $T$22, width accuracy $T$23, and width precision $T$24—through a power-law ansatz

$T$25

For well-sampled OGLE-RIC, the fit has $T$26; for OGLE-TIC, $T$27; for VVV-RIC and VVV-TIC, $T$28. The conclusion is that the UFoM is only a weak predictor for well-sampled data and fails for under-sampled data.

A Survey-Specific Figure-of-Merit (SFoM) performs better: $T$29 For OGLE, the fitted exponents are $T$30, $T$31, $T$32 under RIC, and $T$33, $T$34, $T$35 under TIC. For VVV, RIC has essentially no predictive power, with $T$36, $T$37, $T$38, while TIC performs substantially better, with $T$39, $T$40, $T$41.

Survey regime	Empirical best procedure	Key fitted evidence
OGLE, well-sampled	RIC with smallest seeing $T$42	$T$43
VVV, under-sampled	TIC with largest seeing $T$44	$T$45

The practical threshold is PSF sampling $T$46. If $T$47, the recommendation is RIC with the smallest $T$48. If $T$49, the recommendation is TIC with the largest $T$50 that still satisfies $T$51 if possible; if all images are under-sampled, the largest $T$52 is preferred anyway. For under-sampled data, the paper further recommends stacking convolved targets onto the reference: $T$53 The important correction to common practice is that “best seeing” is not universally optimal. For under-sampled surveys, the worse-seeing image can be the better reference because it is better sampled.

5. Shared technical patterns across reference-centered methods

Despite the heterogeneity of these literatures, several concrete structural patterns recur. In HARIS, the reference signal enters as language and is first fused locally, then filtered by global sentence feedback before mask decoding. In RISA, the reference enters as a style exemplar and the core comparison variable is the absolute style difference $T$54. In DIA, the reference is the common baseline against which every target is PSF-matched and subtracted, with performance controlled mainly by seeing and sampling rather than by background in the tested dynamic range (Zhang et al., 2024, Guo et al., 2021, Huckvale et al., 2014).

A plausible unifying implication is that reference-conditioned systems are most effective when the reference is injected at the scale relevant to the ambiguity being resolved. HARIS uses global sentence embedding $T$55 to prune local image-text pairings. RISA trains its score on intermediate generator states because the ambiguity lies in fine-grained style quality at the level of a single sample. DIA reduces the selection problem to PSF sampling because the ambiguity is not semantic but optical and numerical. This suggests that the technical burden of “using a reference” varies from multimodal token disambiguation, to pairwise style metric learning, to stable kernel estimation.

The same comparison also clarifies what these methods are not. HARIS is not an image-to-image synthesis method; its output is a binary mask. RISA is not a segmentation framework; its output is a scalar quality score. DIA reference selection is not a downstream recognition problem; it is a preprocessing and calibration problem whose failure corrupts all subsequent photometry. Treating these as a single field under the RIS acronym obscures their different objective functions, supervision signals, and error models.

6. Terminological collision with reconfigurable intelligent surface imaging

A separate line of work uses RIS to mean Reconfigurable Intelligent Surface, not reference image. In “RIS-Aided Radar Imaging Utilizing the Virtual Source Principle,” the problem is single-antenna radar imaging with a reconfigurable intelligent surface, and configuring RIS phase shifts is interpreted as synthetic aperture radar with a moving virtual source (Ilgac et al., 2024). The paper formulates a wavenumber-domain forward model under Born scattering: $T$56 with radar location $T$57, RIS element positions $T$58, adjustable phase shifts $T$59, and scene reflectivity $T$60. The complete baseband received signal is

$T$61

where $T$62 is the Green’s function from RIS element $T$63 to target point $T$64 and back to the radar, including free-space spreading loss.

In the far field, planar-wave approximation yields a wavenumber-domain expression in which the RIS aperture contributes a weighted discrete Fourier sum $T$65. By choosing linear phase ramps

$T$66

the RIS steers a reflected beam into direction $T$67, mimicking a moving transmitter at

$T$68

which traces a circular synthetic aperture around the RIS center. Sweeping $T$69 from $T$70 to $T$71 synthesizes a continuous SAR trajectory in the angular domain.

The paper then distinguishes far- and near-field inversion. For far-field imaging, it defines

$T$72

and reconstructs by matched-filter or $T$73–$T$74 backprojection: $T$75 For near-field imaging, it directly inverts the exact model by delaying and summing along range contours: $T$76 with a Fresnel-zone approximation used in practice to accelerate computation.

The reported performance analysis follows classical SAR laws but with RIS-specific constraints. Range resolution is

$T$77

cross-range resolution from an angular span $T$78 is

$T$79

and the physical-aperture Rayleigh limit is

$T$80

Under far-field matched filtering, the PSF is approximated by

$T$81

SNR scales as $T$82 in the coherent sum over $T$83 RIS elements; element spacing $T$84 causes grating lobes; and phase quantization to $T$85 bits introduces error $T$86, raising sidelobes by approximately $T$87. The design constraints are correspondingly stated as $T$88, phase-reconfiguration speed $T$89, and $T$90 to keep sidelobes below $T$91 dB.

This radar usage is conceptually unrelated to reference-image methods, but it is important because it demonstrates how the same acronym can map to entirely different forward models and inverse problems. In one literature, RIS resolves semantic or stylistic ambiguity through a reference signal; in the other, RIS is the programmable aperture that makes the measurement operator itself reconfigurable.