DUDE: Diffusion-Based UCIR

Updated 10 July 2026

DUDE is a diffusion-based unsupervised cross-domain image retrieval method that disentangles object content from domain-specific style.
It leverages a frozen Stable Diffusion backbone and a progressive mutual-neighbor alignment strategy to achieve robust, domain-invariant representations.
Empirical results show DUDE outperforms previous methods on benchmarks like PACS, Office-Home, and DomainNet with significant precision gains.

DUDE, short for Diffusion-Based Unsupervised cross-Domain imagE retrieval, is a method for unsupervised cross-domain image retrieval (UCIR) that addresses retrieval across visual domains such as photo, art painting, cartoon, sketch, clipart, and infograph without labels or paired training data. Its central claim is that UCIR should not align entangled whole-image features directly; instead, it should first separate domain-invariant object content from domain-specific style, and only then align the object features across domains. DUDE operationalizes this idea with a frozen text-to-image diffusion model, specifically Stable Diffusion, and a progressive mutual-neighbor alignment procedure, and reports state-of-the-art performance on PACS, Office-Home, and DomainNet (Yang et al., 4 Sep 2025).

1. Task formulation and motivation

UCIR considers two domains,

$D^A = \{I_i^A\}_{i=1}^n,\quad D^B = \{I_i^B\}_{i=1}^m,$

that differ in visual style but share semantic categories. Given a query image $I_i^A$ from category $C$ , the objective is to retrieve images of the same category $C$ from domain $B$ , with no labels or paired data available during training (Yang et al., 4 Sep 2025).

The difficulty is not merely domain shift in the conventional domain-adaptation sense. In UCIR, the category-relevant object signal is frequently entangled with domain cues such as sketch strokes, infograph text, color palette, or rendering style. Previous UCIR methods, including DD, ProtoOT, CoDA, PCS, and CDS, primarily learn global image representations and align them through instance-level, cluster-level, or prototype-based objectives. DUDE argues that such alignment is often performed on features of the whole image, so the model can over-focus on style similarity rather than object identity (Yang et al., 4 Sep 2025).

This reframes UCIR as a disentangle-then-align problem. The method assumes that the domain word in a text prompt can absorb style information, while a learned token can be forced to encode object semantics. A plausible implication is that cross-domain retrieval improves when alignment is carried out only after this factorization.

2. Diffusion-based object disentanglement

DUDE uses a pre-trained, frozen Stable Diffusion backbone rather than training a generative model from scratch. The trainable component is a feature extractor $f_\theta$ , instantiated as a ResNet-50 pre-trained on ImageNet, which maps an image $I_i^d$ to an embedding $z_i^d = f_\theta(I_i^d)$ . This embedding becomes a learnable object token $[z_i^d]$ inside a prompt template (Yang et al., 4 Sep 2025).

For each domain, DUDE defines a hand-crafted prompt with an explicit style word. On PACS, examples include:

“a painting of a $[z]$ ”
“a cartoon of a $I_i^A$ 0”
“a photo of a $I_i^A$ 1”
“a sketch of a $I_i^A$ 2”

The domain token is fixed per domain and is intended to carry style. The learnable token $I_i^A$ 3, produced by $I_i^A$ 4, is pressured to carry object content. The image is encoded by the frozen VAE $I_i^A$ 5, noise is added at diffusion timestep $I_i^A$ 6,

$I_i^A$ 7

and the frozen UNet noise predictor $I_i^A$ 8 is conditioned on the prompt embedding $I_i^A$ 9. DUDE then optimizes only $C$ 0 through the reconstruction-style objective

$C$ 1

Because Stable Diffusion’s parameters remain frozen, gradients flow only into the feature extractor. The intended effect is functional factorization: the explicit domain word represents “painting / sketch / infograph / photo,” while $C$ 2 must encode the semantic object information needed for denoising. The paper supports this qualitatively by swapping domain words while reusing an extracted object token; the object remains while the style changes (Yang et al., 4 Sep 2025).

A common misunderstanding is that DUDE is a diffusion model for retrieval at inference time. It is not. The diffusion model is a training-time disentangler; the deployed retrieval system uses only the learned discriminative encoder.

3. Progressive mutual-neighbor alignment

After disentanglement, DUDE aligns the object-centric features through a Progressive Alignment Module. The method uses a momentum encoder $C$ 3 updated in MoCo style and domain-specific memory banks $C$ 4 and $C$ 5 to provide stable keys, negatives, and neighbor pools (Yang et al., 4 Sep 2025).

The progression has three stages.

First, DUDE performs instance augmentation alignment. For an image $C$ 6 and its augmentation $C$ 7, it computes

$C$ 8

and applies an InfoNCE-style loss between $C$ 9 and $C$ 0. This initializes augmentation invariance and stabilizes the representation.

Second, DUDE performs in-domain mutual-neighbor alignment. For each sample, it computes top- $C$ 1 neighbors by cosine similarity within the same domain and defines a symmetric adjacency matrix $C$ 2 where a pair is positive only if the two samples are mutual top- $C$ 3 neighbors. This mutuality criterion filters noisy asymmetric matches. In-domain positives are then injected into an InfoNCE objective: $C$ 4

Third, once in-domain structure is more reliable, DUDE adds cross-domain mutual-neighbor alignment. It computes top- $C$ 5 neighbors from $C$ 6 to $C$ 7 and from $C$ 8 to $C$ 9, forms cross-domain adjacency matrices $B$ 0 and $B$ 1, and uses only mutual cross-domain neighbors as positives in cross-domain InfoNCE: $B$ 2

The progression is explicit. After a disentanglement warm-up, DUDE first optimizes

$B$ 3

and then moves to

$B$ 4

The rationale is that early cross-domain alignment on immature features is unstable; mutual neighbors become reliable only after augmentation invariance and within-domain organization are established (Yang et al., 4 Sep 2025).

4. Architecture, optimization, and retrieval inference

At the architectural level, DUDE combines four components: a trainable ResNet-50 feature extractor $B$ 5; frozen Stable Diffusion components including the VAE encoder, UNet noise predictor, and text encoder; a momentum encoder $B$ 6; and domain-specific memory banks (Yang et al., 4 Sep 2025).

The optimization pipeline can be summarized as follows. During warm-up, $B$ 7 is trained with $B$ 8 so that $B$ 9 becomes object-centric under domain-conditioned diffusion denoising. In the first progressive stage, the model adds instance and in-domain mutual-neighbor contrastive objectives. In the second stage, it maintains in-domain structure while introducing cross-domain mutual-neighbor alignment. This ordering is central to the method’s stability claims.

At inference, the diffusion model is discarded. Given a query $f_\theta$ 0 and target-domain database images $f_\theta$ 1, DUDE computes

$f_\theta$ 2

then ranks database images by cosine similarity,

$f_\theta$ 3

Thus the final retrieval system is computationally similar to standard embedding-based retrieval despite the heavier training procedure (Yang et al., 4 Sep 2025).

This separation between training and deployment is operationally important. The method uses a generative prior to shape the representation, but retrieval itself remains a nearest-neighbor problem in the learned feature space.

5. Benchmarks, baselines, and empirical results

DUDE is evaluated on three benchmarks over 13 domains: PACS, Office-Home, and DomainNet, with DomainNet restricted to Clipart, Infograph, Painting, Real, and Sketch, excluding Quickdraw (Yang et al., 4 Sep 2025).

Dataset	Domains	Evaluation
PACS	Photo, Art Painting, Cartoon, Sketch	P@50, P@100, P@200
Office-Home	Art, Clipart, Product, Real	P@1, P@5, P@15
DomainNet	Clipart, Infograph, Painting, Real, Sketch	P@50, P@100, P@200

The comparison set includes ID, ProtoNCE, CDS, PCS, DD, CoDA, and ProtoOT. A retrieved image is counted as correct if it shares the same category label as the query; labels are used only for evaluation (Yang et al., 4 Sep 2025).

On PACS, DUDE substantially improves over ProtoOT in average retrieval precision across all domain directions. The reported averages are 71.09 vs 50.26 at P@50, 67.49 vs 48.78 at P@100, and 61.78 vs 46.38 at P@200, corresponding to gains of +20.83, +18.71, and +15.40 points. On individual directions, the gains can be even larger; for example, Art→Cartoon P@50 reaches 88.21 for DUDE versus 69.97 for ProtoOT (Yang et al., 4 Sep 2025).

On Office-Home, DUDE again improves the average performance over ProtoOT: 52.36 vs 43.85 at P@1, 50.84 vs 41.29 at P@5, and 48.04 vs 37.98 at P@15, i.e., +8.51, +9.55, and +10.06. The paper highlights Product→Real P@1 as 74.97 vs 64.01 (Yang et al., 4 Sep 2025).

On DomainNet, the reported average gains are particularly pronounced: 82.62 vs 66.63 at P@50, 80.07 vs 62.98 at P@100, and 74.90 vs 56.12 at P@200, corresponding to +15.99, +17.09, and +18.78. On difficult pairs such as Clipart→Sketch, ProtoOT achieves 70.46 at P@50 and 67.41 at P@200, whereas DUDE reaches 93.07 and 90.30. For Sketch→Clipart, the corresponding values are 82.79 and 71.31 for ProtoOT versus 95.21 and 91.89 for DUDE (Yang et al., 4 Sep 2025).

The ablations clarify which components drive these gains. In prompt design, a varying prompt such as “a sketch/clipart/painting/... of a [z]” outperforms a fixed prompt such as “a photo of a [z]”; for Clipart→Sketch on DomainNet, P@50 is 75.76 with the varying prompt versus 66.42 with the fixed prompt. This supports the claim that explicit domain words help isolate style from object semantics.

Loss ablations show that disentanglement alone is already strong. A plain ResNet-50 baseline yields average P@50(1)/100(5)/200(15) = 35.13 / 31.03 / 26.66, whereas using only $f_\theta$ 4 lifts this to 50.41 / 46.78 / 41.93. The full combination $f_\theta$ 5 reaches 68.69 / 66.23 / 61.57, the best overall configuration (Yang et al., 4 Sep 2025).

6. Interpretation, limitations, and nomenclature

The most direct interpretation of DUDE is that it treats domain shift in UCIR as partly a representation factorization problem rather than only an alignment problem. By forcing style into an explicit domain token and object semantics into $f_\theta$ 6, it constructs features that are more amenable to neighbor-based cross-domain alignment. The reported t-SNE observations are consistent with this: DUDE produces more compact class clusters and less domain separation than ProtoOT. This suggests that semantic neighborhoods become more reliable once style contamination is reduced (Yang et al., 4 Sep 2025).

The method also has evident limitations. Training is heavier than purely discriminative UCIR because Stable Diffusion’s VAE and UNet remain in the loop, even though they are frozen. Performance depends on manually chosen domain-style words such as “clipart,” “painting,” or “infograph.” The paper also notes sensitivity to hyperparameters such as neighborhood size $f_\theta$ 7 and the coefficients $f_\theta$ 8 and $f_\theta$ 9, although it describes the method as reasonably robust. A further plausible concern, already implied in the paper, is dependence on Stable Diffusion’s semantic prior: if the pre-trained model does not capture a specialized domain well, the disentanglement mechanism may be less effective.

The proposed extensions are correspondingly natural: replacing Stable Diffusion with newer or domain-specific diffusion models, extending from pairwise to multi-domain alignment, and transferring the disentangled representation to other cross-domain tasks such as classification, segmentation, or domain adaptation.

The acronym itself is overloaded across fields. In arXiv literature, “DUDE” can also denote Downlink/Uplink Decoupling in cellular networks, Discrete Universal DEnoiser and its neural or sequencing variants, Deep Unsigned Distance Embeddings, Document Understanding Dataset and Evaluation, Dual Distribution-Aware Context Prompt Learning, and Dual Decomposition of Weights and Singular Value Low Rank Adaptation (Boccardi et al., 2015, Lee et al., 2015, Moon et al., 2016, Venkatesh et al., 2020, Landeghem et al., 2023, Nguyen et al., 2024, Han et al., 20 May 2025). In the computer-vision retrieval context, however, DUDE specifically names a diffusion-based UCIR method whose defining contribution is the combination of diffusion-driven object-style disentanglement with progressive mutual-neighbor alignment.