CSD-CLIP: Contrastive Style Similarity

• Style similarity quantifies shared aesthetic attributes independent of content by comparing cosine similarities of learned image embeddings.
• CSD-CLIP employs a CLIP-initialized Vision Transformer and hybrid contrastive objectives on a large, multi-label LAION-Styles dataset for robust style analysis.
• The framework demonstrates state-of-the-art retrieval and attribution performance, advancing applications in style transfer and generative model auditing.

Style similarity quantifies the degree to which two images share global visual characteristics such as color palettes, textures, brushwork, composition, or other factors associated with artistic style, independent of depicted semantic content. In the context of large generative models and diffusion-based image synthesis, style similarity measures—particularly those leveraging learned vision-LLMs—have become essential for tasks such as style retrieval, attribution, transfer, and auditing. CSD-CLIP represents a principled approach to learning such metrics via a contrastive style descriptor (CSD) framework, employing CLIP-initialized Vision Transformers trained on large-scale, multi-label style datasets, combined with self-supervised objectives to ensure invariance and transferability. Complementary CLIP-based methodologies for style disentanglement and transfer, as instantiated in frameworks such as StyleDiffusion, further demonstrate the versatility and empirical effectiveness of these representations.

1. Definitional Framework and Mathematical Formulation

Style similarity in modern vision models is parameterized via a learned style embedding function $f_{\text{style}} : x \to \mathbb{R}^d$, where $x$ denotes an image and $d$ is the embedding dimension. For any pair of images $x, y$, style similarity is defined as the cosine similarity of their embeddings: $$s(x, y) = \cos\left(f_{\text{style}}(x), f_{\text{style}}(y)\right)$$ This formulation is adopted in CSD-CLIP, where $f_{\text{style}}$ is typically implemented as a Vision Transformer backbone initialized from CLIP weights, augmented with a learned projection head. During training, embedding similarity is used both in supervised and self-supervised contrastive learning objectives to model the shared or distinctive style attributes between images. By explicitly decoupling stylistic features from content, such metrics can robustly support retrieval and attribution—even in the presence of dramatic content variation (Somepalli et al., 2024).

2. Dataset Construction and Style Annotation

Central to effective style-similarity modeling is the construction of large, richly annotated style datasets. The LAION-Styles dataset underpins CSD-CLIP training, assembling 511,921 images labeled with 3,840 style tags curated from LAION-Aesthetics based on aesthetic score ($\geq6$), filtered artist/movement/media prompts, and deduplication via SSCD embeddings at a cosine threshold of 0.8. The multi-label nature of these annotations supports fine-grained, disjoint, and overlapping style notions in a highly imbalanced label regime. This comprehensive data foundation distinguishes CSD-CLIP from prior art, which typically lacks multi-artist or multi-movement supervision at such scale and variety (Somepalli et al., 2024).

3. CSD-CLIP Architecture and Training Objectives

CSD-CLIP utilizes two principal architectural variants:

• CSD-ViT-B and CSD-ViT-L, both initialized from their respective CLIP ViT backbones.
• The final embedding is produced from the [CLS] token or via pooling, followed by a small MLP projection.

Training is governed by a hybrid contrastive objective:

• Multi-label Contrastive Loss (MCL): For batch embeddings $\{f_{\text{style}}(x_i)\}$ and binary label vectors $\{l_i\} \in \{0,1\}^L$, the pairwise similarity $s_{i,j}$ is supervised via:

$x$0

• Self-Supervised Contrastive Loss (SSL): Each image is augmented with style-preserving transformations (e.g., flips, rotations), and a SimCLR-style loss is applied to embeddings of paired augmentations:

$x$1

The total objective is $x$2, with $x$3 as optimal. Optimization employs SGD with momentum, learning rates $x$4 (backbone), $x$5 (head), and temperature $x$6. Style-preserving augmentations are carefully restricted to maintain invariance to content while exposing variability in artistic characteristics (Somepalli et al., 2024).

4. Evaluation Protocols and Empirical Efficacy

CSD-CLIP’s efficacy is established on retrieval and attribution benchmarks:

• DomainNet (Painting): $x$7-image gallery, $x$8-image queries, styles as domains.
• WikiArt: $x$9 paintings by $d$0 artists. Retrieval evaluated on $d$1 query images. Metrics include Recall@$d$2 and mAP@$d$3 (for $d$4). CSD-CLIP achieves state-of-the-art results, e.g., on WikiArt mAP@1: CSD ViT-L $d$5 vs. CLIP ViT-L $d$6 and on DomainNet mAP@1: CSD ViT-B $d$7 vs. CLIP ViT-B $d$8. Gains persist across width and depth of the backbone, and consistent benefits emerge for style retrieval, even when content is highly variable (Somepalli et al., 2024). Qualitatively, CSD-CLIP retrieves stylistically congruent neighbors for both real and generated artworks, including for diffusion outputs by artist name, and produces interpretable style spaces for attribution and diversity assessment.

5. Comparative Frameworks and Relationship to Disentanglement Approaches

Alternative CLIP-based paradigms for style similarity emphasize explicit content–style disentanglement. StyleDiffusion employs a style disentanglement loss in CLIP image space, using the difference between an image’s embedding and its content-only counterpart (extracted via a style-removal diffusion process). The style similarity metric is the $d$9 and directional (cosine) distance between the style shift vectors: $x, y$0 This loss enforces that stylized outputs align in both magnitude and direction with the reference style embedding. Additional style reconstruction priors restrict stylization drift. Empirically, this method improves over standard Gram-based measures and achieves superior semantic and perceptual alignment with targeted artistic styles (Wang et al., 2023).

6. Utility in Attribution, Auditing, and Style Transfer

CSD-CLIP establishes a practical style similarity kernel for database search, forensic attribution, and geneological analysis of generated artworks. In applications such as diffusion model auditing, CSD-CLIP enables direct attribution of generated images to training corpus archetypes, exposing fine-grained, content-agnostic stylistic copying. For style transfer, style similarity underpins retrieval of reference styles, transfer model evaluation, and style-consistency diagnostics. MegaStyle (Gao et al., 9 Apr 2026) further demonstrates the value of large-scale, high-diversity datasets for learning such metrics, and corroborates the empirical impact: on StyleRetrieval, CSD-ViT-L achieves mAP@1 of $x, y$1, far exceeding standard CLIP ($x, y$2), with the MegaStyle-Encoder (SoViT) setting state-of-the-art at $x, y$3 mAP@1. This underscores substantial advances realized through explicitly style-supervised contrastive learning.

Method	DomainNet mAP@1	WikiArt mAP@1
CLIP ViT-B/16	73.7	52.2
CSD ViT-B (ours)	78.3	56.2
CLIP ViT-L/14	74.0	59.4
CSD ViT-L (ours)	78.3	64.6

A plausible implication is that robust, scalable style similarity is now feasible across diverse image domains and generative model outputs, enabling new directions in dataset curation, generative model auditing, and empirical analysis of artistic style evolution.

7. Limitations and Prospective Extensions

Challenges remain in CSD-CLIP and related style similarity frameworks. The reliance on noisy caption-derived artist tags in LAION-Styles introduces label errors and false negatives; the style definition is principally bound to artist/movement tags, omitting genre and compositional nuances. Current contrastive frameworks do not explicitly address multi-style mixing as seen in composite prompts, and the scope of applied style-preserving augmentations may limit robustness to certain photometric transformations or compositional shifts. Future work extends toward cleaner, hierarchically structured style supervision, unsupervised segmentation of overlapping style modes, and deeper integration with both text-to-image and content–style disentanglement paradigms (Somepalli et al., 2024, Wang et al., 2023, Gao et al., 9 Apr 2026).