Visual Art Recommender Systems

• Visual Art Recommender Systems are multimodal engines that fuse visual cues, semantic insights, and contextual metadata to curate personalized artwork suggestions.
• They employ diverse methodologies including sequential modeling, content-based filtering, and multimodal fusion to tackle challenges like subjectivity and cold-start issues.
• Applications span digital art communities, museum collections, creative curation, stakeholder design, and therapeutic contexts, demonstrating broad impact in art recommendation.

Searching arXiv for recent and relevant papers on visual art recommender systems to ground the article. {"query":"visual art recommender systems art recommendation multimodal museum arXiv", "max_results": 10} {"query":"visual art recommender systems art recommendation multimodal museum site:arxiv.org", "max_results": 10, "source": "arxiv"} Visual Art Recommender Systems (VA RecSys) are recommender systems for artworks whose core signals are visual appearance, art semantics, creator identity, contextual metadata, temporal behavior, or affective response rather than only transactional co-occurrence. Across the literature, the term covers markedly different regimes: next-item prediction in digital art communities, content-based recommendation for museum and gallery collections, style reranking for creative professionals, multistakeholder curation, and therapeutic recommendation for clinical support. A common theme is that artworks are highly subjective objects whose relevance depends on factors such as visual style, conceptual content, artist effects, session context, or emotional safety, so VA RecSys has developed as a multimodal and domain-specific branch of recommender systems rather than a direct transplant of generic collaborative filtering (He et al., 2016, Yilma et al., 2023, Yilma et al., 2024, Yilma et al., 2024).

1. Historical emergence and problem scope

Early work established that art recommendation cannot be reduced to standard item recommendation with a few metadata tags. In digital art, Vista models Behance as a large sequential environment in which users are both creators and consumers, and where recommendations must combine visual appearance, creator effects, and temporal browsing consistency (He et al., 2016). In physical art markets, content-based recommendation became prominent because items are often one-of-a-kind, sparse, and unavailable for repeated interaction, which weakens the assumptions behind conventional collaborative filtering (Messina et al., 2017). CuratorNet then pushed this direction toward a learned visual architecture whose parameters are trained once and can generalize to new users and new artworks without retraining (Messina et al., 2020).

Museum and cultural-heritage settings widened the field further. Text-driven and multimodal systems for paintings in the National Gallery, London, treat recommendation as matching latent semantic relationships rather than only surface-level visual similarity, and they evaluate recommendation quality with user-centric notions such as novelty, serendipity, and diversity (Yilma et al., 2020, Yilma et al., 2023). In contemporary art, Docent addresses a different regime again: a content-based system for artwork-to-artwork recommendation without any user-item interaction data, using curated visual descriptors and artist context to approximate professional curatorial judgment (Fosset et al., 2022).

Recent work extends VA RecSys beyond individual preference matching. MOSAIC frames art recommendation as a multistakeholder problem involving visitors, crowd popularity, and curatorial representativeness across story groups (Yilma et al., 2024). OpArt reformulates public art curation as an equity-aware assignment problem over locations and artist groups, making fairness objectives explicit in the recommendation process (Haensch et al., 2022). Therapeutic systems for post-intensive care recovery and art therapy introduce affective appropriateness, safety filtering, and cross-domain preference elicitation from music, showing that VA RecSys now includes clinical and well-being-oriented use cases as well as discovery and commerce (Yilma et al., 2024, Yilma et al., 18 Jul 2025).

2. Data regimes and task formulations

The literature spans several distinct interaction regimes. Vista uses two timestamped Behance datasets: appreciations with 373,771 users, 982,002 items, and 11,807,103 appreciate events, and clicks with 381,376 users, 972,181 items, and 48,118,748 click events. About 52.7% of users are also creators, about 2.3% of items have multiple creators, and each project has a cover image represented by a 4096-dimensional VGG feature vector. The task is sequential next-item prediction over per-user sequences $$\mathcal{S}^u = (\mathcal{S}^u_1,\ldots,\mathcal{S}^u_{|\mathcal{S}^u|})$$ (He et al., 2016).

Physical-art commerce introduces a different constraint structure. UGallery contains 1,371 users, 3,490 paintings, and 2,846 purchase transactions, with positive-only feedback over one-of-a-kind items; CuratorNet’s online art store dataset contains 2,378 users, 6,040 artworks, and 5,336 purchases, with ~78% of items being one-of-a-kind paintings and 81.7% of transactions involving those items (Messina et al., 2017, Messina et al., 2020). These settings motivate content-based or visually grounded models because co-purchase patterns are extremely sparse and repeated interaction with the same item is structurally limited.

Museum collections often have the opposite problem: rich item descriptions but very little user feedback. Two National Gallery studies use 2,368 paintings with images and curator-written metadata, while preference elicitation is based on small explicit rating sets from users rather than logs of continuous interaction (Yilma et al., 2020, Yilma et al., 2023). Docent moves further toward pure cold start by assembling a proprietary contemporary-art dataset of $> 1{,}000$ artists and $\approx 5{,}000$ artworks, annotated with $\approx 40$ categorical variables per artwork and contextual artist metadata, but without collaborative data in the core recommender (Fosset et al., 2022).

Professional creative workflows define yet another task. In Shutterstock’s marketplace, the primary problem is not recommending semantically similar images to end-consumers but learning relatively stable long-term visual style preferences for creatives whose project semantics shift rapidly. The system uses $\sim 10$M clicks from $\sim 90$K users over 6 months and explicitly separates stage-1 semantic retrieval from stage-2 style reranking (Bruballa et al., 2022). Therapeutic systems simplify interaction further: one line of work uses a single chosen “healing” painting as the query for content-based recommendation, while another uses ratings on 11 music tracks to infer an affective profile that is then mapped to paintings (Yilma et al., 2024, Yilma et al., 18 Jul 2025).

3. Representation learning and multimodal semantics

A central organizing axis in VA RecSys is how artworks are represented. Visual-only pipelines typically start from transfer learning. Vista extracts a 4096-dimensional cover-image vector with a pre-trained VGG network and learns low-dimensional item factors through linear embeddings; CuratorNet uses ResNet-50 features of dimension 2048 and maps them into a learned recommendation space through shared dense layers (He et al., 2016, Messina et al., 2020). The early UGallery study compares manually curated metadata, explicit attractiveness-based visual features such as brightness and colorfulness, and 4096-dimensional AlexNet features, finding the DNN-derived representation strongest among the single-source approaches (Messina et al., 2017).

Textual modeling emerged as an important counterweight to pure visual similarity. One painting recommender trains LDA on enriched textual descriptions and represents each painting by a topic-distribution vector $\theta_{p_i}\in\mathbb{R}^{10}$, showing that topic-based semantics can uncover non-obvious relationships and support explanation through topic words and word clouds (Yilma et al., 2020). A later comparative study extends this with SBERT embeddings and BERTopic, using all-MiniLM-L6-v2, UMAP, HDBSCAN, and c-TF-IDF, and finds that textual features compare favourably with visual ones while late fusion of text and image captures the most suitable hidden semantic relationships (Yilma et al., 2023).

Contemporary-art systems often embed domain knowledge directly. Docent couples frozen ImageNet backbones such as AlexNet, VGGNet, and ResNet-50 with classifiers over art-specific criteria, and then concatenates predicted visual classes with artist embeddings derived from biographies, press articles, date of birth, working cities, CV data, and manually defined artistic tags from a curated art dictionary. Its artwork embedding has the form

$$\mathbf{x}_i = \Big[\alpha_1 \mathbf{c}_i^{(1)},\alpha_2 \mathbf{c}_i^{(2)},\dots,\alpha_K \mathbf{c}_i^{(K)},\mathbf{e}_{a_i}\Big],$$

so classifier reliability and artist context both shape similarity (Fosset et al., 2022).

Multimodal pretraining has become increasingly important. MOSAIC uses CLIP and BLIP as feature extractors for paintings with images, metadata, and curatorial descriptions; BLIP is particularly motivated by its smaller modality gap through image-text contrastive, image-text matching, and language-modeling objectives (Yilma et al., 2024). In therapeutic recommendation, BLIP serves as a multimodal backbone while ResNet provides a purely visual baseline; the pilot results there show that text-heavy engines can surface semantically aligned but therapeutically unsafe works, whereas visual and multimodal engines are more appropriate for the setting (Yilma et al., 2024).

The representation problem is not only multimodal but also social and affective. Vista factorizes creator identity explicitly through ownership signals, while cross-domain art therapy models map both music and paintings into valence-arousal spaces or joint latent spaces. Mozart aligns music and paintings through affect-aware contrastive learning, Haydn matches them directly in $2$-dimensional valence-arousal space, and Salieri combines MERT audio features, LLM-generated music descriptions, ResNet50 painting features, and VLM-generated painting descriptions before cross-domain recommendation (He et al., 2016, Yilma et al., 18 Jul 2025).

Before turning to models, it is useful to summarize several canonical representational choices.

System	Primary item representation	User or context representation
Vista	VGG cover-image features plus creator ownership	user, creator, and sequence factors
Docent	weighted art criteria plus artist context	no core user model
CuratorNet	ResNet50 visual embeddings	pooled history of consumed images
MOSAIC	CLIP or BLIP image-text embeddings	weighted similarity to rated paintings
Artful Path / AT CDR	ResNet or BLIP, or affective cross-domain embeddings	seed painting or music-rated affect profile

This variety shows that “visual” in VA RecSys does not denote image-only processing. In many systems, visual evidence is fused with creator identity, textual semantics, curatorial structure, or affect labels to make the latent space behaviorally or therapeutically meaningful.

4. Recommendation architectures and optimization

The model families in VA RecSys range from sequential latent-factor models to nearest-neighbor retrieval, two-tower rerankers, and constrained optimization.

Vista is the canonical sequential latent-factor model. Its first-order prediction combines long-term user-creator and user-item affinity with short-term creator-creator and item-item consistency: $\begin{multline} P(X^u_t = i \mid \mathcal{S}^u_{t-1}) \propto \langle \phi_u,\phi_{o_i}\rangle + \langle \gamma_u,\gamma_i\rangle \ + w_u \cdot \big(\langle \phi_{o_i},\phi_{o_{\mathcal{S}^u_{t-1}}}\rangle + \langle \psi_i,\psi_{\mathcal{S}^u_{t-1}}\rangle\big). \end{multline}$ It generalizes to a $> 1{,}000$0-th order personalized Markov chain with decay

$> 1{,}000$1

and injects visual content through

$> 1{,}000$2

Learning uses sequential Bayesian Personalized Ranking (S-BPR), stochastic gradient descent, reduced update frequency for the visual embedding matrices, and asynchronous lock-free SGD (Hogwild) for scale (He et al., 2016).

CuratorNet adopts a different logic suited to sparse, one-of-a-kind art catalogs. It transforms each ResNet feature vector $> 1{,}000$3 through shared dense layers, aggregates a user’s consumed artworks by average pooling and max pooling, and scores candidate items by an inner product

$> 1{,}000$4

Training is pairwise and triplet-based, and a major contribution is domain-aware triplet sampling: leave-one-out basket prediction, sequential purchase prediction, favorite-artist continuation, and artificial single-item profiles all outperform naïve random negatives (Messina et al., 2020).

A large share of museum-oriented VA RecSys remains similarity-based rather than fully parametric. Both the LDA recommender and the multimethod “Elements” study score a candidate painting $> 1{,}000$5 for user $> 1{,}000$6 by weighted aggregation over the paintings $> 1{,}000$7 the user rated: $> 1{,}000$8 Here $> 1{,}000$9 is a precomputed painting-painting similarity matrix obtained from LDA topic distributions, SBERT embeddings, ResNet features, or a multimodal fusion, and $\approx 5{,}000$0 is the normalized rating weight (Yilma et al., 2020, Yilma et al., 2023).

Docent implements content-based nearest-neighbor recommendation in a hand-crafted multimodal feature space. Distances are computed with the $\approx 5{,}000$1 norm,

$\approx 5{,}000$2

and recommendation returns the top-5 closest artworks subject to the constraint that they come from 5 different artists. The combined visual-contextual distance is weighted, and contextual variables receive higher weights in the final system (Fosset et al., 2022).

Style-oriented recommendation for creatives is architecturally closer to large-scale industrial retrieval. The Shutterstock system uses a two-tower model: a user tower over user, organization, country, language, and subscription features; and an image tower over categories, contributor country, colors, image angle, photo style, and Inception deep features. The image tower combines an MLP with a parallel Deep Cross Network, then scores candidates by

$\approx 5{,}000$3

Training uses a symmetric cross-entropy loss over in-batch negatives, and the deployed system acts as a stage-2 style reranker over semantically filtered search results (Bruballa et al., 2022).

Multistakeholder curation introduces optimization objectives that are not reducible to pairwise ranking. MOSAIC defines a personalized score $\approx 5{,}000$4, a popularity-augmented score

$\approx 5{,}000$5

and a representativeness term

$\approx 5{,}000$6

It then solves mixed-integer formulations that trade off user preference, popularity, and representative coverage across curator-defined story groups through the parameter $\approx 5{,}000$7 (Yilma et al., 2024). OpArt goes further from personalized ranking: it defines a cost matrix over locations and artwork groups, optimizes a soft assignment matrix $\approx 5{,}000$8 under capacity constraints, and solves the resulting convex problem by projected gradient descent in order to de-prioritize in-group preferences and satisfy minimum representation and exposure criteria (Haensch et al., 2022).

5. Evaluation paradigms and application domains

Evaluation in VA RecSys is unusually heterogeneous because the applications differ so strongly. Sequential digital-art recommendation uses standard next-item ranking metrics. On Behance appreciations, Vista+ reaches an overall AUC of $\approx 5{,}000$9 versus $\approx 40$0 for FPMC, and on cold items reaches $\approx 40$1 versus $\approx 40$2 for FPMC; on click data, improvements over FPMC are smaller overall but remain large for cold items (He et al., 2016). In physical art commerce, content-based top-$\approx 40$3 recommendation is evaluated with precision@$\approx 40$4, recall@$\approx 40$5, and nDCG@$\approx 40$6; the hybrid model using DNN, explicit visual features, and metadata yields the best results, with nDCG@10 $\approx 40$7, recall@10 $\approx 40$8, and precision@10 $\approx 40$9 (Messina et al., 2017). CuratorNet improves on both VBPR and VisRank across AUC, recall@20, precision@20, nDCG@20, recall@100, precision@100, and nDCG@100 (Messina et al., 2020).

Museum and cultural-heritage systems rely heavily on user-centric evaluation. In one study, the LDA-based recommender is rated at roughly $\sim 10$0 for predictive accuracy versus roughly $\sim 10$1 for ResNet, and at roughly $\sim 10$2 for diversity versus roughly $\sim 10$3 for ResNet, while all users report that the topic-based explanations helped them understand the recommendations (Yilma et al., 2020). A later comparison with larger-scale crowdsourcing finds that textual features are at least competitive with visual features and that the LDA+ResNet fusion yields the strongest overall recommendation quality across accuracy, novelty, diversity, and serendipity (Yilma et al., 2023).

Content-based contemporary-art recommendation is evaluated against expert judgment rather than interaction logs. Docent reports 11% meaningful recommendations for a random baseline, 65% for visual-only recommendations, 60% for contextual-only recommendations, and 75% for the final weighted visual+contextual system, based on ratings from seven art experts (Fosset et al., 2022). This is one of the clearest demonstrations that purely content-based art recommendation can be professionally credible when visual and contextual features are art-specific.

Professional creative workflows introduce a different evaluation philosophy. The Shutterstock style reranker measures Acc@$\sim 10$4, catalog coverage, Effective Catalog Size, visual diversity, and MAP in search-engine integration. The best “all features” model reaches Acc@10 $\sim 10$5, Acc@100 $\sim 10$6, Catalog Coverage $\sim 10$7, ECS@10 $\sim 10$8, and Visual Diversity $\sim 10$9; replacing Inception deep features with CLIP harms style generalization, which the authors interpret as overemphasis on semantics rather than transversal style (Bruballa et al., 2022).

Stakeholder-aware and therapeutic settings require still broader evaluation. MOSAIC combines offline overlap analysis with a user study of 100 crowdworkers and finds a strong effect of popularity, which is positively perceived by users, and a minimal effect of representativeness; BLIP-based variants generally outperform CLIP-based ones on subjective accuracy, novelty, and serendipity (Yilma et al., 2024). OpArt evaluates fairness in public-art curation through a representation metric $\sim 90$0, showing that optimized assignments can substantially increase self-representative exposure for disadvantaged groups relative to the current curation (Haensch et al., 2022). In therapy-oriented VA RecSys for post-intensive care recovery, all three conditions—expert curation, a visual ResNet engine, and a multimodal BLIP engine—improve temporal affective states, and the machine-learning engines compare favourably with the expert baseline (Yilma et al., 2024). A related cross-domain study with 200 users shows that music-driven preference elicitation can match or surpass a visual-only baseline for therapeutic painting recommendation, with “Hope” emerging as the dominant theme in user reflections across engines (Yilma et al., 18 Jul 2025).

6. Misconceptions, limitations, and research directions

A recurring misconception is that visual features uniformly dominate textual ones in art recommendation. The evidence is more specific than that. In physical-art commerce, DNN visual features outperform curated metadata and explicit attractiveness features (Messina et al., 2017), but in museum settings textual topic representations compare favourably with visual ones, and text-image fusion is strongest overall in user-centric evaluation (Yilma et al., 2020, Yilma et al., 2023). A second misconception is that “art recommendation” is only recommendation to end-consumers. The Shutterstock work shows that creatives can have rapidly changing short-term project semantics but relatively stable long-term visual style preferences, which alters both architecture and evaluation (Bruballa et al., 2022). A third misconception is that collaborative filtering is always the natural solution. One-of-a-kind physical artworks, sparse museum feedback, and cold-start contemporary art all motivate content-based, creator-aware, or multimodal retrieval approaches instead (Messina et al., 2017, Messina et al., 2020, Fosset et al., 2022).

The limitations are correspondingly domain-specific. Vista relies on fixed pre-trained VGG features and a Markov-chain sequential component rather than end-to-end visual backbones or transformer sequence models (He et al., 2016). Docent uses frozen CNN backbones, manual tag curation, weighted $\sim 90$1 similarity, and proprietary data, while lacking explicit user modeling in its reported system (Fosset et al., 2022). CuratorNet is visually grounded and cold-start friendly, but it does not integrate rich text, general social structure, or explicit temporal dynamics in the scoring function (Messina et al., 2020). The museum similarity models are explainable and lightweight, but they do not use interaction logs, collaborative signals, or end-to-end optimization on recommendation criteria (Yilma et al., 2020, Yilma et al., 2023). Therapeutic systems narrow the candidate space through safety considerations and curated positive-valence selections, which is necessary clinically but limits the emotional range of recommendation (Yilma et al., 2024, Yilma et al., 18 Jul 2025). Multistakeholder systems show that explicit representativeness objectives may have only small marginal effects when the base embedding space already distributes content broadly (Yilma et al., 2024).

Several forward directions recur across the literature. Vista explicitly suggests replacing VGG with stronger multimodal embeddings such as CLIP, replacing Markov chains with recurrent or transformer models, and jointly learning visual representations with recommendation losses (He et al., 2016). Docent proposes user personalization, art-specific BERT, playlist-style inputs, and richer art-history/context tags (Fosset et al., 2022). MOSAIC suggests that stakeholder trade-offs should remain user-tunable through interpretable parameters such as $\sim 90$2 and $\sim 90$3 (Yilma et al., 2024). Therapeutic work points toward richer affective metadata, longitudinal clinical validation, and tighter human-in-the-loop workflows (Yilma et al., 2024, Yilma et al., 18 Jul 2025).

Adjacent multimodal recommendation research also points to plausible next steps for VA RecSys. “RAG-VisualRec” shows that LLM-generated textual augmentation, CCA-based fusion of text and visual features, retrieval-augmented generation, and LLM reranking can substantially improve recommendation quality in a sparse-metadata domain (Tourani et al., 25 Jun 2025). This suggests a concrete blueprint for art collections with short titles, sparse descriptions, or cold-start artworks. Likewise, interpretable image representations and explicit aesthetic feature learning in visually-aware recommendation outside the art domain suggest that future VA RecSys may benefit from art-specific attribute spaces and aesthetic encoders that improve both controllability and explanation (Packer et al., 2018, Yu et al., 2019).

Taken together, the field no longer centers on a single canonical problem. VA RecSys now includes sequential digital-art recommendation, one-of-a-kind marketplace ranking, museum discovery, contemporary-art advisory systems, style reranking for creatives, stakeholder-aware exhibit curation, equity-aware public-art allocation, and therapeutic affect-sensitive recommendation. What unifies these settings is the need to model artworks as multimodal, subjective, and socially situated objects, and to evaluate recommendation not only by retrieval accuracy but also by cold-start behavior, explanation, diversity, serendipity, stakeholder alignment, and, in some settings, emotional safety.