Cold-Start Active Preference Learning

• Cold-Start Active Preference Learning is a research area that tackles preference estimation when little to no prior interaction data is available, combining active and meta-learning techniques.
• It encompasses a variety of regimes such as new-user, new-item, and cross-domain recommendations, utilizing query selection and initialization strategies to optimize data acquisition.
• Methods leverage uncertainty, diversity sampling, and reinforcement learning to achieve rapid adaptation and improve performance with minimal labeled data.

Searching arXiv for recent and foundational papers on cold-start active preference learning, preference elicitation, and cold-start recommendation. Cold-start active preference learning addresses preference estimation when a system begins with insufficient or no labeled interactions and must therefore decide which items, pairs, explanations, or dialogue actions to present in order to maximize information acquisition with minimal user effort. In the literature, the setting spans new-user and new-item recommendation, sequential recommendation with sparse histories, pure zero-label pairwise preference learning, information filtering under limited attention, and conversational preference elicitation. The corresponding methods combine active learning, meta-learning, self-supervised initialization, transfer, and sequential decision making (Cantürk et al., 2023, Lee et al., 2019, Fayaz-Bakhsh et al., 7 Aug 2025, Zhao et al., 2014).

1. Problem scope and task regimes

The cold-start condition is instantiated in several distinct but related regimes. User cold-start concerns new users with little or no interaction history; item cold-start concerns new or minimally interacted items; user-item cold-start combines both; time-sensitive cold-start concerns users with historical interactions but sparse recent interactions; and cross-domain cold-start concerns transfer from a source domain with richer interactions to a target domain with much fewer interactions or newly arriving contents (Zheng et al., 2020, Huang et al., 2021, Neupane et al., 2022, Zhou et al., 2021).

A common formalization treats each user as a task in few-shot meta-learning. Support sets contain the few observed interactions used for fast adaptation, and query sets evaluate generalization after adaptation. Other formulations are pairwise: in socio-economic preference learning, the learner queries unlabeled pairs and receives noisy Bradley-Terry labels; in information filtering, the learner chooses how many items to forward from each category and receives binary relevance feedback; in conversational systems, the learner must infer and ground preferences across multiple turns (Yu et al., 2020, Fayaz-Bakhsh et al., 7 Aug 2025, Zhao et al., 2014, Kong et al., 2023).

Regime	Representative formulation	Representative papers
User cold-start recommendation	Few-shot task per user; support/query adaptation	(Lee et al., 2019, Huang et al., 2021)
Item cold-start recommendation	Active user selection or few-shot item matching	(Zhu et al., 2018, Zheng et al., 2020)
Zero-label pairwise preference learning	Self-supervised pre-training, then active pair queries	(Fayaz-Bakhsh et al., 7 Aug 2025)
Explainable elicitation	Query items plus explanation feedback	(Cantürk et al., 2023)
Sequential decision filtering	Exploration-exploitation over forwarded items	(Zhao et al., 2014)

This heterogeneity is important because the available signals differ sharply across settings. Some methods assume content or semantic tags, some assume only user-item interactions, some assume no access to other users’ data, and some explicitly operate without side information at all (Zheng et al., 2020, Cantürk et al., 2023, Huang et al., 2021).

2. Acquisition objectives and query selection criteria

A central design question is how to select the next query before a reliable model exists. In explainable active learning for preference elicitation, the benchmarked strategies include uncertainty-based sampling, diversity-based sampling, and combined strategies. The uncertainty scores are given explicitly as normalized least confidence,

$\frac{|C|}{|C|-1}\left(1 - P_\theta(y_1|x)\right),$

margin score,

$$1 - \left(P_\theta(y_1|x) - P_\theta(y_2|x)\right),$$

and normalized entropy,

$$\frac{-\sum_{i\in C} P_\theta(y_i|x)\log P_\theta(y_i|x)}{\log |C|},$$

with $y_1$ and $y_2$ the most and second-most probable labels (Cantürk et al., 2023).

For new-user recommendation, MeLU introduces an evidence candidate selection strategy in which informativeness is estimated by the Frobenius norm of the gradient of the local update,

$$\left\Vert \nabla_{\theta_2^i} \mathcal{L}_i(f_{\theta_1,\theta_2^i}) \right\Vert_F,$$

and then combined with item popularity after normalization to $[0,1]$. The stated goal is to balance system informativeness and user awareness rather than rely only on popularity-based onboarding items (Lee et al., 2019).

For new-item recommendation, attribute-driven active learning uses four explicit user-selection criteria: willingness to rate, diversity of potential ratings, objectivity of ratings, and representativeness. These are combined in an integer quadratic programming objective,

$$\max_{\mathbf{q}} \; \alpha \sum_m q(m)p(m) + \beta \sum_{m,n} q(m)q(n)D(m,n) - \gamma \sum_m q(m)o(m) + \sigma \sum_{m,n} q(m)(1-q(n))S(m,n),$$

subject to $q(m)\in\{0,1\}$ and $\sum_m q(m)=k$. The formulation explicitly leverages both item attributes and users’ rating history (Zhu et al., 2018).

A recurring limitation is that uncertainty by itself can be ineffective at true cold start. Cold-start active correlation clustering states that conventional uncertainty-based methods perform poorly when no pairwise similarity information is available initially, because uncertainty estimates are uninformative and selection bias leads to poor coverage. The proposed remedy is a coverage-aware allocation over dynamically defined regions, with region query proportion

$$1 - \left(P_\theta(y_1|x) - P_\theta(y_2|x)\right),$$0

This directly encodes early diversity and reduced batch redundancy (Aronsson et al., 29 Sep 2025).

3. Initialization without labels or with extremely sparse labels

Several recent lines of work focus on the initialization phase itself, rather than only on later active updates. In cold-start active learning for medical imaging, foundation model embeddings are used in place of raw pixel features for clustering-based initialization. The procedure computes embeddings for all unlabeled samples, clusters them, and selects samples nearest to cluster centroids. On ChestX-Det, with initial budgets $$1 - \left(P_\theta(y_1|x) - P_\theta(y_2|x)\right),$$1, foundation model-based clustering outperformed random sampling and naive clustering across classification and segmentation metrics such as AUPRC, F1, DSC, and HD; the paper emphasizes that naive clustering on raw features suffers from convergence speed in high-dimensional data (Yuan et al., 2024).

A more explicit zero-label preference formulation appears in socio-economic domains. The model first computes one-component PCA,

$$1 - \left(P_\theta(y_1|x) - P_\theta(y_2|x)\right),$$2

then uses residuals

$$1 - \left(P_\theta(y_1|x) - P_\theta(y_2|x)\right),$$3

to control both pair sampling and the pre-training size

$$1 - \left(P_\theta(y_1|x) - P_\theta(y_2|x)\right),$$4

Pseudo-pairs are labeled by comparing PCA scores, a binary preference model is pre-trained with binary cross-entropy, and the resulting warmed-up model enters an uncertainty-sampling loop with a noisy Bradley-Terry oracle

$$1 - \left(P_\theta(y_1|x) - P_\theta(y_2|x)\right),$$5

Across financial credibility, career success rate, and socio-economic status datasets, the reported pattern is higher F1 with substantially fewer labeled pairs than blank-slate active learning, and F1-scores close to those achieved by other models with $$1 - \left(P_\theta(y_1|x) - P_\theta(y_2|x)\right),$$6–$$1 - \left(P_\theta(y_1|x) - P_\theta(y_2|x)\right),$$7 more labeled data (Fayaz-Bakhsh et al., 7 Aug 2025).

Explainable elicitation provides a weaker but practically important initialization regime: a diverse candidate set $$1 - \left(P_\theta(y_1|x) - P_\theta(y_2|x)\right),$$8 is created by clustering, an initial labeled set $$1 - \left(P_\theta(y_1|x) - P_\theta(y_2|x)\right),$$9 is selected for the user, and Label Spreading propagates labels to form a pseudo-labeled set $$\frac{-\sum_{i\in C} P_\theta(y_i|x)\log P_\theta(y_i|x)}{\log |C|},$$0. This is neither pure zero-shot nor standard supervised seeding; rather, it is an integrated UL-SSL-SL warm start that uses true labels, pseudo-labels, and explanation-derived synthetic samples (Cantürk et al., 2023).

4. Meta-learning, transfer, and rapid adaptation

Meta-learning is one of the dominant paradigms for cold-start preference estimation because it optimizes for fast adaptation from a small support set. MeLU is an early formulation in which user content and item content are embedded and passed through a multilayer network, while MAML-style local updates adapt decision layers from a few observed interactions: $$\frac{-\sum_{i\in C} P_\theta(y_i|x)\log P_\theta(y_i|x)}{\log |C|},$$1 On MovieLens 1M and BookCrossing, MeLU reduced at least $$\frac{-\sum_{i\in C} P_\theta(y_i|x)\log P_\theta(y_i|x)}{\log |C|},$$2 mean absolute error relative to two comparative models, and its evidence candidates produced higher user satisfaction in a user study (Lee et al., 2019).

PAML modifies the standard MAML update by replacing a fixed inner-loop step size with a user-specific adaptive learning rate: $$\frac{-\sum_{i\in C} P_\theta(y_i|x)\log P_\theta(y_i|x)}{\log |C|},$$3 The motivation is explicitly task imbalance: major user groups are overrepresented, while minor users are underrepresented. PAML adds a similarity-based method to find similar users, a kd-tree and approximated kd-tree for fast retrieval, and a memory agnostic regularizer

$$\frac{-\sum_{i\in C} P_\theta(y_i|x)\log P_\theta(y_i|x)}{\log |C|},$$4

so that REG-PAML attains constant space complexity while maintaining performance. On MovieLens-1M, BookCrossing, and a Taobao production dataset, REG-PAML and AT-PAML outperform the baselines on MSE, nDCG@K, AUC, and Negative Entropy Loss, and the reported $$\frac{-\sum_{i\in C} P_\theta(y_i|x)\log P_\theta(y_i|x)}{\log |C|},$$5-values indicate that REG-PAML’s expected error for minor users is statistically indistinguishable from that of major users $$\frac{-\sum_{i\in C} P_\theta(y_i|x)\log P_\theta(y_i|x)}{\log |C|},$$6 (Yu et al., 2020).

Sequential recommendation requires additional machinery because the sparse evidence is temporally ordered. Mecos formulates item cold-start as few-shot matching over support and query sequences, using a sequence encoder

$$\frac{-\sum_{i\in C} P_\theta(y_i|x)\log P_\theta(y_i|x)}{\log |C|},$$7

and a recurrent matching processor with cosine similarity. It requires no side information and reports average improvement up to $$\frac{-\sum_{i\in C} P_\theta(y_i|x)\log P_\theta(y_i|x)}{\log |C|},$$8, $$\frac{-\sum_{i\in C} P_\theta(y_i|x)\log P_\theta(y_i|x)}{\log |C|},$$9, and $y_1$0 in HR@10 over state-of-the-art baselines on Steam, Amazon Electronic, and Tmall. metaCSR instead combines a Diffusion Representer, a self-attention Sequential Recommender, and a MAML-style Meta Learner; on MovieLens-1M it reports AUC $y_1$1 $y_1$2 and MAP $y_1$3 $y_1$4 over the best baseline, while on Amazon-Video it reports $y_1$5 MAP. A dynamic meta-learning model for time-sensitive cold-start further decomposes user state into time-specific and time-evolving factors,

$y_1$6

and reports that the combined model outperforms stand-alone modules and standard baselines across Netflix, MovieLens-1M, and Last.fm (Zheng et al., 2020, Huang et al., 2021, Neupane et al., 2022).

Transfer and auxiliary structure extend the same logic. MetaKG couples a collaborative-aware meta learner, a knowledge-aware meta learner, and an adaptive task scheduler on a collaborative knowledge graph; the reported gains are $y_1$7–$y_1$8 improvement on Recall/NDCG@20 in cold-start scenarios. COUPLE models user history, content, and group preference with semantic tags, a frequency encoder, and a hierarchical memory tree, and reports superior CTR in a one-week online A/B test on Taobao APP. MetaDPA generates diverse ratings in a target domain using Dual-CVAEs with Multi-domain InfoMax and Mutually-Exclusive constraints and then meta-trains on both original and augmented tasks; on Books under cold-start users, it reports HR@10 $y_1$9 versus $y_2$0 for MeLU and $y_2$1 for MetaCF (Du et al., 2022, Zhou et al., 2021, Zhang et al., 2022).

5. Interactive, explainable, and conversational elicitation

Active preference learning is not only about sample selection; it also concerns how feedback is elicited and reused. In explainable active learning for food recommendation, the learner integrates unsupervised clustering, semi-supervised Label Spreading, and supervised logistic regression. Users provide like/dislike labels for selected recipes and can also validate or correct feature-level explanations. Those corrections are encoded as synthetic labeled samples—for each feature, an item with only that feature is created and labeled accordingly—and are added to the training set with increased weight. The reported results include efficiency with limited user-labeled data, CU and Ub as strong query strategies, $y_2$2 participants in the human study, and up to $y_2$3 improvement in explanation correctness at $y_2$4 steps, alongside higher trust and low perceived burden (Cantürk et al., 2023).

Multi-turn conversational preference learning pushes the interactional aspect further. The TAI system uses BERT-based encoders over current utterance, past utterances, current entities, past entities, and past actions,

$y_2$5

with separate models for action prediction, argument filling, and named entity recognition. A seeker-provider interaction loop generates diverse dialogues from cold start, while a centralized knowledge base stores grounded preferences for reuse. On $y_2$6 simulated training dialogues and out-of-sample human-authored tests, the full NER+AP+AF stack achieves $y_2$7 turn-level accuracy out of sample and has been adopted in production (Kong et al., 2023).

A distinct interactive perspective places control on the end user rather than on the service provider. Pretender assumes a source history $y_2$8, a target item set $y_2$9, and a small budget $$\left\Vert \nabla_{\theta_2^i} \mathcal{L}_i(f_{\theta_1,\theta_2^i}) \right\Vert_F,$$0 of target interactions. It constructs a target empirical distribution

$$\left\Vert \nabla_{\theta_2^i} \mathcal{L}_i(f_{\theta_1,\theta_2^i}) \right\Vert_F,$$1

and selects target items so as to minimize a distributional discrepancy to the source empirical distribution $$\left\Vert \nabla_{\theta_2^i} \mathcal{L}_i(f_{\theta_1,\theta_2^i}) \right\Vert_F,$$2, using IPM, MMD, or Wasserstein objectives. The method provides discrete-quadrature-style guarantees and, on MovieLens, Last.fm, and Amazon, reports that Pretender approaches the continuous relaxation and outperforms Random and Greedy in both intersection and no-intersection settings (Sato, 18 Feb 2025).

6. Decision-theoretic formulations, empirical regularities, and recurring misconceptions

Cold-start active preference learning has a strong decision-theoretic strand. In Bayesian information filtering, each category $$\left\Vert \nabla_{\theta_2^i} \mathcal{L}_i(f_{\theta_1,\theta_2^i}) \right\Vert_F,$$3 has an unknown relevance probability $$\left\Vert \nabla_{\theta_2^i} \mathcal{L}_i(f_{\theta_1,\theta_2^i}) \right\Vert_F,$$4, the state is the posterior sufficient statistics $$\left\Vert \nabla_{\theta_2^i} \mathcal{L}_i(f_{\theta_1,\theta_2^i}) \right\Vert_F,$$5, and the action is the number $$\left\Vert \nabla_{\theta_2^i} \mathcal{L}_i(f_{\theta_1,\theta_2^i}) \right\Vert_F,$$6 of items to forward. The objective is

$$\left\Vert \nabla_{\theta_2^i} \mathcal{L}_i(f_{\theta_1,\theta_2^i}) \right\Vert_F,$$7

which makes the exploration-exploitation trade-off explicit because forwarding potentially irrelevant items can improve later decisions. The paper derives a decomposed dynamic program, a Bayes-optimal forwarding algorithm when each forwarded article is examined, and an index policy for limited attention that reduces to the Gittins index when $$\left\Vert \nabla_{\theta_2^i} \mathcal{L}_i(f_{\theta_1,\theta_2^i}) \right\Vert_F,$$8 and all queues are nonempty (Zhao et al., 2014).

A related but modern alternative is to absorb elicitation into online reinforcement learning. A cold-user recommender based on matrix factorization plus Double and Dueling DQNs treats recommendation as an MDP with a $$\left\Vert \nabla_{\theta_2^i} \mathcal{L}_i(f_{\theta_1,\theta_2^i}) \right\Vert_F,$$9-dimensional binary state over already shown popular items and an action space of the top $[0,1]$0 popular items. It reports mean RMSE $[0,1]$1 for Double DQN, $[0,1]$2 for Dueling DQN, and $[0,1]$3 for standard DQN, versus $[0,1]$4 for Popularity and $[0,1]$5 for PopError, with Dueling DQN giving the lowest RMSE at $[0,1]$6 items $[0,1]$7 (Zhao, 28 Aug 2025).

Several misconceptions recur across the literature. One is that cold start is adequately handled by uncertainty sampling alone. The initialization and coverage papers explicitly dispute this: uncertainty is weak when the model begins from a blank slate, and clustering or self-supervised warm starts materially change downstream learning curves (Yuan et al., 2024, Fayaz-Bakhsh et al., 7 Aug 2025, Aronsson et al., 29 Sep 2025). Another is that meta-learning automatically solves cold start; PAML shows that standard MAML can fit major users and ignore minor users under user imbalance (Yu et al., 2020). A third is that side information is always necessary; Mecos and metaCSR are designed for settings where only user-item interactions are available, while explainable elicitation assumes no access to other users’ data (Zheng et al., 2020, Huang et al., 2021, Cantürk et al., 2023). Taken together, these results suggest that the decisive design choices are not only the learner class, but also the cold-start initialization, the acquisition geometry in the zero-label phase, the structure of the interaction channel, and the extent to which the method accounts for imbalance, noise, and limited user effort.