Cold-Start Probability Analysis

• Cold-start probability analysis is a framework to estimate probabilities and predictions when historical data is sparse, addressing new entities in various domains.
• It leverages techniques such as causal GNN forecasting, k-NN imputation, and text-to-distribution LLMs to generate reliable insights from limited data.
• The approach integrates multiscale Metropolis chains and side-information methods, providing theoretical guarantees and empirical improvements in prediction and sampling tasks.

Cold-start probability analysis is concerned with the challenge of accurately estimating probabilities, predictions, or recommendation scores when the underlying data is sparse or missing, particularly for new entities—users, items, attributes, or time series—that have little or no historical information. This phenomenon arises in a range of fields, including recommender systems, multivariate time-series forecasting, Markov chain mixing, and cluster-based collaborative filtering. The aim is both to formally characterize the uncertainty and convergence properties associated with cold-start scenarios and to design principled, computationally tractable frameworks that mitigate prediction degradation under such constraints.

1. Formalization of the Cold-Start Problem

Cold-start describes regimes where a subset of variables, users, items, or system states lack historical observation. In multivariate time series, as in the CDF-cold model (Fatemi et al., 2023), for $N$ entities each with $A$ attributes and $T$ time steps, cold-start is defined for a subset $C \subseteq \{1,\ldots,A\}$ of attributes, where $x_{i,t}^j$ for $j\in C$ has no available past window. In recommendation systems, cold-start typically refers to new users or items with no feedback or interactions data, necessitating side-information-based or imputation methods (Cortes, 2018, Liu et al., 24 Feb 2025). In probabilistic sampling, cold-start refers to mixing from initial distributions with possibly very high density ratios with respect to the target (e.g., $M$-warm or exponentially unbounded) (Narayanan et al., 2022).

Mathematically, the cold-start estimation task is to infer $P(y|x)$ or $\hat y$ given $x$, when the relevant conditional likelihood or model parameters for $x$ are not accessible or are underdetermined by data.

2. Modeling Frameworks and Probabilistic Formulation

2.1 Multivariate Time Series: CDF-cold

CDF-cold (Fatemi et al., 2023) approaches cold-start forecasting through a two-stage framework: (a) causal demand forecasting (CDF) for attributes with available history, and (b) imputation for cold attributes using similarity-based aggregation. The causal graph $M \in \{0,1\}^{A\times A}$ is extracted via VARLiNGAM, enforcing via GNN layers that only causal-parent features are aggregated per series. Cold attributes are imputed by averaging top-$k$ nearest-neighbor forecasts, identified by GMM clustering or ERos pairwise distances, rather than by explicit probabilistic models or variational inference.

There is no generative or latent-variable model posited for cold-start; thus, uncertainties in imputed series are not quantified, and point forecasts are evaluated by RMSE, MAE, and MAPE.

2.2 Recommender Systems: FilterLLM and CMF

FilterLLM (Liu et al., 24 Feb 2025) introduces a “Text-to-Distribution” paradigm wherein, for item text $x \in \mathbb{R}^k$, the LLM predicts a probability distribution $P(u|x)$ across the entire user base, leveraging an augmented vocabulary of user embedding tokens $z_u \in \mathbb{R}^d$. The distribution is given by:

$$P(u|x) = \frac{\exp(h_x^\top z_u)}{\sum_{v\in U} \exp(h_x^\top z_v)}$$

where $h_x$ is the contextual representation of the item under the Transformer backbone.

Cold-start for new items is handled by sampling top-$K$ users via $P(u|x)$ and pseudo-interacting in downstream collaborative optimizers to generate cold-item embeddings; for new users, analogous approaches are possible if user metadata is available but remain unexplored in this framework.

CMF-based methods (Cortes, 2018) handle cold-start by reconstructing latent factors from side information via ridge regression:

$a_{u_0} = (C^\top C + \lambda I)^{-1} C^\top u_0$

and additive offsets formulations, which offer closed-form cold-start predictions using matrix-vector multiplication.

2.3 Markov Chain Mixing and Sampling

Sampling from convex sets with cold start (Narayanan et al., 2022) formalizes cold-start in probabilistic mixing as initial distributions $\nu$ on a convex body $K$ with density $f = d\nu/d\pi_K$ possibly as large as $e^{\text{poly}(n)}$. The analysis introduces multiscale Metropolis chains over Whitney cube decompositions to achieve rapid mixing even under cold-start, leveraging isoperimetric inequalities in a boundary-magnifying Finsler metric.

The conductance profile for such chains controls mixing time bounds:

$$\Phi_{M_p} \geq \Omega\left(\frac{r}{R n^{2+1/p}}\right), \quad T=O(n^{4+2/p} (r/R)^{-2}\log(M/\epsilon)).$$

3. Algorithmic Strategies for Cold-Start Mitigation

Domain	Methodology	Key Algorithmic Steps
Multivariate TS (CDF-cold)	Causal GNN + k-NN Imputation	Causal graph discovery, GNN/LSTM forecasting, similarity-based average
Recommender (FilterLLM)	Text-to-Distribution LLM	Augmented user-token vocabulary, softmax prediction, sampled top-$K$ users
Recommender (CMF/offsets)	Side-info factorization	Ridge regression, attribute mapping, offset decomposition
Probabilistic sampling	Multiscale Metropolis chain	Dyadic cube tiling, boundary-magnifying metric, Metropolis transitions

In CDF-cold (Fatemi et al., 2023), cold-start imputation is performed at inference by transferring forecasts from similar historical entities, leveraging clustering or distance-based similarity. In FilterLLM (Liu et al., 24 Feb 2025), cold items are seeded with synthetic top-$K$ user interactions generated from distributional outputs. CMF (Cortes, 2018) and offsets formulations bridge attribute vectors to latent collaborative factors, enabling immediate scoring via closed-form formulas.

The multiscale cube-based Metropolis chains (Narayanan et al., 2022) achieve rapid probabilistic mixing from cold-start by adapting step size and transitions near the boundary, obviating the need for extensive preprocessing.

4. Quantitative Analysis and Evaluation Metrics

Empirical and theoretical analysis of cold-start scenarios utilizes domain-specific metrics:

• Forecasting (CDF-cold): RMSE, MAE, and MAPE evaluated on simulated cold-start for Google data center time-series; best results achieved with cluster-based aggregation and k ≈ 5–8 neighbors.
• Recommendation (FilterLLM/CMF): Recall@20 and NDCG@20 for recommendation accuracy; FilterLLM demonstrated Recall@20 improvement from 0.1035 to 0.1604 in strict cold subsets, with >30× speedup in inference compared to baseline (Liu et al., 24 Feb 2025). CMF/offsets models yielded improvements over non-personalized baselines, with users being easier to cold-start than items (Cortes, 2018).
• Markov Chain Mixing: Mixing time bounds in terms of $n$ (dimension), aspect ratio $R/r$, and initial $M$-warmness; conductance-based total-variation bounds (Narayanan et al., 2022).
• Clustering-based activity estimation: Probability of correct cluster assignment $P_n$ and Davies–Bouldin index as a function of revealed ratings; minimal ratings needed for stable assignment empirically at $N_{\min}\approx 50$ (MovieLens) or $68$ (Jester) (Visnovsky et al., 2021).

5. Theoretical Guarantees and Limitations

Cold-start strategies vary in theoretical depth:

• CDF-cold (Fatemi et al., 2023) offers empirical improvements but does not quantify uncertainty in imputed series or adopt probabilistic regularizers; causal mask is only enforced structurally in the GNN.
• FilterLLM (Liu et al., 24 Feb 2025) provides distributional outputs for user–item engagement probability but does not model cold-new user onboarding; scaling to billions of tokens in Transformer output is nontrivial.
• CMF (Cortes, 2018) and offsets formulations are model-based and provide ridge-regression guarantees, balancing cold-start accuracy with computational efficiency, though at some expense to warm-start performance.
• Cold-start mixing analysis (Narayanan et al., 2022) establishes rigorous polynomial mixing-time bounds for multiscale Metropolis chains and the coordinate hit-and-run walk. Dependence on $n$, $R/r$, and the necessity of $\ell_p$-distance oracles for boundary localization are identified as potential bottlenecks.
• Minimal activity estimation (Visnovsky et al., 2021) formalizes the required rating count $N_{\min}$ for cluster stabilization based on centroid separation, but may only approximate true dynamic cluster behavior.

6. Practical Implementation and Deployment

In production-scale systems, cold-start probability analysis is implemented via scalable architectures:

• FilterLLM was deployed in Alibaba to serve cold-start recommendations for over one billion items daily, demonstrating online gains in pageviews, click-through rate, and gross merchandise value relative to established baselines, with a 97% reduction in inference time (Liu et al., 24 Feb 2025).
• CMF/offsets formulations enable real-time scoring for new users or items by leveraging pre-computed matrix factorizations and attribute maps; constant-time evaluation per entity is achievable (Cortes, 2018).
• CDF-cold manages cold-series forecasting for hundreds of attributes and daily snapshots in data center network traffic, with key improvements in total traffic prediction error (Fatemi et al., 2023).
• Cold-start mixing in convex sampling (Narayanan et al., 2022) advances theoretical understanding necessary for efficient randomized algorithms in high-dimensional geometry and Bayesian inference.

7. Open Challenges and Research Directions

Key unresolved issues and areas for further exploration include:

• Incorporating uncertainty or calibrated posterior distributions for cold-start imputation, especially in deep time-series models (Fatemi et al., 2023).
• Scaling softmax prediction to handle token vocabularies with cardinality $10^9$–$10^{10}$; mixture-of-softmax or hierarchical strategies are needed (Liu et al., 24 Feb 2025).
• Extending text-to-distribution paradigms for new user (cold-consumer) cold-start in recommendation (Liu et al., 24 Feb 2025).
• Dynamic adaptation of clustering as new users arrive, and optimizing active query strategies to reduce $N_{\min}$ for stable assignment (Visnovsky et al., 2021).
• Further development of geometric and coupling-based proofs in Markov chain mixing from cold starts, and application to other classes of random walks (Narayanan et al., 2022).

Cold-start probability analysis continues to evolve, integrating causal inference, scalable generative modeling, and geometric sampling to address the inherent uncertainty and computational bottlenecks associated with sparse initial data across domains.