LLM-Based User Personas for Recommendations at Scale

Abstract: LLMs offer unprecedented potential for enhancing recommendation systems through their world knowledge and reasoning capabilities. However, existing approaches often rely on structured IDs or offline processing, limiting semantic richness, real-time adaptability, and user-facing interpretability. In this paper, we introduce a novel framework that enables real-time generation of LLM-based user interest personas for a large-scale commercial video recommendation platform. Our method generates natural-language user interest personas that address the exploitation-exploration trade-off by combining the summarization of existing interests with novel topics, directly during serving. To overcome the computational challenges of online LLM inference at a billion-user scale, we design a cost-efficient architecture leveraging knowledge distillation, asynchronous inference, and input optimization via semantically clustered video representations. Extensive offline evaluations, user studies, and live A/B tests demonstrate significant improvements in viewer value. This work bridges the gap between high-level semantic understanding and industrial-scale recommendation, paving the way for more dynamic, explainable, and satisfying personalized experiences.

• The paper demonstrates that LLM-generated natural language personas improve recommendation quality, with 57% of users preferring them over traditional ID-based models.
• It utilizes asynchronous inference, hierarchical planning, and a knowledge distillation protocol to achieve real-time, scalable persona generation at billion-user scale.
• Quantitative evaluation shows measurable gains including a +0.04% increase in watch time and +0.03% in daily active users, with exploratory recommendations 13.6% more likely to be watched.

LLM-Based User Personas for Scalable Recommendations: Framework, Evaluation, and Implications

Introduction

"LLM-Based User Personas for Recommendations at Scale" (2606.12198) introduces a real-time framework for constructing and deploying natural language–driven user personas, generated by LLMs, into the serving path of a commercial-scale video recommender system. The approach addresses semantic limitations and recency bias of legacy ID-based models, while introducing new architectures and distillation protocols for tractable live deployment at unprecedented scale. Contrasts to prior work include both the use of open-ended natural language personas (instead of cluster or ID outputs) and the system’s ability to generate, serve, and continually update these personas efficiently in production environments.

Framework Overview

The paper’s system targets the dual objectives of exploitation and exploration in recommendations by extracting from user interaction histories two persona components per user: (1) summarized interests, capturing established affinities, and (2) exploration interests, proposing novel yet coherent directions beyond the user’s historical patterns. These are distilled into concise natural language texts via LLM prompt engineering, leveraging world knowledge and reasoning capabilities for semantic nuance and diversity.

Figure 1: Training data collection pipeline leveraging multi-step LLM reasoning to distill both summarized and exploratory topics from semantically clustered user watch histories.

Operationally, a hierarchical planning paradigm is employed. A high-level LLM generates textual user personas, and a downstream item retrieval model grounds these personas in item space, using restricted nearest neighbor search within a semantically filtered candidate set.

Scalable Inference Architecture and Distillation

A primary challenge is maintaining real-time LLM persona generation at billion-user scale, subject to latency and cost constraints. The solution introduces an asynchronous online inference mechanism: persona computation is decoupled from direct user requests and is executed in background jobs that update a database, with frequency and user history window size tunable by system operators.

Figure 2: Asynchronous online inference architecture enabling scalable persona generation, retrieval, and safe serving decoupled from latency-critical user traffic.

To make LLM inference economically viable, the authors design a knowledge distillation regimen. Large Gemini teacher models (notably Gemini Pro) are used to create high-fidelity training data via a multi-step chain-of-thought protocol; compact student models (Gemini Nano and Flash) are then supervised on this data to reproduce both high-quality summarization and creative exploration interests. Model quantization is applied to further reduce serving costs.

User Representation: Semantic Clustering and Input Structuring

The paper empirically demonstrates that text-based, semantically clustered representations of user histories (using salient terms extracted from metadata) yield higher persona quality than either raw ID sequences or low-level audio-visual clustering. Structured input—specifically, grouping watched videos into concept clusters—materially improves the specificity and depth of generated personas.

Figure 3: Offline evaluation of user representation strategies, demonstrating large performance gains for semantically clustered, title-based inputs on summarization BLEURT metrics.

Larger teacher models consistently outperform smaller architectures on summarization BLEURT and creativity measures until a performance plateau, motivating the adoption of distillation to transfer these capabilities to cost-effective inference models.

Quantitative and Qualitative Evaluation

Extensive user studies, offline metrics, and live production A/B experiments support the following core findings:

• Persona Quality: Over 80% of user study participants judged LLM personas as highly accurate, and 71% expressed strong willingness to consume additional recommended content derived from these interests. In direct comparison, 57% of users preferred LLM personas over knowledge-graph–based baselines.
• Live Impact: Deployment on production traffic—serving billions—yields statistically significant, albeit numerically modest, metric lifts: +0.04% in watch time and +0.03% in daily active users, both $p<0.05$ (translating to substantial absolute increases given scale).

Figure 4: Time-series of live production metrics showing persistent improvement in watch time and DAUs attributable to LLM persona deployment.

• Exploration Efficacy: Exploration-based recommendations, though surfaced less frequently due to ranking system conservatism, are 13.6% more likely to be watched conditional on being shown, confirming that the exploratory personas drive genuine discovery.
• User Segmentation: Gains are concentrated among casual users, presumably due to improved cold-start generalization and superior handling of sparse activity histories.

Architectural and Methodological Implications

This work advances several fronts in industrial recommender systems:

• Interpretability and User-Facing Applications: By generating natural language interests, the system unlocks direct user-facing features (e.g., profile summaries), increases transparency, and facilitates richer feedback and personalization loops.
• Feedback Loop Mitigation: Explicit inclusion of novel, LLM-proposed interests counteracts entrenchment effects typical of legacy exploitation-focused pipelines. The modular structure allows for future integration of interactive user-in-the-loop mechanisms.
• Scalable LLM Integration: The decoupled asynchronous design, together with distillation and quantization, offers a template for deploying LLM-driven semantics in cost- and latency-sensitive environments—demonstrating real, live benefits beyond academic benchmarks.
• Safety and Robustness: The inclusion of real-time output safety filtering and fallbacks ensures that persona generation remains robust and user-appropriate even at massive scale.

Future Directions

Open directions include extension of personas to richer multimodal and cross-domain signals, continuous refinement via user feedback, and integration into novel interaction paradigms within recommenders (e.g., user-guided retrieval, dialog-based exploration). Methodologically, improved methods for efficient reasoning distillation and more dynamic balancing of exploitation and exploration (including adaptive update rates based on user state) could further enhance both user satisfaction and platform value.

Conclusion

This work establishes the feasibility and utility of real-time, LLM-generated user personas in production recommendation stacks. Through innovations in semantic representation, input structuring, hierarchical planning, and scalable LLM deployment via distillation, the system achieves measurable improvements in both engagement and discovery at industrial scale. The approach facilitates richer, interpretable, and more adaptive recommendation pipelines, providing a foundation for next-generation personalized content delivery.