360Brew V1.0: Unified RecSys Model

• 360Brew V1.0 is a unified 150B-parameter recommendation model that verbalizes structured data via natural language, streamlining task coverage.
• It leverages a Mixtral decoder-only transformer with MoE capabilities, eliminating manual feature engineering through prompt optimization.
• The model consistently matches or exceeds legacy systems across 30+ tasks, reducing technical debt and enabling seamless cross-domain application.

360Brew V1.0 is a 150B-parameter decoder-only foundation model designed as a unified solution for ranking and recommendation tasks. Built atop a Mixtral family transformer architecture and operationalized through a natural language interface, it verbalizes structured data—such as member profiles, job descriptions, historical interactions, and social graphs—allowing direct reasoning on textual representations. Trained on LinkedIn's large-scale first-party datasets, 360Brew V1.0 achieves performance comparable to or exceeding legacy production systems across more than 30 predictive tasks, all without task-specific fine-tuning. Its design centralizes modeling, eliminates manual feature engineering, and significantly reduces technical debt traditionally accumulated in large-scale recommendation platforms.

1. Model Architecture and Data Representation

360Brew V1.0 is implemented using a decoder-only transformer architecture, specifically based on the Mixtral open-source model family. In its largest configuration, Mixtral operates as a Mixture-of-Experts (MoE) transformer (e.g., Mixtral 8×22B), comprising solely a stack of transformer decoder layers. All inputs—ranging from member profiles and job descriptions to interaction history and social connections—are systematically verbalized into natural language strings. This approach enables the model to process structured, tabular, or graph-based data as unified text segments, facilitating direct token-level output for prediction tasks.

The model is trained to approximate joint probability distributions over member profiles and temporal sequences of interactions: $P(m, (e_1, i_1), \dots, (e_T, i_T))$ where $m$ represents a member’s profile in textual format, and each $(e_t, i_t)$ denotes the $t$th historical entity and its associated interaction. This probabilistic formulation operationalizes next-token prediction and allows for flexible in-context learning across task domains.

2. Unified Modeling and Task Generalization

360Brew V1.0 obviates the need for hundreds or thousands of specialized models commonly deployed across ranking and recommendation surfaces. Instead, it serves as a foundation model that can manage heterogeneous predictive workloads spanning clicks, likes, job applications, skill suggestions, and feed re-ranking. Its decoder-only architecture confers robust reasoning and few-shot adaptation capabilities, enabling seamless generalization to novel recommendation domains—whether in-domain or out-of-domain—without requiring task-specific fine-tuning.

By consolidating all tasks within a single parameter space, maintenance overhead is substantially reduced and the system’s extensibility to new predictive surfaces is markedly increased. This centralization is especially pronounced when compared to legacy architectures where complex DAGs of feature and model dependencies are maintained at scale.

3. Natural Language Interface and Prompt Engineering

All input and output interactions with 360Brew V1.0 are mediated via natural language prompts. Both task definitions (e.g., “Predict which jobs this member is likely to apply to”) and member behaviors (“Member clicked X, dismissed Y, liked Z…”) are expressed textually. The model directly predicts tokenized actions (“apply”, “click”, “dismiss”), bypassing the need for manual ID-based feature engineering.

This paradigm enables developers to define tasks using human-readable instructions, shifting engineering focus from traditional feature extraction to prompt optimization. It also facilitates end-to-end reasoning over social graphs, behavioral histories, and meta-data, supporting more nuanced and combinatorial recommendation tasks. The elimination of elaborate DAG-based dependencies marks a significant change in operational requirements for large-scale ranking systems.

4. Evaluation Protocols and Performance Metrics

360Brew V1.0 is evaluated on multiple axes, primarily using offline metrics derived from predicted binary outcomes. For example, the likelihood that a member will “like” a post is inferred by computing logit scores over candidate tokens, aggregating these into probability estimates and calculating metrics such as AUC. These metrics are computed over both:

• In-domain (T1) tasks, where training data is well-represented
• Out-of-domain (T2) tasks, such as novel skill suggestions, where model generalization capabilities are probed

Across the spectrum of 30+ predictive tasks, the model consistently matches or exceeds the best performance benchmarks set by extant production systems. Further robustness is measured using “needle-in-a-haystack” (NIAH) retrieval tests, where the model’s long-context retrieval is assessed via extended prompts, and via temporal generalization studies that ascertain model effectiveness across evolving data distributions.

5. Training Strategy and Technical Workflow

The training schema for 360Brew V1.0 is structured in sequential phases:

• Continuous Pre-Training (CPT): Utilizing LinkedIn’s proprietary data (member, job, graph, interaction), with inputs verbalized using varied templates and context packed to maximize 16K token window utilization. This stage prioritizes coverage and statistical diversity in the model’s contextual understanding.
• Instruction Fine-Tuning (IFT): Conditioning the model for improved compliance with human instructions and prompt-driven zero-/few-shot generalization. Data sources include proprietary sets and open-source corpora (e.g., UltraChat). Preference alignment algorithms such as DPO/ORPO are applied to further calibrate outputs.
• Supervised Fine-Tuning (SFT): Training on a Multi-Turn Chat (MTC) format with alternating “<member>” and “<assistant>” roles, using a weighted loss: $$L_{\text{SFT}} = w_p \cdot L_p + (1-w_p) \cdot L_{\text{mtc}}$$ where $L_p$ is the auto-regressive prompt loss and $L_{\text{mtc}}$ is a masked loss on assistant responses. Empirically, $w_p \approx 10^{-2}$ is effective.

Training protocols include distributed learning via PyTorch Lightning and Full Sharding Data Parallelism (FSDP), mixed precision (bf16 for model weights, fp32 for updates), and attention optimizations (FlashAttention v2), permitting efficient scaling to hundreds of GPU units.

6. Deployment Scenarios and General Applicability

Within LinkedIn, 360Brew V1.0 serves as a unified backbone for diverse recommendation surfaces: job suggestions, feed ranking, member engagement prediction, and skill inference are all addressed by the same foundation model. The model’s generalized learned representations allow effortless transition between content types and interaction modalities.

A plausible implication is the feasibility of porting this approach to other recommendation-intensive domains, such as e-commerce and social platforms. The model’s reliance on universal prompt engineering and its capacity for in-context learning suggest superior diffusibility and maintainability beyond traditional vertical silos.

7. Impact and Future Directions

360Brew V1.0 fundamentally alters the maintenance and extension paradigm for recommendation systems. By unifying task coverage within a large-scale decoder-only transformer and leveraging natural language interfaces, it minimizes technical debt associated with feature engineering and DAG dependency management. Offline evaluation demonstrates that a single model can reach or surpass the accuracy of mature, hand-crafted systems without specialized retraining.

The ongoing research trajectory suggests further exploration of scaling, context comprehension, and application specificity may yield additional efficiency and generalization gains. Continued integration of preference alignment and advanced prompt methodologies is likely to enhance future iterations’ applicability across global recommendation and ranking challenges.