AdNanny: Unified Offline Ad LLM

• AdNanny is a unified, reasoning-centric LLM designed for diverse offline advertising tasks such as query-ad relevance, ranking, and recommendation.
• It leverages a fine-tuned 671B-parameter model with hybrid dense–MoE parallelism and reasoning-augmented supervision to improve label accuracy and interpretability.
• Adaptive multi-task supervised fine-tuning combined with reinforcement learning alignment boosts key metrics like recall@K and predicted CTR, streamlining production deployment.

AdNanny is a unified, reasoning-centric LLM architecture specifically designed to serve as a backbone for diverse offline advertising tasks such as retrieval, ranking, recommendation, query-ad relevance labeling, keyword-based query generation, and user profiling. Built on a fine-tuned 671B-parameter DeepSeek-R1 checkpoint, AdNanny employs hybrid dense–Mixture-of-Experts (MoE) parallelism, receives supervision from reasoning-augmented corpora, and is aligned via both supervised learning and direct optimization of downstream advertising metrics. Its deployment in production environments, notably within Bing Ads, has resulted in significant reduction of maintenance costs and manual effort by consolidating multiple task-specific models into a single, high-capacity foundation model (Hu et al., 2 Feb 2026).

1. Model Architecture and Parallelism

AdNanny’s architecture is anchored in the DeepSeek-R1 public checkpoint containing 671 billion parameters, with each token activating a sparse MoE capacity of 37 billion parameters. The model sequentially interleaves standard dense Transformer layers and Mixture-of-Experts MLP blocks. Each MoE block contains 256 experts (distinct feed-forward sub-networks), alongside a compact gating "router" layer that computes softmax-based routing scores $g = \text{softmax}(W_g h) \in \Delta^{256}$ for input token representations $h \in \mathbb{R}^d$. Only the top-1 or top-2 experts per token are utilized per forward pass, supplemented by a "shared" overflow expert.

Training leverages a Megatron-based parallelism schema comprising:

• Pipeline parallelism: The model is partitioned into 31 pipeline stages, each hosting approximately 1–2 Transformer layers per GPU node.
• Expert parallelism (EP): 256 MoE experts are contiguous-sharded over 8 GPUs (e.g., experts 0–31 on rank 0; 32–63 on rank 1, etc.), with the shared expert replicated across GPUs.
• Data parallelism (DP): Applied across the 8-GPU group for dense parameters and shared experts.

This hybrid parallelism approach enables practical training of ultra-large models, minimizing cross-node communication for MoE routing and balancing memory-compute footprints.

2. Reasoning-Augmented Data and Supervision Pipeline

AdNanny departs from classical discrete-label supervision by utilizing reasoning-augmented corpora. Each training tuple is constructed as ⟨input, label(s), reasoning⟩, where the reasoning—explicit, step-by-step natural language explanations—is generated by prompting DeepSeek-R1 itself. For instance, a query–ad relevance tuple:

• Query: "cheap hotel in downtown Paris"
• Ad title: "Luxury 5-star hotel in central Paris"
• Labels: {Location=Good, Relevance=Fair, Quality=Good}
• Reasoning: > The user wants an inexpensive stay in central Paris. The ad matches location but not price, so relevance is only fair…

Construction of the datasets follows a multi-stage "data-cooking" pipeline:

1. Reasoning Generation: Prompt DeepSeek-R1 with existing labels or rule-based outputs to elicit stepwise explanations.
2. Golden-Set Validation: Prompts are iteratively refined on trusted subsets to guarantee logical faithfulness, correct formatting, and comprehensive coverage.
3. Rejection Sampling & Filtering: Multiple candidate explanations are sampled; consistency and adherence to schema are verified via embedding-based and explicit filters; hallucinated or boilerplate responses are discarded.

This methodology is systematically extended across offline advertising tasks such as query–ad relevance, ad–user relevance, keyword-to-query generation, model evaluation, user profiling, and ad optimization, enforcing a "label+reasoning" schema as the supervisory backbone.

3. Supervised Fine-Tuning and Reinforcement Learning Alignment

AdNanny employs a two-stage alignment pipeline:

3.1 Multi-Task Supervised Fine-Tuning (SFT):

Reasoning-augmented data across tasks $\mathcal{T}$ is pooled and optimized under an autoregressive cross-entropy loss. The total supervised loss incorporates both instance-level and task-level adaptive weighting:

$$L_{\text{SFT}}(\theta) = \sum_t \lambda_t \cdot \mathbb{E}_{i \sim D_t} [w_{t,i} \ell_{t,i}(\theta)]$$

where sample weights $w_{t,i}$ and task weights $\lambda_t$ dynamically adjust by measuring learning progress ($\Delta_{t,i}$ in perplexity) and validation-metric difficulty ($d_t = 1 - M_t$), respectively. Samples and tasks with less improvement or lower validation scores are upweighted to encourage progress on harder items.

3.2 Reinforcement Learning (RL) Alignment:

Following SFT, the model is viewed as a policy $\pi_\theta$. For each task $t$, batches of model-generated labels and explanations are scored via offline advertising utility metrics (e.g., recall@K, predicted CTR lift). The metric delta $h \in \mathbb{R}^d$0 is assigned as reward to all outputs in the batch and used to optimize a KL-regularized Generative Reward Policy Optimization (GRPO) objective:

$h \in \mathbb{R}^d$1

This phase directly steers generation towards outputs that improve downstream retrieval and ranking utility.

4. Empirical Results and Deployment

AdNanny’s performance is characterized across several dimensions:

• Query–Ad Relevance:

Across six internal datasets covering various locales and defect rates, AdNanny achieves balanced accuracy (BACC) improvements of +3–10 percentage points (pp) on "bad-defect," +2–9 pp on "fair-defect," +4–14 pp on location labeling, and +2–7 pp on quality labeling relative to a 4-layer 1120-dim baseline.

• Ad–User Relevance:

On two golden ad–user datasets (RR=32%/42%), BACC increases by +3.6 pp / +3.1 pp and AUC by +1.4 pp / +2.5 pp. Model relevance ranking aligns more closely with ground truth.

• Keyword-Based Query Generation:

On an internal suite, relevance improves from 39.33 to 53.65, diversity from 57.32 to 66.49, and difficulty from 69.31 to 82.85 when progressing from DeepSeek‐R1 to AdNanny‐671B. A distilled AdNanny‐7B further increases these metrics above the teacher model.

• Public-Benchmark Generalization:

SFT on ads-only datasets results in a drop from 0.867 to 0.819 averaged over 20 public tasks; supplementing with general instruction data restores the average to 0.867.

• Efficiency and Cost:

FP8 quantization at inference yields a marginal BACC improvement (83.01 → 83.38) while reducing compute and memory demands by half compared to BF16. For query–ad labeling, AdNanny’s cost is $h \in \mathbb{R}^d$2307 per 1M for label+reasoning (0.23× and 0.52× of GPT-4o’s cost, respectively). Throughput reaches 20 samples/s (label only) and 8.8 samples/s (label+reasoning). Production deployment in Bing Ads has replaced numerous smaller models and human labeling pipelines, reducing operational overhead.

5. Key Insights, Limitations, and Prospects

Unifying multiple offline advertising tasks via a single "label + reasoning" LLM reduces engineering complexity and supports consistently high performance across a spectrum of recommendation and evaluation modalities. Reasoning-augmented supervision is found to expedite both accurate label learning and the production of interpretable justifications. Adaptive sample and task reweighting is effective in balancing training dynamics across heterogeneous data and task difficulties. The staged SFT+RL pipeline secures alignment first to human judgment, then to explicit downstream business metrics.

Limitations arise from the bounded expressiveness and task coverage of available reasoning-augmented corpora; RL alignment optimizes readily available offline proxies (such as recall@K or offline CTR lift) rather than long-term online objectives. Future research directions include:

• Broadening domain and locale coverage (e.g., to policy checking, campaign diagnostics).
• Enhancing alignment with live A/B test outcomes to close the proxy–reality gap.
• Systematic distillation into compact student models or adapters tailored for specific locales/tasks.
• Incorporation of internal retrieval or policy tools within the reasoning loop to enhance factual grounding and system governance.

Collectively, AdNanny demonstrates that a properly parallelized, reasoning-supervised, and RL-aligned ultra-LLM can serve as a coherent platform for offline advertising tasks, subsuming the maintenance burden and inconsistency issues prevalent with narrowly specialized models, while delivering measurable gains in accuracy, cost efficiency, and interpretability (Hu et al., 2 Feb 2026).