RedOne 2.0: Tailored LLMs for Social Networks

• RedOne 2.0 is a domain-specific large language model that uses a three-stage pipeline—exploratory RL, targeted fine-tuning, and refinement—to address dynamic SNS challenges.
• It leverages a 4B-parameter Qwen3 backbone to handle heterogeneous workloads, rapidly shifting slang, and multilingual data while mitigating the ID/OOD performance trade-off.
• The approach achieves superior SNS task performance and data efficiency compared to larger models, reducing compute costs and lessening catastrophic forgetting.

RedOne 2.0 is a domain-specific LLM post-training approach tailored for the operational demands of social networking services (SNS). As contemporary SNS platforms demand adaptability to heterogeneous workloads, non-stationary language phenomena, and high linguistic diversity, conventional supervised fine-tuning (SFT) pipelines face pronounced trade-offs between in-distribution (ID) performance and out-of-distribution (OOD) robustness—especially at compact model scales. RedOne 2.0 addresses these challenges with a data-efficient, three-stage post-training pipeline emphasizing RL-based exploration, targeted repair, and refinement, producing a 4B-parameter architecture that delivers high task performance and strong generalization under distribution shift.

1. Motivations and Domain Challenges

SNS platforms present four critical challenges for LLM post-training:

• Heterogeneous Workloads: SNS models serve disparate pipelines, including moderation, creator assistance, dialogue and recommendation systems, and community management—each with unique input-output formats, latency constraints, safety profiles, and stylistic considerations.
• Rapidly Shifting Norms and Slang: The introduction and obsolescence of memes, hashtags, and in-group vernacular induce sharp non-stationarities in data distributions, outpacing standard LLM adaptation cycles.
• Multilingual, Diverse Corpora: Global SNS platforms aggregate informal, mixed-script, code-switched, and highly variable linguistic data, increasing the complexity of the learning signal.
• Catastrophic Forgetting and Distribution Shift: SFT on SNS-specific data may lead to overfitting to ID benchmarks and severe forgetting of general capabilities, manifesting as the "seesaw" between domain gain and OOD robustness.

RedOne 2.0 responds by structuring model post-training into an RL-prioritized, staged curriculum capable of aligning domain-specific proficiency without sacrificing general-domain stability.

2. Model Architecture and Post-Training Stages

2.1 Model Backbone

RedOne 2.0 employs a Qwen3-4B transformer backbone with 32 layers and 4 billion parameters, a maximum attention context length of 18,192 tokens, and inherent general instruction-following capabilities.

2.2 Three-Stage Post-Training Pipeline

Each stage initializes the next, composing learning signals systematically:

Stage I: Exploratory Learning

• Data: $\mathcal{D}_{\mathrm{SNS}_1}$ (750K SNS examples) and $\mathcal{D}_{\mathrm{GEN}_1}$ (50K general examples with rationales), aggregated as $\mathcal{D}_1$.
• Objective: Domain alignment via DAPO-style RL, where multiple generations per prompt are sampled, scored for task-specific reward $\mathcal{R}_i$, and the DAPO surrogate loss is optimized:

$$\mathcal{L}_{\rm DAPO}(\theta) = \mathbb{E} \left[\frac{1}{\sum_{i,t}1}\sum_{i=1}^{G}\sum_{t=1}^{|O_i|} \min\left(r_{i,t}(\theta)\hat A_{i,t}, \mathrm{clip}(r_{i,t}(\theta),1-\varepsilon_l,1+\varepsilon_h)\hat A_{i,t}\right) \right]$$

where

$$r_{i,t}(\theta) = \frac{\pi_\theta(O_{i,t}|Q,O_{i,<t})}{\pi_{\theta_{\rm old}}(O_{i,t}|Q,O_{i,<t})}, \quad \hat A_{i,t} = \frac{\mathcal{R}_i - \overline{\mathcal{R}}}{\mathrm{Std}(\{\mathcal{R}_j\})}$$

• Reward Design: Task-dependent; e.g., exact match, BLEU/chrF++, execution success, or pattern matching.

Stage II: Targeted Fine-Tuning

• Data: $\mathcal{D}_2$: 1.7M SNS "failure" exemplars post-Stage I, plus 0.1M general-domain examples—mixed at $\alpha = 0.0556$.
• Objective: SFT to close identified gaps and mitigate forgetting:

$$\mathcal{L}_{\rm SFT} = -\mathbb{E}_{(Q,A)\sim\mathcal{D}_2} \sum_{t} \log \pi_\theta(A_t|Q,A_{<t})$$

Stage III: Refinement Learning

• Data: 400K curated SNS examples (57.18% with rationales).
• Objective: Re-apply DAPO-RL, using the same hyperparameters as Stage I, to consolidate and harmonize improvements.

Training Hyperparameters:

Stage	Data	Steps/ Epochs	Batch	LR / Clip
Exploratory (RL)	750K SNS + 50K Gen	500 steps	1,024 prompts ×16 decodes	$5\times10^{-6}$, $\varepsilon_l=0.2$, $\varepsilon_h=0.28$
Targeted (SFT)	1.7M SNS + 0.1M Gen	2 epochs	64 seqs (pack16K)	$5\times10^{-6}$ (cosine, 10% warmup)
Refinement (RL)	400K mixed	500 steps	same as Stage I	same as Stage I

All stages employ AdamW optimizer (weight decay 0.1), with sequence lengths capped at 18,192 tokens and an overlength penalty.

3. Evaluation Framework and Baselines

3.1 Benchmarks and Metrics

Three evaluation suites:

• General-Bench: Knowledge reasoning (MMLU, CMMLU, C-Eval), math (GSM8K, MATH500, AIME 2025), code (HumanEval, MBPP, LiveCodeBench), translation (WMT22–24, FLORES), instruction following (IFEval), hallucination detection (HaluEval). Aggregation via OpenCompass; metric: average percentage.
• SNS-Bench: 6,658 cases across 8 SNS tasks (taxonomy, hashtag suggestion, alignment, MRC, NER, gender appeal, highlighting, query generation). Metric: exact match accuracy.
• SNS-TransBench: 2,858 SNS-style English↔Chinese translation examples; metrics: BLEU, chrF++.

3.2 Baseline Models

• Proprietary LLMs: GPT-4o, Gemini-2.0, Claude-3.7.
• Popular Opensource LLMs: Qwen3-4B/8B/30B, GLM-4.5/9B/32B, LLaMA-3, Mistral, Mistral-Small-24B, InternLM3-8B, Phi-4-14B, GPT-OSS, DeepSeek.
• RedOne (7B): SFT-based pipeline for SNS, predecessor to RedOne 2.0.

4. Empirical Results

Main results for 4B-scale LLMs:

Model	General-Bench	SNS-Bench	SNS-TransBench
Qwen3-4B (base)	69.80	51.81	38.22
RedOne 2.0 (4B)	70.80	67.57	47.67
RedOne (7B)	63.83	66.88	48.11

• Domain uplift: RedOne 2.0 (4B) surpasses RedOne (7B) by 0.69 pp on SNS-Bench and 6.97 pp on General-Bench.
• Base improvement: Over Qwen3-4B, RedOne 2.0 yields +1.00 pp (General), +15.76 pp (SNS), +9.45 pp (Trans), averaging +8.74 pp.
• Data efficiency: RedOne 2.0 achieves these gains using < 50% of the SNS-domain data of RedOne (2M vs. 4.5M examples), with:

$$\text{Efficiency} = \frac{|\mathcal{D}_{\rm R2.0}|}{|\mathcal{D}_{\rm RedOne}|} < 0.5, \quad \text{Gain}_{\rm R2.0} > \text{Gain}_{\rm RedOne}$$

• Scaling: Per-task and scaling analyses (see Figure 1 in the source) show smooth improvements up to the 30B parameter range, without overfitting.

5. Analysis of Training Dynamics and Trade-offs

• Trade-off Stabilization: Stage I RL exploration prevents early overspecialization, Stage II low-α mixing ($\alpha \approx 5.6\%$) curtails forgetting, and Stage III RL harmonizes residual task gaps. Ablation studies (see Table 3 in the source) indicate the necessity of all three stages for optimal balance; omitting any stage degrades performance on either General-Bench or SNS-Bench.
• Cost-effectiveness: At 4B parameters, RedOne 2.0 matches or surpasses the SNS-task performance of models >30B parameters, achieving substantial computational and memory cost reductions (1/8th the compute, 1/10th the size vs. proprietary models such as GPT-4o).
• Stability and Robustness: In contrast to pure SFT-driven pipelines, which demonstrate a pronounced "seesaw" in ID/OOD trade-off, RedOne 2.0 consistently achieves ID performance gains without significant OOD regression. Stability is substantiated by scores on MMLU-Pro, AIME 2025, and LiveCodeBench benchmarks.

6. Practical Implications and Data Efficiency

RedOne 2.0 establishes a viable baseline for SNS-specific LLM deployment at compact scales, achieving:

• Rapid Domain Adaptation: The RL-prioritized exploratory curriculum facilitates fast alignment to shifting SNS language phenomena.
• Robust Generalization: Controlled gap-patching and RL-based refinements ensure retention of general capabilities amid domain optimization.
• Superior Data Efficiency: Equivalent or superior in-domain gains are achieved with significantly fewer SNS-specific examples, reducing both compute resource and curation costs.

This suggests that progressive, RL-prioritized post-training may represent an effective paradigm for other high-drift, heterogeneous language domains where SFT alone exacerbates ID/OOD trade-offs. A plausible implication is that mixing small fractions of general-domain data during SFT (as in Stage II) may be generally beneficial for compact LLMs undergoing domain specialization.

In sum, RedOne 2.0 demonstrates robust, efficient, and scalable post-training methods for LLMs in dynamic, multilingual SNS environments, advancing both methodology and practical deployment readiness in resource-constrained settings.