TaoSR1: Taobao Search Relevance Model v1
- TaoSR1 is an LLM-based generative model designed for e-commerce query-product relevance, integrating CoT supervision with direct online deployment.
- It addresses key challenges such as deployment latency, error accumulation, and discriminative hallucination through supervised fine-tuning, DPO, and GRPO.
- It introduces CumPT for streamlined online tiering, achieving improved macro-F1 scores and robustness in handling diverse, long-tail queries.
TaoSR1, short for Taobao Search Relevance Model v1, is an LLM-based “thinking model” for e-commerce relevance search that is designed to directly deploy a generative LLM online for query-product relevance classification (Dong et al., 17 Aug 2025). It is presented as a deployment-ready paradigm for relevance classification in which the LLM remains the final online model, rather than being used only for discriminative fine-tuning or distilled back into a smaller encoder. The framework targets the practical difficulties of applying chain-of-thought (CoT) reasoning to search relevance, specifically deployment latency, CoT error accumulation, and discriminative hallucination, and organizes its solution around supervised fine-tuning with CoT, offline pass@N sampling with Direct Preference Optimization (DPO), difficulty-based dynamic sampling with Group Relative Policy Optimization (GRPO), and online deployment via post-CoT and cumulative probability-based tiering (Dong et al., 17 Aug 2025).
1. Task definition and relevance taxonomy
TaoSR1 addresses query-item relevance prediction in an e-commerce search engine. Given a query and a candidate product, the model predicts one of four relevance labels: 4-Excellent, 3-Related, 2-Mismatch, and 1-Irrelevant (Dong et al., 17 Aug 2025). In the Taobao setup described in the paper, this is not limited to lexical or shallow semantic matching; it must also handle negation queries, “alternative to X” queries, question-answering queries, knowledge-based queries, and long-tail semantic cases requiring business-rule reasoning (Dong et al., 17 Aug 2025).
The four-class labeling scheme is used internally, while online serving requires a coarser three-tier partition. The production tiers are Good, Mid, and Bad, corresponding respectively to , , and (Dong et al., 17 Aug 2025). This distinction matters because the model is simultaneously a fine-grained classifier and a production ranking component.
| Four-tier label | Production tier |
|---|---|
| 4-Excellent | Good |
| 3-Related | Good |
| 2-Mismatch | Mid |
| 1-Irrelevant | Bad |
The paper emphasizes that relevance judgments may depend on asymmetric and context-sensitive business rules. Its example is that “Mate50 Pro” retrieved for query “Mate50” may be 3-Related, whereas the reverse can be 2-Mismatch (Dong et al., 17 Aug 2025). This indicates that TaoSR1 is intended for a relevance regime in which taxonomy, attributes, product granularity, and domain policy interact nontrivially.
2. Motivation and limitations of prior paradigms
The motivation for TaoSR1 begins from a limitation attributed to BERT-based relevance models. These models are described as strong for ordinary lexical and semantic matching but insufficient for difficult relevance cases because they mainly model text interaction rather than richer reasoning, because long-tail queries require explicit logic and business rules, and because the BERT-style paradigm is limited in scaling and representational ceiling (Dong et al., 17 Aug 2025).
The paper also identifies limits in prior LLM-based relevance methods. Although recent LLM approaches improve semantic understanding, most still either use discriminative optimization rather than true generative reasoning or distill back into BERT-like models for deployment (Dong et al., 17 Aug 2025). In that view, the LLM’s reasoning ability is not fully exploited online.
A further claim is that simply attaching CoT to relevance classification is insufficient. Three practical obstacles are singled out. First, deployment latency becomes prohibitive when full CoT must be generated for hundreds of candidate items per query. Second, CoT error accumulation allows hallucinations or deviations in intermediate reasoning steps to propagate into the final label. Third, discriminative hallucination can occur when the reasoning appears plausible but the final relevance label is still wrong (Dong et al., 17 Aug 2025). The framework is therefore framed not merely as a larger model, but as a response to an optimization and systems mismatch between generative reasoning and online relevance serving.
3. Generative formulation, CoT supervision, and post-CoT
At the supervised fine-tuning stage, TaoSR1 does not train a discriminative classifier. Instead, it uses a generative objective in which the model directly generates the label text, such as "4-Excellent" (Dong et al., 17 Aug 2025). The paper gives the baseline language-model objective as
$\mathcal{L}_{lm}(\pi)=\mathbb{E}_{(\mathbf{x},\mathbf{y}) \sim \mathcal{D}\left[-\sum_{t=1}^{|\mathbf{y}|-1} \log \pi\left(\mathbf{y}_{t+1} \mid \mathbf{x}_1, \ldots, y_t\right)\right].$
Here, denotes the input query-item pair and features, the output sequence, the model distribution, the training data, and the token index (Dong et al., 17 Aug 2025). The formulation is standard next-token prediction over the label text, but its use for relevance classification is central to the framework.
For deployment, TaoSR1 extracts a continuous score from the probability of the first generated token rather than relying only on the final generated label string. The paper defines
$\mathbf{P}\left(\mathbf{y}_1=c \mid \mathbf{x}\right)=\left(\mathbf{p}_1\right)_{\text{id(c)}=\frac{\exp \left(\pi(\mathbf{x}\right)_{\text{id(c)})}{\sum_{j=1}^{4} \exp \left(\pi(\mathbf{x}\right)_{\text{id}(c_j)})}.$
Its interpretation is that 0 is the class token, 1 is the first-token probability vector, and the probabilities are normalized over the four label tokens (Dong et al., 17 Aug 2025). This first-token distribution later supports online tiering.
To teach domain-specific reasoning, the training pipeline synthesizes CoT supervision using Retrieval-Augmented Generation (RAG). The structured reasoning template has five steps: Query Understanding, Product Comprehension, Category Matching, Attribute Matching, and Relevance Class Determination (Dong et al., 17 Aug 2025). Human annotation supplies atomic factors or fine-grained reasons; these atomic reasons retrieve matching business rules from a structured knowledge base; the retrieved rules are inserted into the prompt; and DeepSeek-R1 is used to synthesize CoT examples. Each resulting sample has the form
2
A notable architectural result is the comparison between think-then-respond and respond-then-think. The former outputs 3; the latter outputs 4 (Dong et al., 17 Aug 2025). The paper reports that, counterintuitively, think-then-respond performs worse because of error accumulation in the reasoning chain. The preferred design is therefore respond-then-think, called post-CoT or postCoT, in which the label is predicted first and the reasoning comes after (Dong et al., 17 Aug 2025). In context, this means the model uses reasoning as a support mechanism without forcing the classification decision to wait for a long and error-prone latent chain.
4. Preference optimization and reinforcement learning stages
The second stage of TaoSR1 exploits the gap between single-sample accuracy and pass@N accuracy. The paper reports that the model’s pass@N from multiple offline samples is much higher than its single-pass performance: LLM base: 75.01, TaoSR1(CoT) Pass@1: 67.38, Pass@2: 74.26, Pass@3: 77.68, Pass@4: 80.18, and Pass@5: 81.73 (Dong et al., 17 Aug 2025). This suggests that correct predictions are often present among sampled candidates even when the top sample is wrong.
On that basis, TaoSR1 constructs preference data for DPO from two disjoint subsets. For solvable cases where pass@N 5, one correct sample is chosen as 6 and one incorrect sample as 7, creating self-correction pairs. For hard cases where pass@N 8, DeepSeek-R1 generates a correct output, which is paired with the model’s incorrect response; approximately 50% of pass@N 9 instances are curated into this supplemental preference dataset (Dong et al., 17 Aug 2025). The paper presents the DPO objective in terms of a trainable policy 0, a reference model 1, chosen and rejected responses 2 and 3, temperature parameter 4, and sigmoid 5, with the effect of pushing the model to assign higher relative probability to chosen responses than rejected ones (Dong et al., 17 Aug 2025). The stated role of DPO is to improve correct-answer preference, robustness on hard cases, and the ability to recover from CoT process errors.
The third stage applies GRPO to address discriminative hallucination and further improve reasoning quality. The paper’s motivation is that even a seemingly good reasoning path may end in the wrong final label, and that naive group-based policy optimization wastes effort on groups that are all correct or all incorrect (Dong et al., 17 Aug 2025). TaoSR1 therefore uses difficulty-based dynamic sampling: it only backpropagates for groups whose empirical accuracy lies in
6
This excludes groups where everything is right and groups where everything is wrong, focusing optimization on partially solvable, difficult instances (Dong et al., 17 Aug 2025). The paper also reports that a balanced label distribution improves GRPO performance, and accordingly majority classes are downsampled to balance the training set.
The GRPO objective is given as
7
subject to
8
with
9
The symbols are defined in the paper as follows: $\mathcal{L}_{lm}(\pi)=\mathbb{E}_{(\mathbf{x},\mathbf{y}) \sim \mathcal{D}\left[-\sum_{t=1}^{|\mathbf{y}|-1} \log \pi\left(\mathbf{y}_{t+1} \mid \mathbf{x}_1, \ldots, y_t\right)\right].$0 is the number of sampled responses in a group, $\mathcal{L}_{lm}(\pi)=\mathbb{E}_{(\mathbf{x},\mathbf{y}) \sim \mathcal{D}\left[-\sum_{t=1}^{|\mathbf{y}|-1} \log \pi\left(\mathbf{y}_{t+1} \mid \mathbf{x}_1, \ldots, y_t\right)\right].$1 the $\mathcal{L}_{lm}(\pi)=\mathbb{E}_{(\mathbf{x},\mathbf{y}) \sim \mathcal{D}\left[-\sum_{t=1}^{|\mathbf{y}|-1} \log \pi\left(\mathbf{y}_{t+1} \mid \mathbf{x}_1, \ldots, y_t\right)\right].$2-th sampled output, $\mathcal{L}_{lm}(\pi)=\mathbb{E}_{(\mathbf{x},\mathbf{y}) \sim \mathcal{D}\left[-\sum_{t=1}^{|\mathbf{y}|-1} \log \pi\left(\mathbf{y}_{t+1} \mid \mathbf{x}_1, \ldots, y_t\right)\right].$3 token $\mathcal{L}_{lm}(\pi)=\mathbb{E}_{(\mathbf{x},\mathbf{y}) \sim \mathcal{D}\left[-\sum_{t=1}^{|\mathbf{y}|-1} \log \pi\left(\mathbf{y}_{t+1} \mid \mathbf{x}_1, \ldots, y_t\right)\right].$4 of sample $\mathcal{L}_{lm}(\pi)=\mathbb{E}_{(\mathbf{x},\mathbf{y}) \sim \mathcal{D}\left[-\sum_{t=1}^{|\mathbf{y}|-1} \log \pi\left(\mathbf{y}_{t+1} \mid \mathbf{x}_1, \ldots, y_t\right)\right].$5, $\mathcal{L}_{lm}(\pi)=\mathbb{E}_{(\mathbf{x},\mathbf{y}) \sim \mathcal{D}\left[-\sum_{t=1}^{|\mathbf{y}|-1} \log \pi\left(\mathbf{y}_{t+1} \mid \mathbf{x}_1, \ldots, y_t\right)\right].$6 the importance sampling ratio, $\mathcal{L}_{lm}(\pi)=\mathbb{E}_{(\mathbf{x},\mathbf{y}) \sim \mathcal{D}\left[-\sum_{t=1}^{|\mathbf{y}|-1} \log \pi\left(\mathbf{y}_{t+1} \mid \mathbf{x}_1, \ldots, y_t\right)\right].$7 the PPO-style clip parameter, $\mathcal{L}_{lm}(\pi)=\mathbb{E}_{(\mathbf{x},\mathbf{y}) \sim \mathcal{D}\left[-\sum_{t=1}^{|\mathbf{y}|-1} \log \pi\left(\mathbf{y}_{t+1} \mid \mathbf{x}_1, \ldots, y_t\right)\right].$8 the normalized advantage, $\mathcal{L}_{lm}(\pi)=\mathbb{E}_{(\mathbf{x},\mathbf{y}) \sim \mathcal{D}\left[-\sum_{t=1}^{|\mathbf{y}|-1} \log \pi\left(\mathbf{y}_{t+1} \mid \mathbf{x}_1, \ldots, y_t\right)\right].$9 the verifiable reward, 0 the KL regularization to the reference policy, and 1 the difficulty threshold (Dong et al., 17 Aug 2025). The reward is the verifiable correctness of the final relevance class. The reported GRPO setup uses rollout batch size 64, 16 responses per sample, temperature 0.99, top-k 100, clipping ratio 2, and advantage clipping 3 (Dong et al., 17 Aug 2025).
5. Online serving and cumulative probability tiering
A central contribution of TaoSR1 is the claim that the LLM is made deployable online (Dong et al., 17 Aug 2025). The deployable form is the post-CoT version: the model first predicts the label and then produces CoT. This serves two purposes stated in the paper. First, it avoids latency-heavy reasoning before classification. Second, it reduces error accumulation by preventing long reasoning chains from directly corrupting the label prediction (Dong et al., 17 Aug 2025).
For online tiering, the production system does not use raw labels directly. The paper contrasts a conventional score-threshold approach with its Cumulative Probability Tiering (CumPT) method. The conventional method requires multiple hyperparameters, specifically score anchors 4 and thresholds 5, with the score defined as
6
and the tier mapping
7
TaoSR1 instead uses CumPT, which requires only one hyperparameter, 8 (Dong et al., 17 Aug 2025). The model sums class probabilities from 4 downward, checks whether the cumulative sum exceeds 9, and assigns the tier accordingly. The algorithm logic is: if 0, predict Good; else if 1, still Good; else if 2, predict Mid; else predict Bad (Dong et al., 17 Aug 2025).
The paper attributes several practical advantages to CumPT: only one threshold, no complex calibration, robust online behavior, online macro-F1 tracks offline macro-F1 closely, and easier deployment and better stability (Dong et al., 17 Aug 2025). A plausible implication is that the first-token class distribution is treated as an intrinsic ranking signal rather than as an intermediate score requiring extensive post hoc calibration.
6. Experimental setting, quantitative results, and interpretation
The evaluation uses a dataset collected from Taobao online search logs with around 70,000 manually annotated query-item pairs, focused on difficult query types including negation, affordable alternatives, QA, and knowledge-based queries (Dong et al., 17 Aug 2025). The label distribution is reported as L4: 38,739 (50%), L3: 3,863 (5%), L2: 27,913 (36%), and L1: 6,865 (9%) (Dong et al., 17 Aug 2025). Offline metrics are macro-F1, per-class F1, and accuracy; online metrics are GSB (Good/Same/Bad side-by-side human evaluation), Query Goodrate, and Item Goodrate (Dong et al., 17 Aug 2025).
The baselines are BERT, Qwen3-0.6B, Qwen3-30B-A3B, and an LLM base defined as a Tbstar-42B SFT-only baseline; TaoSR1 is then evaluated in multiple variants, including TaoSR1(CoT), TaoSR1(CoT)+DPO, TaoSR1(CoT)+DPO+GRPO, and post-CoT versions of these models (Dong et al., 17 Aug 2025). Tbstar is described as a Taobao-trained MoE LLM with 42B total parameters and 3.5B active parameters (Dong et al., 17 Aug 2025).
Offline results show BERT at Macro F1 = 61.33, Accuracy = 69.36, Qwen3-0.6B at Macro F1 = 52.95, Qwen3-30B-A3B at Macro F1 = 62.01, and LLM base at Macro F1 = 62.22, Accuracy = 75.04 (Dong et al., 17 Aug 2025). Among TaoSR1 variants, TaoSR1(CoT) reaches Macro F1 = 51.54, TaoSR1(CoT)+DPO reaches 63.81, TaoSR1(CoT)postCoT reaches 60.01, TaoSR1(CoT)+DPO postCoT reaches 65.03, and TaoSR1(CoT)+DPO+GRPO postCoT reaches 67.12 (Dong et al., 17 Aug 2025). The final model therefore achieves +4.9 points macro-F1 over LLM base, and the paper states that it gives the best class-wise F1 overall in the final configuration, especially on the difficult middle classes (Dong et al., 17 Aug 2025).
These results are used to support several qualitative claims. First, CoT alone hurts classification, attributed to error propagation in naive think-then-respond reasoning (Dong et al., 17 Aug 2025). Second, respond-then-think is better, making post-CoT both an optimization choice and a deployment choice. Third, DPO is characterized as a bridge before GRPO, and the paper reports that starting GRPO from the DPO-refined checkpoint performs better than starting directly from SFT (Dong et al., 17 Aug 2025). Fourth, GRPO is presented as further reducing discriminative hallucination, especially when trained on balanced and difficulty-selected data (Dong et al., 17 Aug 2025).
The CumPT ablation compares conventional multi-threshold mapping with the cumulative method. The paper reports that formula-based thresholding at 3 yields Online Macro F1 = 65.26, whereas CumPT at 4 yields Online Macro F1 = 67.15 (Dong et al., 17 Aug 2025). The stated interpretation is that CumPT is both simpler and better aligned with the model’s intrinsic distribution.
Online human evaluation is conducted on 2,000 queries. TaoSR1 improves side-by-side judgments, with GSB gains of +16.62% on QA, +34.43% on Alternative, +10.92% on Negative, and +18.45% on Knowledge queries (Dong et al., 17 Aug 2025). Positive lifts in Query Goodrate and Item Goodrate are also reported across all categories, with especially strong gains on alternative queries, which the paper describes as notoriously difficult for lexical matching (Dong et al., 17 Aug 2025).
Taken together, the paper presents TaoSR1 as a complete deployment-ready paradigm for applying CoT to relevance classification: it keeps the LLM generative, uses CoT to encode domain business logic, addresses the mismatch between reasoning and classification via post-CoT, exploits offline multiple-sample success through pass@N + DPO, targets discriminative hallucination through difficulty-aware GRPO, and introduces CumPT for online multi-tier relevance calibration (Dong et al., 17 Aug 2025). This suggests that the system’s novelty lies less in any single optimization component than in the coordination of reasoning supervision, preference learning, reinforcement learning, and serving-time calibration into a unified relevance pipeline.