Search Preference Optimization for E-commerce

• Search Preference Optimization is a framework that reformulates e-commerce retrieval by generating structured multi-span product identifiers aligned with human click preferences.
• It combines supervised fine-tuning, Direct Preference Optimization on click data, and constraint-based beam search to enhance retrieval quality and address noisy, long-tail queries.
• The approach improves interpretability by ensuring output spans match real catalog entries, thereby reducing hallucinations and bolstering reliable retrieval.

Search Preference Optimization (SPO) is a framework for e-commerce retrieval that reformulates product search as autoregressive generation of an item identifier and then aligns that generator with human click preferences through preference optimization. In this formulation, a user query $q\in\mathcal{Q}$ is mapped not directly to a ranked list over a catalog $\mathcal{D}$, but to an identifier $s\in\mathcal{S}$, defined as a sequence of one or more text spans, such that the identifier picks out a target item $d$. The framework combines supervised fine-tuning, Direct Preference Optimization (DPO) on click-derived preference triples, and constraint-based beam search over valid spans, with the stated goals of improving retrieval quality, handling noisy e-commerce titles, strengthening long-tail behavior, and making results interpretable (Li et al., 2024).

1. Conceptual formulation

SPO begins from the observation that classical retrieval maps a user query into a ranking over a catalog, whereas generative retrieval can instead ask a model to generate the identifier of a pertinent document or item. In the e-commerce setting, the objective is to learn parameters $\theta$ of a model

$p_\theta(s \mid q)$

so that, at inference, the top-scoring identifier

$\hat s=\arg\max_s p_\theta(s\mid q)$

corresponds to an item $d$ that maximizes user-preference utility $U(q,d)$. The framework is therefore not merely a generative retriever; it is a generative retriever explicitly aligned with observed human choice behavior through click data (Li et al., 2024).

The paper motivates this design by listing several search-specific difficulties: the complexity of generating detailed item titles from brief queries, the presence of noise in item titles with weak language order, issues with long-tail queries, and the interpretability of results. SPO addresses these difficulties by replacing raw title generation with generation of structured identifiers, then constraining decoding so that generated spans correspond to real catalog content. A plausible implication is that the framework treats retrieval quality, alignment, and auditability as coupled design constraints rather than independent stages.

2. Representation learning and generative retrieval

At the model level, SPO uses an autoregressive Transformer decoder built on BART-large. Each raw product title $T=[i_1,i_2,\ldots,i_n]$ is first segmented into overlapping n-grams, such as brand, color, or model, then sorted, because global word order in e-commerce titles is often arbitrary. The sorted n-grams are grouped into $\mathcal{D}$0 non-overlapping spans,

$\mathcal{D}$1

so that the task shifts from “query2title” to “query2multi-span” (Li et al., 2024).

This change in representation is central to the framework. The paper states that the transformation reduces noise and increases the chance that different products share common spans. Under supervised fine-tuning, if a target identifier $\mathcal{D}$2 has length $\mathcal{D}$3, the training objective is

$\mathcal{D}$4

where $\mathcal{D}$5. Inference uses the same autoregressive probabilities to propose candidate spans.

The empirical role of this task redefinition is explicit in the ablation study. “query2title” and “title2query” both yield Recall@500 $\mathcal{D}$6, whereas “query2multi-span” with spans of length $\mathcal{D}$7 and $\mathcal{D}$8 reaches $\mathcal{D}$9. This suggests that, within the reported system, the multi-span identifier is not a minor engineering choice but a prerequisite for making generative retrieval viable on noisy product titles (Li et al., 2024).

3. Preference optimization from click data

After supervised fine-tuning, SPO aligns the autoregressive model with user preferences using click data. For each query $s\in\mathcal{S}$0, the training pipeline assembles preference triples $s\in\mathcal{S}$1, where $s\in\mathcal{S}$2 is a span from a clicked item and $s\in\mathcal{S}$3 is a span from an exposed but unclicked item or a random negative. The framework then adopts the DPO loss of Rafailov et al., with $s\in\mathcal{S}$4 as the base supervised policy and $s\in\mathcal{S}$5 controlling deviation strength: $s\in\mathcal{S}$6

The stated purpose of this objective is to push up the likelihood of clicked spans and down that of negatives, thereby aligning the model’s sampling distribution with human choice behavior. The paper contrasts this with full RLHF by describing preference alignment via DPO as a lightweight alternative that directly leverages existing click logs without reward modeling. In the reported ablations, removing DPO reduces head-query performance by up to $s\in\mathcal{S}$7, indicating that preference optimization is especially valuable when many candidate items match a query (Li et al., 2024).

The framework therefore couples two distinct learning signals. Supervised fine-tuning teaches the model how to generate identifiers, while DPO teaches it which identifiers are preferred by users. A plausible implication is that SPO treats click data as a ranking supervision source at the generation level rather than as a post-retrieval reranking signal.

4. Constraint-based decoding and interpretability

SPO does not rely on unconstrained autoregressive decoding. To prevent hallucinated spans that do not correspond to any real product, it employs constrained beam search using an FM-index of all valid n-grams. At each decoding step, it maintains a beam of size $s\in\mathcal{S}$8 and scores each partial span $s\in\mathcal{S}$9 by

$d$0

where $d$1 is the normalized frequency of $d$2 in the product catalog, or $d$3 if $d$4 is out-of-vocabulary. Only the top-$d$5 candidates whose next tokens are valid under the FM-index constraints are retained (Li et al., 2024).

The paper attributes two functions to this step. First, it ensures that every generated span actually occurs in at least one item title, which is presented as the basis for full interpretability and the elimination of hallucinations. Second, it improves retrieval quality: disabling FM-index constraints allows invalid spans to pass and drops Recall@500 on head queries by $d$6–$d$7. The practical guidance given in the paper is that beam sizes of $d$8 strike a balance between coverage and precision, and that once spans are generated, simple FM-index lookups retrieve all matching items in milliseconds, after which a downstream reranker can apply business-specific heuristics (Li et al., 2024).

A common misconception is that generative retrieval is inherently opaque. In the SPO formulation, interpretability is explicitly tied to constrained generation over catalog-grounded spans. The generated identifier can therefore be inspected as a sequence of valid n-grams rather than as an unconstrained text string.

5. Experimental results and reported gains

The framework is trained and evaluated on a proprietary e-commerce log of $d$9 billion click events. Retrieval quality is measured by Recall@K with $\theta$0 and $\theta$1 over the full catalog. To analyze long-tail behavior, queries are bucketed into five groups by the number of matching items: $\theta$2, $\theta$3, $\theta$4, $\theta$5, and $\theta$6. Online A/B tests use uplift in user conversion rate (UCVR) (Li et al., 2024).

The main quantitative highlights reported for the most tail case $\theta$7 are summarized below.

System or setting	Metric	Reported value
Dense retrieval baseline (RSR)	Recall@500	$\theta$8
Lexical generative retrieval (SEAL+SFT)	Recall@500	$\theta$9
GenR-PO + SFT (no preference)	Recall@500	$p_\theta(s \mid q)$0
GenR-PO + DPO + cons	Recall@1000	$p_\theta(s \mid q)$1

The same section reports that adding DPO raises tail Recall@500 to $p_\theta(s \mid q)$2 without constraints and then to $p_\theta(s \mid q)$3 with constraints, with further gains on head queries, and that online A/B testing shows a statistically significant $p_\theta(s \mid q)$4 UCVR lift with $p_\theta(s \mid q)$5 (Li et al., 2024).

These numbers should be read together with the ablation findings. Task redefinition to multi-span identifiers accounts for a large change relative to direct title generation, DPO contributes especially on head queries, and constrained search improves both stability and interpretability. Within the paper’s framing, the framework’s effectiveness is therefore distributed across representation design, preference alignment, and constrained decoding rather than concentrated in a single component.

6. Scope, limitations, and acronym ambiguity

The practical implications stated for SPO are specific. Multi-span identifiers are described as dramatically reducing the generative space by focusing on meaningful n-grams, making training more stable and generalizable, particularly on long-tail queries. Preference alignment via DPO is described as a lightweight alternative to full RLHF. Constrained beam search is said to trade a small increase in inference time, due to FM-index lookups, for a large drop in hallucination (Li et al., 2024).

The framework also has an important terminological complication: the acronym “SPO” is overloaded across several research areas. In audio-text alignment, “SPO” refers to “Standardized Preference Optimization,” a CLAPScore-based training method that standardizes listener scores into z-scores in order to learn relative preferences and mitigate individual scoring biases; the reported XACLE system achieved $p_\theta(s \mid q)$6th place with SRCC $p_\theta(s \mid q)$7 (Takano et al., 6 Jan 2026). In LLM alignment, “SPO” refers to “Self-supervised Preference Optimization,” which augments DPO with a self-supervised preference-degree loss and is reported to improve win rates and MT-Bench performance (Li et al., 2024). In database theory, the acronym is associated with semantic optimization of preference queries, where optimization revolves around the winnow operator, integrity constraints, containment of preference relations, and order-axiom satisfaction rather than click-aligned generative retrieval [0510036].

This suggests that “Search Preference Optimization” should be disambiguated by domain. In the e-commerce literature, it denotes a retrieval framework that combines generative identification, DPO-based click alignment, and FM-index-constrained decoding. In other literatures, the same acronym names distinct methods with different objectives, data assumptions, and mathematical structure.