Intent-Aware Query Expansion

• Intent-aware query expansion is a method that reformulates user queries by incorporating terms aligned with inferred search intent rather than relying solely on lexical similarity.
• It leverages diverse methodologies such as behavioral session mining, intent-conditioned neural generation, grammatical role analysis, and taxonomy-driven approaches to control query drift.
• Empirical evaluations show that these techniques significantly enhance retrieval performance, improving metrics like click-through and conversion rates while balancing intent preservation and diversification.

Searching arXiv for recent and foundational papers on intent-aware query expansion and closely related query reformulation methods. First, I’ll look up the specific 2025 papers and a few broader query expansion references. Intent-aware query expansion is the formulation of alternative queries, rewrites, or added terms that are selected not merely for lexical relatedness but for consistency with an inferred underlying search goal. Across the literature, the central problem is the “intention gap”: users often express needs with short, ambiguous, or lexically sparse queries whose surface form does not adequately communicate the intended target, whether in general information retrieval, e-commerce search, or intent classification (Selvaretnam et al., 2020). Recent work has operationalized intent-aware expansion through behavioral sequence mining, intent-conditioned neural generation, linguistic role analysis, taxonomy-driven semantic resources, knowledge-base structures, and LLM generation. In these formulations, expansion is not treated as unrestricted semantic broadening; it is constrained by session dynamics, latent intent categories, grammatical roles, semantic relations, or attribute-aligned matching so that enrichment improves retrieval relevance, reformulation fidelity, or downstream engagement rather than inducing query drift (Yetukuri et al., 25 Jul 2025).

1. Conceptual foundations and scope

Intent-aware query expansion differs from generic query expansion in the criterion by which candidate additions are judged. In generic expansion, terms may be selected because they are distributionally similar, taxonomically related, or statistically associated with the query. In intent-aware expansion, those same signals are subordinated to an explicit or implicit model of user purpose. The 2020 survey on natural language technology and query expansion frames this as enrichment of the “significant constituents that characterize the query intent” through meaningful terms, phrases, or latent representations, with particular attention to ambiguity, synonymy, polysemy, morphology, syntax, semantics, and discourse cues (Selvaretnam et al., 2020). This definition places intent not only in topic labels but also in linguistic structure, query type, and task orientation.

In e-commerce search, intent-aware expansion is grounded in buyer behavior and downstream actions. “AI Guided Accelerator For Search Experience” defines it as producing alternative queries that preserve the latent shopping intent observed in the user’s session trajectory while introducing controlled lexical and semantic diversity (Yetukuri et al., 25 Jul 2025). “Intent-Aware Neural Query Reformulation for Behavior-Aligned Product Search” similarly treats reformulation as alignment with latent buyer intent inferred from explicit interactions and implicit behavioral cues rather than surface lexical overlap (Yetukuri et al., 29 Jul 2025). These formulations emphasize that exploratory behavior, refinement, and purchase-oriented convergence are structurally different phases of the same search task.

The topic also includes systems that map queries to explicit intent classes. The taxonomy-based framework in “Evaluation of semantic relations impact in query expansion-based retrieval systems” defines intent-aware query expansion as reformulating a query into the label of a pre-existing taxonomy by augmenting the query representation with semantically related terms derived from taxonomy labels (Massai, 2022). A different but related formulation appears in “ConQX,” where expansions are conditioned to support intent detection for short spoken queries by generating coherent, intent-revealing paraphrastic continuations rather than isolated terms (Yilmaz et al., 2021).

A useful organizing distinction is between intent preservation and intent diversification. Intent preservation requires that candidate expansions remain within the same task-consistent semantic boundary; intent diversification permits multiple plausible subintents or realizations to be represented, provided they remain grounded in the underlying information need. This distinction is explicit in contextual clue sampling, where multiple generated contexts are filtered and fused to balance diversity and relevance (Liu et al., 2022), and in e-commerce session modeling, where transitional queries capture exploration without abandoning the eventual conversion trajectory (Yetukuri et al., 25 Jul 2025).

2. Representations of intent

A major axis of the literature concerns how intent is represented. One family of methods represents intent as a latent variable over a query sequence or behavioral session. In the eBay accelerator framework, a user session is modeled as a sequence of queries and events,

$Q = [q_1, q_2, \dots, q_T],$

with associated conversion markers such as buy, bid, offer, watch, ask, and cart-click (“bbowac”). Each query $q_t$ is associated conceptually with a latent intent $z_t$, and the framework motivates a Markovian transition structure across intents,

$T_{ij} = p(z_t = j \mid z_{t-1} = i),$

although the paper does not estimate $T_{ij}$ explicitly (Yetukuri et al., 25 Jul 2025). Intent is instead operationalized through embedding-based similarity between adjacent queries.

A second family represents intent as a discrete conditioning variable. The 2025 neural reformulation system operationalizes intent into three canonical buckets—Same Intent, Similar Intent, and Inspired Intent—and at a broader level references transactional, navigational, and informational distinctions (Yetukuri et al., 29 Jul 2025). It trains an intent-conditioned sequence-to-sequence model,

$P_\theta(r \mid q, i),$

where $q$ is the source query, $i$ is the intent tag, and $r$ is the rewrite. This formulation treats intent as a controllable variable appended to the input, allowing one model to express multiple reformulation regimes.

A third family represents intent in linguistic structure. The linguistic frameworks of 2020 define roles such as Concepts-of-Interest (CoI), Descriptive Concepts (Dc or DC), Relational Concepts (Rc or RC), and Structural Concepts (Sc or SC) to identify which query constituents encode the search goal and which merely modify or connect them (Selvaretnam et al., 2020, Selvaretnam et al., 2020). Here intent is not a label attached after parsing; it is extracted from the grammatical organization of the query itself. Expansion is then restricted to content-bearing roles and weighted by their functional importance.

A fourth representation is graph- or knowledge-based. Structural Query Expansion uses Wikipedia entities and categories as an intent proxy, assuming that articles structurally close to the entities linked from the query reflect the user’s target topic (Guisado-Gámez et al., 2016). Knowledge-based microblog expansion uses Freebase aliases, names, notable types, and descriptions to create an enriched query LLM that better matches entity-centric social media usage (Qiang et al., 2015).

A fifth representation is multi-intent and attribute-aligned. The multi-intent attribute-aware matching model represents both query-side and item-side inputs as mixtures over $K$ intent components with intent distributions aligned by a Kullback-Leibler divergence objective (Li et al., 2024). This suggests a view of expansion in which terms are not globally relevant to a query, but relevant to one of several aligned query-item intent components.

These representations are compatible rather than mutually exclusive. A plausible implication is that modern systems can combine session-derived latent intent, explicit intent tags, grammatical role structure, and attribute-conditioned multi-intent mixtures to obtain both better constraints and better diversity.

3. Core methodological families

The methodological landscape of intent-aware query expansion can be organized into several recurring designs.

3.1 Behavioral and session-based mining

Behavioral mining is central in e-commerce. The eBay accelerator framework reconstructs structured query trajectories from logs via sessionization, splitting at bbowac events, normalization, de-duplication, and an Intent Filter that walks backward from a converting query and retains predecessors only when semantic similarity exceeds a threshold $q_t$0 (Yetukuri et al., 25 Jul 2025). The resulting chain is segmented into source, transitional, and converging queries. A representative example removes “macbook” from a sequence that converges on “iphone 12 128gb” because the similarity to the subsequent query falls below $q_t$1.

The complementary 2025 neural reformulation paper mines three classes of source-target pairs: in-session $q_t$2-hop buyer reformulations, cross-session co-engaged queries, and cross-session 1-hop co-clicked queries (Yetukuri et al., 29 Jul 2025). These signals capture corrections, intent-preserving refinements, and more exploratory behaviorally linked rewrites. Post-filtering uses taxonomy categories, Named Entity Recognition, categorical alignment, buffered recall similarity, and query length compatibility.

3.2 Intent-conditioned neural generation

Neural reformulation systems use sequence models to generate rewrites directly. The 2025 product-search reformulation system trains BiLSTM and lightweight Transformer models on unified mined datasets with intent tags appended to each instance (Yetukuri et al., 29 Jul 2025). The learning objective is maximum likelihood over intent-conditioned rewrites,

$q_t$3

This yields rewrites whose type distribution can be compared with real rewrite behavior through metrics such as Rewrite Type Agreement Score.

LLMs are used differently in the eBay accelerator. There, Solar-10B-Instruct is instruction fine-tuned with few-shot examples built from mined user journeys, and the model is asked to generate alternate converging queries aligned with a transitional query’s intent while excluding already mined converging queries (Yetukuri et al., 25 Jul 2025). The role of the LLM is therefore not unconditional rewriting but controlled expansion beyond what collaborative filtering alone can provide.

ConQX shows a parallel design for intent detection rather than retrieval: GPT-2 is conditioned with mined prompts and optional examples to generate short natural-language expansions that clarify ambiguous spoken queries, and those expansions are appended before fine-tuning BERT or RoBERTa classifiers (Yilmaz et al., 2021). The conditional generation model is written as

$q_t$4

where $q_t$5 includes the prompt context.

3.3 Linguistic and ontology-driven expansion

Linguistically driven frameworks emphasize grammatical analysis before expansion. The coupled intrinsic/extrinsic framework parses the query, detects non-compositional phrases, assigns concept roles from typed dependencies, performs WordNet-based sense disambiguation using Adapted Lesk, and weights original and expansion concepts through a global role-weight vector $q_t$6 learned by a genetic algorithm (Selvaretnam et al., 2020). Expansion concepts are scored by average relatedness to base terms:

$q_t$7

The LSQE framework instead mines expansion candidates from Google web n-grams using dependency-derived base pairs that include CoI or Dc terms, then assigns role-based weights and integrates them into Indri queries via #weight (Selvaretnam et al., 2020).

3.4 Taxonomy, semantic-relation, and knowledge-base methods

The taxonomy-based intent classifier expands category gazetteers with semantic relations including synonymy, hypernymy, hyponymy, meronymy, holonymy, adjectival and nominal collocations, triggers, and frequent predecessors or followers, then predicts the best label by matching expanded query terms to gazetteers (Massai, 2022). Structural and semantic relation choice is treated as an ablation variable, revealing which relations improve recall or harm precision.

Knowledge-base methods use structured resources rather than behavioral logs. Structural Query Expansion derives expansion features solely from motifs in the Wikipedia article-category graph, especially triangular and square motifs with reciprocal article links and category constraints (Guisado-Gámez et al., 2016). Knowledge-based microblog search uses Freebase concepts and temporal priors to construct a knowledge-aware query model:

$q_t$8

Candidate knowledge terms are selected by association scoring over pseudo-relevant tweets, optionally with an exponential recency prior (Qiang et al., 2015).

3.5 Multi-intent matching and intent-aware retrieval fusion

Some methods treat query expansion as retrieval-time evidence fusion rather than direct rewriting. Contextual clue sampling generates multiple answer-bearing contexts with BART-large, clusters them with a lexical similarity threshold of $q_t$9, keeps the highest-probability representative per cluster, and fuses retrieval scores using generation likelihoods:

$z_t$0

This balances diversity and relevance and can be read as a form of intent-aware expansion through multiple contextual realizations (Liu et al., 2022).

The multi-intent attribute-aware matching model goes further by aligning query-side and item-side intent distributions via symmetric KL divergence,

$z_t$1

then using the aligned components for matching (Li et al., 2024). This suggests expansion candidates can be scored by how well they activate aligned query-item intents.

4. Intent preservation, diversification, and drift control

A central technical problem in intent-aware query expansion is avoiding drift while still generating enough lexical or semantic breadth to improve retrieval. The literature repeatedly converges on controlled diversity rather than maximal expansion.

In session-based e-commerce, drift is controlled by the Intent Filter threshold $z_t$2, embedding-based similarity derived from retrieved item sets, and backward traversal from a converting query (Yetukuri et al., 25 Jul 2025). The practical effect is that exploratory mid-journey queries are retained only if they remain within an intent-consistent chain. The paper reports that intent-filtered suggestions without LLM augmentation underperform because they are too narrow, which indicates that drift control alone is insufficient if lexical diversity is overly constrained (Yetukuri et al., 25 Jul 2025).

In the unified neural reformulation framework, drift is limited by categorical compatibility, buffered recall similarity, length compatibility, and downstream post-processing. Intent buckets also act as structural constraints: Same Intent preserves goal, Similar Intent changes specificity, and Inspired Intent permits broader exploratory reformulation under behavioral supervision (Yetukuri et al., 29 Jul 2025). This taxonomy formalizes that not all non-identical rewrites constitute drift.

In contextual clue sampling, the drift problem arises from hallucinated or irrelevant language-model generations. The method addresses it with cluster-and-filter selection: candidates are grouped using fuzzy string matching, then only the highest-probability candidate per cluster is retained (Liu et al., 2022). The result is a filtered set averaging 24 contexts per query for Natural Questions and 33 for TriviaQA, with 70% fewer retrieval calls than the unfiltered set and improved Top-5/20/100 retrieval accuracy (Liu et al., 2022).

Linguistic and ontology-driven systems control drift by restricting which tokens may seed expansion and which semantic relations are permitted. The 2020 role-based framework expands only CoI and DC terms and excludes RC and SC terms from semantic expansion, while WordNet relation pools are ranked by sense-aware relatedness (Selvaretnam et al., 2020). The semantic-relation impact study shows why such selectivity matters: synonymy improves recall, F-measure, and accuracy with minimal precision harm, whereas antonymy, part-whole relations, and some broad collocational relations can inject substantial noise (Massai, 2022).

The tension between preservation and diversification can be summarized as follows. Preservation mechanisms include typed dependencies, sense disambiguation, category compatibility, similarity thresholds, attribute alignment, and exclusion of repeated or previously mined rewrites. Diversification mechanisms include multi-intent buckets, multiple sampled contexts, LLM alternates, cross-session co-click signals, and square structural motifs. The strongest systems do not choose one over the other; they use preservation to define the admissible region and diversification to explore within it.

5. Evaluation paradigms and empirical findings

Evaluation varies considerably by application domain, but the core empirical pattern is consistent: intent-aware expansion outperforms purely lexical or weakly constrained alternatives when the intent model is sufficiently informative.

In the eBay accelerator framework, online proxies are click-through rate and conversion rate relative to the production Related Searches baseline (Yetukuri et al., 25 Jul 2025). Intent-filtered-only suggestions yield CTR $z_t$3 and conversions $z_t$4, while the LLM Alternator augmented with generated alternates yields CTR $z_t$5 and conversions $z_t$6 relative to production Related Searches (Yetukuri et al., 25 Jul 2025). These results imply that session-derived transitional modeling is useful but incomplete unless paired with controlled generative diversity.

The unified neural reformulation system evaluates offline rewrite fidelity and type alignment rather than online engagement (Yetukuri et al., 29 Jul 2025). Its lightweight Transformer trained on the unified dataset achieves the highest Rewrite Type Agreement Score among single-output models at $z_t$7, while the top-5 generation variant reaches coverage $z_t$8, recall $z_t$9, BLEU $T_{ij} = p(z_t = j \mid z_{t-1} = i),$0, and rewrite-type frequency–weighted recall $T_{ij} = p(z_t = j \mid z_{t-1} = i),$1 (Yetukuri et al., 29 Jul 2025). These numbers indicate that intent-conditioned training on richer behavioral mining sources better matches real-world rewrite distributions than in-session-only or heuristic baselines.

Contextual clue sampling evaluates retrieval rather than rewriting. On Natural Questions, “Ours-multi” reaches Top-100 accuracy $T_{ij} = p(z_t = j \mid z_{t-1} = i),$2 versus DPR at $T_{ij} = p(z_t = j \mid z_{t-1} = i),$3, with a sparse index of $T_{ij} = p(z_t = j \mid z_{t-1} = i),$4 GB versus DPR’s $T_{ij} = p(z_t = j \mid z_{t-1} = i),$5 GB; on TriviaQA, “Ours-multi” reaches Top-100 accuracy $T_{ij} = p(z_t = j \mid z_{t-1} = i),$6 versus DPR at $T_{ij} = p(z_t = j \mid z_{t-1} = i),$7 (Liu et al., 2022). The multi-target fusion plus DPR hybrid is stronger still. Although this paper is not framed as user-intent modeling in the same sense as e-commerce work, it shows that multiple generated contexts, filtered and likelihood-weighted, can act as an intent-aware expansion mechanism that improves high-recall retrieval.

The linguistic role-based frameworks report retrieval gains primarily in MAP. The coupled intrinsic/extrinsic WordNet framework reports statistically significant improvements over unigram LM, relevance-model feedback, and dependence-based QE, especially with synonyms and hypernyms (Selvaretnam et al., 2020). LSQE similarly improves MAP over LM, RM, and a sequential-dependence baseline across multiple TREC collections; for example, on AP88–90 it rises from $T_{ij} = p(z_t = j \mid z_{t-1} = i),$8 for LM to $T_{ij} = p(z_t = j \mid z_{t-1} = i),$9 for LSQE (Selvaretnam et al., 2020). These findings support the claim that syntactic role assignment and dependency-based base-pair formation add value beyond frequency-based expansion.

The semantic-relation impact study evaluates query-to-taxonomy intent reformulation with precision, recall, F-measure, and accuracy (Massai, 2022). Its baseline has Precision $T_{ij}$0, Recall $T_{ij}$1, F-measure $T_{ij}$2, and Accuracy $T_{ij}$3. SYN improves recall by $T_{ij}$4 and F-measure by $T_{ij}$5 with minimal precision change, whereas TRG gives the largest recall gain at $T_{ij}$6 but harms precision (Massai, 2022). The best balanced multi-relation combinations include SYN and TRG, often with PAR or SPC.

Structural Query Expansion reports more than 150% improvement over non-expanded queries and identifies expansion features in less than 0.2 seconds in the worst case scenario (Guisado-Gámez et al., 2016). Knowledge-based microblog expansion shows significant superiority over baseline methods on TREC Twitter corpora; on TREC’13, QEFB+SMM reaches MAP $T_{ij}$7 and P@30 $T_{ij}$8, outperforming SimpleKL at $T_{ij}$9 and $P_\theta(r \mid q, i),$0 respectively (Qiang et al., 2015).

For intent detection, ConQX improves weighted F1 across Banking, CLINC, and SNIPS. On SNIPS with BERT, performance rises from $P_\theta(r \mid q, i),$1 with no expansion to $P_\theta(r \mid q, i),$2 with one-shot ConQX; on Banking with RoBERTa, it rises from $P_\theta(r \mid q, i),$3 to $P_\theta(r \mid q, i),$4 with zero- or one-shot ConQX (Yilmaz et al., 2021). These are modest but consistent gains over strong baselines, suggesting that natural-language expansions can clarify intent even when the downstream task is classification rather than retrieval.

6. Limitations, misconceptions, and future directions

A common misconception is that intent-aware query expansion simply means adding semantically similar terms. The evidence does not support that simplification. Several papers show that unconstrained semantic similarity, heuristic token dropping, or naive context generation either underperform or introduce harmful noise (Yetukuri et al., 29 Jul 2025, Liu et al., 2022). Intent awareness requires a mechanism that distinguishes semantically related but off-goal additions from useful refinements.

Another misconception is that more diversity is always better. The eBay accelerator explicitly observes that transitional chains alone are too narrow, but it also notes that LLM alternates can over-concentrate within minor attribute variations, reducing carousel attractiveness (Yetukuri et al., 25 Jul 2025). Contextual clue sampling shows a similar pattern: generating 100 candidates increases answer coverage, but filtering is needed to avoid redundancy and hallucination (Liu et al., 2022). Diversity is beneficial only when paired with consolidation, clustering, ranking, or intent-aligned weighting.

Several limitations recur across the literature. Sparse or long-tail intents remain difficult because behavioral co-engagement, bbowac signals, or item interaction data may be insufficient (Yetukuri et al., 25 Jul 2025, Yetukuri et al., 29 Jul 2025). Ambiguous or multi-intent queries can defeat single-intent classifiers or overly rigid constraints; both the 2025 product-search paper and the multi-intent attribute-aware model suggest richer context and explicit multi-intent modeling as remedies (Yetukuri et al., 29 Jul 2025, Li et al., 2024). Ontology-driven methods are constrained by resource coverage, sense disambiguation errors, and the risk of overgeneralization or lateral drift, especially with hyponyms and coordinate terms (Selvaretnam et al., 2020). Knowledge-base methods are vulnerable to entity-linking errors and stale or incomplete resources (Guisado-Gámez et al., 2016, Qiang et al., 2015).

The most plausible future direction is hybridization. The 2025 eBay work explicitly proposes hybrid approaches combining behavioral mining with generative models for richer personalization and more robust intent inference (Yetukuri et al., 25 Jul 2025). The neural reformulation paper points toward Retrieval-Augmented Generation, multimodal signals, and session-aware ranking integration (Yetukuri et al., 29 Jul 2025). The multi-intent attribute-aware framework suggests a route toward expansion conditioned on aligned query-item latent mixtures rather than only query-side evidence (Li et al., 2024). The broader survey literature likewise identifies session-aware modeling, personalization, diversification over subintents, and multi-source evidence integration as central open challenges (Selvaretnam et al., 2020).

Intent-aware query expansion has therefore evolved from a narrow retrieval heuristic into a family of methods for modeling and operationalizing search goals. Whether implemented through behavioral trajectories, intent tags, grammatical roles, taxonomy labels, knowledge-graph motifs, generated contexts, or aligned multi-intent components, its defining property is the same: the expansion is judged by how well it preserves or productively elaborates the user’s underlying objective, not merely by whether it resembles the original query.