Event-Based Candidate Expansion

• Event-Based Candidate Expansion is an advanced search paradigm that leverages explicit event detection to capture dynamic shifts in user intent.
• It employs techniques like heuristic filtering, machine learning classification, and joint embedding to generate contextually rich candidate expansions.
• Empirical evaluations show significant improvements in recall and precision, validating its effectiveness for real-time, time-sensitive information retrieval.

Event-based candidate expansion is an advanced information retrieval paradigm that enhances query expansion (QE) by dynamically leveraging real-world events to improve the relevance and timeliness of retrieval results. In contrast to traditional QE methods, which often rely on static corpus co-occurrence statistics or slowly updated long-term search logs, event-based candidate expansion employs explicit event detection, normalization, and embedding techniques to reflect sudden shifts in user intent—especially for time-sensitive queries about ongoing or recent developments. Prominent instantiations, such as Event-Centric Query Expansion (EQE) and Event-Driven Query Expansion (EDQE), integrate large-scale event collection, joint embedding of events and words, prompt-driven generation, and sophisticated online ranking to address both recall and precision for event-related search scenarios (Zhang et al., 2023, Rosin et al., 2020).

1. Concepts and Motivation

Event-based candidate expansion resides within the broader field of query expansion but emphasizes events as unitary, context-rich information objects. The technique seeks to identify and select expansion terms or phrases from recent or historic real-world events relevant to a user query, thereby improving both recall (retrieving more relevant results) and precision (excluding off-topic results). The primary motivation stems from limitations of conventional QE techniques—especially their inability to rapidly adapt to novel or rapidly-developing events, leading to the "cold-start" problem in time-sensitive search applications.

Event-centric systems leverage the unique properties of events: temporal specificity, entity-centric context, and their encapsulation of user intent. By using event-driven expansions, these systems can rapidly surface documents relating to emergent global and local news, breaking stories, and other transient phenomena, outperforming log-mined expansions which update too slowly to capture new trends (Zhang et al., 2023, Rosin et al., 2020).

2. Event Detection and Candidate Generation

The initial step in event-based candidate expansion involves obtaining a pool of well-formed, relevant events. Systems such as EQE perform continuous headline crawling (≈50 million over six months), applying a two-stage filtering process:

• Coarse Filtering (Heuristic & Trigger Extraction): Headlines are analyzed for event triggers (verb-based) using toolkits such as LTP. Headlines lacking triggers, or with more than two (indicating multiple or ambiguous events), are discarded to retain atomic event mentions.
• Fine Filtering (Machine Learning Classifier): A RoBERTa-based binary event classifier, trained on 200,000 expert-annotated headlines, outputs a probability $p_{\mathrm{event}}$ via sigmoid activation. Headlines are retained if $p_{\mathrm{event}} \geq \tau$, where $\tau$ ensures >95% end-to-end accuracy (Zhang et al., 2023).

Alternative methods (EDQE) retrieve candidate events for a query term $w$ from curated knowledge bases (Wikipedia/DBpedia). Scoring combines embedding similarity (static or temporal) and literal frequency within event articles, ensuring detected events are topically and temporally aligned with the query (Rosin et al., 2020).

3. Normalization, Embedding, and Reformulation

Event normalization and candidate refinement are crucial for downstream semantic retrieval. EQE applies an encoder-decoder model (BART or mT5), guided by prompt tuning and contrastive learning:

• Prompt Guidance: For each headline $H$, a noun keyword $K$ (extracted via KeyBERT) seeds a prompt fed to the decoder. The model is trained to generate concise, factual event phrases suitable for QE, aligned with the key entity.
• Contrastive Regularization: To avoid representation collapse, contrastive loss $L_{CL}$ is jointly optimized with the cross-entropy loss $L_{CE}$. For each minibatch, positive pairs are generated via token swaps, and cosine similarity is used within a temperature-scaled InfoNCE loss.

EDQE projects both words and events into a shared vector space using pre-trained Wikipedia2Vec (static, 100d) and yearly-sliced NYT skip-gram models (temporal, 140d). When an event lacks a direct temporal embedding, it is inserted using a trilateration approach, preserving neighborhood distances to anchors shared across embedding spaces (Rosin et al., 2020).

4. Semantic Retrieval and Neural Ranking

Efficient mapping between queries and event candidates is achieved via dual-tower (Siamese) transformer architectures. In EQE:

• Encoding: Separate RoBERTa encoders map queries $Q$ and candidate events $E$ to $p_{\mathrm{event}} \geq \tau$0 vectors. All event embeddings are indexed in Faiss ($p_{\mathrm{event}} \geq \tau$14M entries), allowing real-time ($p_{\mathrm{event}} \geq \tau$2 ms) nearest-neighbor search via cosine similarity.
• Training: Initial training uses positive pairs from click logs (filtered with Jaccard/BERTScore) and random negatives (stage 1). Stage 2 introduces hard negatives: candidates close in embedding space but not clicked, defined by similarity thresholds $p_{\mathrm{event}} \geq \tau$3. This refinement sharpens decision boundaries under the contrastive loss.

EDQE selects expansion terms by mixing TF–IDF- and query-similarity-based candidates from event articles, scoring them via a weighted combination of features (see Table 1 below) (Rosin et al., 2020).

Feature	Description	Role in Scoring
$p_{\mathrm{event}} \geq \tau$4	TF–IDF for candidate $p_{\mathrm{event}} \geq \tau$5 in $p_{\mathrm{event}} \geq \tau$6	Signals relevance via frequency
$p_{\mathrm{event}} \geq \tau$7 or $p_{\mathrm{event}} \geq \tau$8	Cosine similarity (static/temporal)	Semantic proximity to event
$p_{\mathrm{event}} \geq \tau$9 or $\tau$0	Cosine similarity with query	Links event to query meaning
$\tau$1	Temporal relationship score	Ensures temporal alignment

5. Online Ranking and Integration with Retrieval

Post-retrieval, event-based systems must select the optimal expansion or expansion set to pair with the user query:

• Feature-Based Ranking: EQE utilizes a LightGBM Gradient Boosted Decision Tree model, aggregating three classes of features:
  • Query-side: domain class, named entities, segmentation, IDF-weight statistics.
  • Event-side: timestamp (freshness), popularity (cluster size), source authority.
  • Interaction: semantic similarity, BM25(Q,E*), and entity matches.

The final expansion $\tau$2 maximizing the score $\tau$3 is chosen according to:

$\tau$4

where $\tau$5 is learned from $\tau$650,000 expert-labeled $\tau$7 pairs.

• Retrieval Integration: Expanded queries $\tau$8 are run in parallel to the original query in the retrieval-ranking pipeline, and results from both are merged.

EDQE combines the event-driven term distribution $\tau$9 and the original query model $w$0 via interpolation:

$w$1

Retrieval employs standard weighting (TF–IDF, BM25) over the expanded LLM.

6. Empirical Evaluation and Deployment

Event-based candidate expansion has demonstrated substantial improvements over strong baselines in both offline metrics and large-scale online deployments:

• EQE Results (850K real queries):
  • Recall@100: 0.41 (baseline) → 0.58 (+41.5%)
  • Recall@150: 0.47 → 0.65 (+38.3%)
  • Recall@200: 0.58 → 0.74 (+27.6%)
• Online A/B Testing (30 days, QQ Browser Search):
  • AGSB: +12.5%, CTR: +6.64%, PCTR: +6.23%, UCTR: +5.03%

EDQE reported MAP improvements from 0.26 (BM25 baseline) to 0.31 (temporal event embedding), with all differences statistically significant ($w$2) (Zhang et al., 2023, Rosin et al., 2020).

• Scalability and Latency: EQE’s offline stages (collection, reformulation, indexing) are implemented as Spark/Flink jobs, updating every few minutes. Online, caching ensures per-query QE latency of $w$31 ms (cache hit) or $w$410 ms (cold start), with near real-time event coverage for >100M daily active users.

7. Analysis, Limitations, and Practical Coverage

Ablation studies confirm that event-based expansion gains derive from both the use of events and the incorporation of temporal modeling. The composition parameter $w$5 (fraction of TF–IDF vs. query-similar candidates) shows that most useful expansion terms arise directly from event contexts, with robustness added by a minority of direct query-similar terms. Each element of the event-driven scoring functions contributes roughly equally to MAP; the frequency-based event detector is empirically superior to embedding-only scoring for event identification.

EQE covers approximately 50% of all queries in production; the remaining share consists of navigational or knowledge-based intents less amenable to event-driven expansion (Zhang et al., 2023).

A plausible implication is that event-based candidate expansion is particularly suited for dynamic, temporally localized search tasks, but less effective for static or encyclopedic information needs where no well-formed event context exists. Continued refinement of event detection criteria, embedding alignment, and real-time indexing is likely to further improve both recall and precision for time-sensitive retrieval systems.

---

References:

• Event-Centric Query Expansion in Web Search (Zhang et al., 2023)
• Event-Driven Query Expansion (Rosin et al., 2020)