---
title: Event-Based Candidate Expansion
url: https://www.emergentmind.com/topics/event-based-candidate-expansion
type: topic
---

# Event-Based Candidate Expansion

Event-based candidate expansion is an advanced information retrieval paradigm that enhances query expansion (QE) by dynamically leveraging real-world events to improve the relevance and timeliness of retrieval results. In contrast to traditional QE methods, which often rely on static corpus co-occurrence statistics or slowly updated long-term search logs, event-based candidate expansion employs explicit event detection, normalization, and embedding techniques to reflect sudden shifts in user intent—especially for time-sensitive queries about ongoing or recent developments. Prominent instantiations, such as Event-Centric Query Expansion (EQE) and Event-Driven Query Expansion (EDQE), integrate large-scale event collection, joint embedding of events and words, prompt-driven generation, and sophisticated online ranking to address both recall and precision for event-related search scenarios [2305.19019][2012.12065].

## 1. Concepts and Motivation

Event-based candidate expansion resides within the broader field of query expansion but emphasizes events as unitary, context-rich information objects. The technique seeks to identify and select expansion terms or phrases from recent or historic real-world events relevant to a user query, thereby improving both recall (retrieving more relevant results) and precision (excluding off-topic results). The primary motivation stems from limitations of conventional QE techniques—especially their inability to rapidly adapt to novel or rapidly-developing events, leading to the "cold-start" problem in time-sensitive search applications.

Event-centric systems leverage the unique properties of events: temporal specificity, entity-centric context, and their encapsulation of user intent. By using event-driven expansions, these systems can rapidly surface documents relating to emergent global and local news, breaking stories, and other transient phenomena, outperforming log-mined expansions which update too slowly to capture new trends [2305.19019][2012.12065].

## 2. Event Detection and Candidate Generation

The initial step in event-based candidate expansion involves obtaining a pool of well-formed, relevant events. Systems such as EQE perform continuous headline crawling (≈50 million over six months), applying a two-stage filtering process:

- **Coarse Filtering (Heuristic & Trigger Extraction):** Headlines are analyzed for event triggers (verb-based) using toolkits such as LTP. Headlines lacking triggers, or with more than two (indicating multiple or ambiguous events), are discarded to retain atomic event mentions.
- **Fine Filtering (Machine Learning Classifier):** A RoBERTa-based binary event classifier, trained on 200,000 expert-annotated headlines, outputs a probability $p_{\mathrm{event}}$ via sigmoid activation. Headlines are retained if $p_{\mathrm{event}} \geq \tau$, where $\tau$ ensures >95% end-to-end accuracy [2305.19019].

Alternative methods (EDQE) retrieve candidate events for a query term $w$ from curated knowledge bases (Wikipedia/DBpedia). Scoring combines embedding similarity (static or temporal) and literal frequency within event articles, ensuring detected events are topically and temporally aligned with the query [2012.12065].

## 3. Normalization, Embedding, and Reformulation

Event normalization and candidate refinement are crucial for downstream semantic retrieval. EQE applies an encoder-decoder model (BART or mT5), guided by prompt tuning and contrastive learning:

- **Prompt Guidance:** For each headline $H$, a noun keyword $K$ (extracted via KeyBERT) seeds a prompt fed to the decoder. The model is trained to generate concise, factual event phrases suitable for QE, aligned with the key entity.
- **Contrastive Regularization:** To avoid representation collapse, contrastive loss $L_{CL}$ is jointly optimized with the cross-entropy loss $L_{CE}$. For each minibatch, positive pairs are generated via token swaps, and cosine similarity is used within a temperature-scaled InfoNCE loss.

EDQE projects both words and events into a shared vector space using pre-trained Wikipedia2Vec (static, 100d) and yearly-sliced NYT skip-gram models (temporal, 140d). When an event lacks a direct temporal embedding, it is inserted using a trilateration approach, preserving neighborhood distances to anchors shared across embedding spaces [2012.12065].

## 4. Semantic Retrieval and Neural Ranking

Efficient mapping between queries and event candidates is achieved via dual-tower (Siamese) transformer architectures. In EQE:

- **Encoding:** Separate RoBERTa encoders map queries $Q$ and candidate events $E$ to $\mathbb{R}^d$ vectors. All event embeddings are indexed in Faiss ($\approx$4M entries), allowing real-time ($<1$ ms) nearest-neighbor search via cosine similarity.
- **Training:** Initial training uses positive pairs from click logs (filtered with Jaccard/BERTScore) and random negatives (stage 1). Stage 2 introduces hard negatives: candidates close in embedding space but not clicked, defined by similarity thresholds $[\alpha,\beta]$. This refinement sharpens decision boundaries under the contrastive loss.

EDQE selects expansion terms by mixing TF–IDF- and query-similarity-based candidates from event articles, scoring them via a weighted combination of features (see Table 1 below) [2012.12065].

| Feature         | Description                  | Role in Scoring            |
|-----------------|-----------------------------|----------------------------|
| $\mathrm{tfidf}(c,e)$   | TF–IDF for candidate $c$ in $e$     | Signals relevance via frequency        |
| $\cos(c,e)$ or $\cos_t(c,e)$ | Cosine similarity (static/temporal) | Semantic proximity to event           |
| $\cos(e,q)$ or $\cos_t(e,q)$ | Cosine similarity with query        | Links event to query meaning          |
| $\mathrm{TempRel}(c,e)$ | Temporal relationship score           | Ensures temporal alignment            |

## 5. Online Ranking and Integration with Retrieval

Post-retrieval, event-based systems must select the optimal expansion or expansion set to pair with the user query:

- **Feature-Based Ranking:** EQE utilizes a LightGBM Gradient Boosted Decision Tree model, aggregating three classes of features:
    - Query-side: domain class, named entities, segmentation, IDF-weight statistics.
    - Event-side: timestamp (freshness), popularity (cluster size), source authority.
    - Interaction: semantic similarity, BM25(Q,E*), and entity matches.

The final expansion $E^*$ maximizing the score $S(Q, E)$ is chosen according to:
$$
E^* = \arg\max_{E \in \text{candidates}} S(Q,E)
$$
where $S$ is learned from $\approx$50,000 expert-labeled $\langle Q, E\rangle$ pairs.

- **Retrieval Integration:** Expanded queries $Q \cup E^*$ are run in parallel to the original query in the retrieval-ranking pipeline, and results from both are merged.

EDQE combines the event-driven term distribution $P_{ED}$ and the original query model $P_{ML}$ via interpolation:
$$
P(w \mid q) = \alpha\,P_{ED}(w\mid q) + (1-\alpha)\,P_{ML}(w\mid q)
$$
Retrieval employs standard weighting (TF–IDF, BM25) over the expanded language model.

## 6. Empirical Evaluation and Deployment

Event-based candidate expansion has demonstrated substantial improvements over strong baselines in both offline metrics and large-scale online deployments:

- **EQE Results (850K real queries):**
    - Recall@100: 0.41 (baseline) → 0.58 (+41.5%)
    - Recall@150: 0.47 → 0.65 (+38.3%)
    - Recall@200: 0.58 → 0.74 (+27.6%)
- **Online A/B Testing (30 days, QQ Browser Search):**
    - AGSB: +12.5%, CTR: +6.64%, PCTR: +6.23%, UCTR: +5.03%

EDQE reported MAP improvements from 0.26 (BM25 baseline) to 0.31 (temporal event embedding), with all differences statistically significant ($p < .05$) [2305.19019][2012.12065].

- **Scalability and Latency:** EQE’s offline stages (collection, reformulation, indexing) are implemented as Spark/Flink jobs, updating every few minutes. Online, caching ensures per-query QE latency of $\approx$1 ms (cache hit) or $\approx$10 ms (cold start), with near real-time event coverage for >100M daily active users.

## 7. Analysis, Limitations, and Practical Coverage

Ablation studies confirm that event-based expansion gains derive from both the use of events and the incorporation of temporal modeling. The composition parameter $\lambda$ (fraction of TF–IDF vs. query-similar candidates) shows that most useful expansion terms arise directly from event contexts, with robustness added by a minority of direct query-similar terms. Each element of the event-driven scoring functions contributes roughly equally to MAP; the frequency-based event detector is empirically superior to embedding-only scoring for event identification.

EQE covers approximately 50% of all queries in production; the remaining share consists of navigational or knowledge-based intents less amenable to event-driven expansion [2305.19019].

A plausible implication is that event-based candidate expansion is particularly suited for dynamic, temporally localized search tasks, but less effective for static or encyclopedic information needs where no well-formed event context exists. Continued refinement of event detection criteria, embedding alignment, and real-time indexing is likely to further improve both recall and precision for time-sensitive retrieval systems.

---

**References:**
- Event-Centric Query Expansion in Web Search [2305.19019]
- Event-Driven Query Expansion [2012.12065]

Source: https://www.emergentmind.com/topics/event-based-candidate-expansion