Dynamic Evidence Adjudication

• DEA is a dynamic retrieval engine that organizes edits into semantic clusters to support scalable and accurate multi-hop reasoning tasks.
• The methodology employs a two-stage process: first filtering candidate clusters using cosine similarity, then scoring edits through a combination of literal and inferential evidence.
• Empirical results demonstrate that DEA reduces search complexity by 86.7% while significantly improving retrieval accuracy and reasoning fidelity.

Dynamic Evidence Adjudication (DEA) is a retrieval and selection engine integral to the ALEX knowledge editing framework, designed for efficient and reliable reasoning over hierarchically clustered edit memories in LLM systems. DEA addresses the challenge of scalable retrieval and accurate evidence adjudication in editing tasks, especially in contexts that require multi-hop reasoning. By integrating statistical filtering and semantically motivated evidence scoring, DEA enables substantial reductions in search complexity while preserving or improving answer accuracy and reasoning fidelity (Wang et al., 18 Nov 2025).

1. Two-Stage Retrieval Architecture

DEA operates as a two-stage retrieval process over a hierarchically organized edit memory. The memory, containing $N$ edits grouped into $K$ semantic clusters, supports efficient filtering and evidence evaluation. Upon receiving a query $q$, DEA first performs a coarse-grained filter (Stage I) to identify the most promising clusters. In Stage II, it conducts fine-grained scoring among candidate edits within the filtered clusters, leveraging both literal and inferential evidence. This stratified approach ensures high recall by capturing semantically related information while substantially reducing the number of retrieval computations.

2. Semantic Clustering and Evidence Signals

ALEX organizes edits into semantic clusters using the SMP engine, with each cluster $c$ characterized by a centroid $\mu_c$. In Stage I, DEA computes the cosine similarity between the embedded query representation $\phi(q)$ and each cluster centroid. Scores are standardized via z-score normalization, and only clusters whose z-scores exceed a threshold $\zeta$ (default $1.0$) are advanced to Stage II, subject to a cap ($M=3$) on the number of clusters.

Within each retained cluster, DEA evaluates each edit $e_j$ by combining two signals:

• Literal evidence: $\cos(\phi(q), \phi(e_j))$
• Inferential evidence: $$\max_{h\in \mathcal{H}(e_j)} \cos(\phi(q), \phi(h))$$, where $\mathcal{H}(e_j)$ are $N_h=3$ pseudo-questions generated for each edit by the Inferential Query Synthesis (IQS) module.

Edit selection is based on the maximization of a weighted sum of these signals, with $\alpha=\beta=0.5$ by default.

3. Algorithmic Formulation

The DEA process is formalized as follows:

• Stage I (Cluster Filtering):

$$s_i = \cos(\phi(q),\mu_i),\quad z_i = \frac{s_i-\bar{s}}{\sigma_s}$$

Clusters are selected into $\mathcal{C}$ if $z_i\geq \zeta$, with $|\mathcal{C}|\leq M$.

• Stage II (Evidence Adjudication):

$$\Psi(e_j) = \alpha\,\cos(\phi(q),\phi(e_j)) + \beta\,\max_{h\in \mathcal{H}(e_j)} \cos(\phi(q),\phi(h))$$

The final returned edit is $\displaystyle e^* = \arg\max_{e_j} \Psi(e_j)$.

A high-level pseudocode sketch is as follows:

phi_q = Embed(q) for i in range(K): s_i = cosine(phi_q, mu[i]) mean_s = mean(all s_i) std_s = std(all s_i) z_i = (s_i - mean_s) / std_s candidateClusters = top-M indices where z_i >= zeta bestScore, e_star = -inf, None for c in candidateClusters: for e_j in cluster c: litEvidence = cosine(phi_q, phi(e_j)) infEvidence = max([cosine(phi_q, phi(h)) for h in H(e_j)]) score = alpha * litEvidence + beta * infEvidence if score > bestScore: bestScore, e_star = score, e_j return e_star

4. Complexity and Efficiency Analysis

DEA's design yields significant efficiency gains relative to flat search:

• Stage I: $O(K)$ cosine computations (each query–centroid pairing).
• Stage II: $O(M \cdot C) \approx O(N/K)$, where $C$ is the average cluster size.
• Total: $O(K + N/C)$.

Empirical analysis on MQuAKE-CF-3K-v2 with $K=12$ demonstrates a reduction in average edits examined from approximately $2764$ to $368$ (86.7% reduction) (Wang et al., 18 Nov 2025). This complexity is in contrast to the canonical $O(N)$ in memory-based retrievers.

5. Empirical Effects and Ablation Results

Ablation studies on the MQuAKE benchmarks isolate the contribution of DEA (Table 1 reproduced below). DEA alone improves MultiHop-ACC (MA) and HopWise-ACC (HA) on all tested datasets compared to the baseline, even in the absence of the IQS module.

IQS	DEA	M-CF-3K-v2 MA	M-CF-3K-v2 HA	M-T MA	M-T HA	M-Hard MA	M-Hard HA
×	×	36.87	30.94	70.53	59.79	62.90	57.24
√	×	41.75	35.15	75.92	64.04	74.84	69.77
×	√	48.17	42.68	82.07	71.74	67.55	62.17
√	√	53.50	47.43	87.33	76.49	79.20	74.35

MA: MultiHop-ACC; HA: HopWise-ACC.

The inclusion of DEA yields substantial gains in both retrieval accuracy and search-space efficiency.

6. Implementation Considerations and Hyperparameters

DEA's operation is governed by the following hyperparameters and architectural choices:

Component	Hyperparameter & Value
z-score threshold	$\zeta=1.0$
Maximum clusters per query ($M$)	3
Adjudication weights ($\alpha, \beta$)	0.5 each
Embedding model	MPNet (Sentence-Transformers)
$\mathrm{Agg}$ operator	max-pooling over $\mathcal{H}(e)$
Number of hypothetical questions ($N_h$)	3 (from IQS)

Embedding vectors for edits and pseudo-questions are cached, and at inference only a single forward pass for $\phi(q)$ is required. The cosine similarity metric underpins all evidence signals. The fixed values of $\alpha$ and $\beta$ reflect equal weighting of literal and inferential evidence.

7. Context and Significance in Knowledge Editing

DEA exemplifies a hybrid strategy that combines statistical filtering with semantically rich adjudication for scalable and accurate knowledge editing in LLM settings. Its integration within the ALEX framework enables accurate multi-hop reasoning and reliable retrieval in dynamic memory contexts, meeting emerging requirements for knowledge update, edit localization, and efficient fact retrieval. Experimental results confirm DEA’s critical role in improving both the efficiency and accuracy of multi-step reasoning workflows (Wang et al., 18 Nov 2025). A plausible implication is that similar dual-stage adjudication architectures may provide benefits in other retrieval-intensive domains, particularly where semantic drift and edit history must be resolved at scale.