KDR-Agent: Multi-Agent NER Framework

• KDR-Agent is a multi-agent framework for low-resource NER that decomposes inference into knowledge in-context construction and reflective correction stages.
• It coordinates a central planner with specialized agents for factual retrieval and natural-language disambiguation to resolve entity ambiguity and supplement LLM domain knowledge.
• Empirical evaluations across multiple domains show consistent 4–8 F1 point gains, underscoring the framework’s modular design and practical impact.

Searching arXiv for the specified paper and closely related NER/ICL context. KDR-Agent is a multi-agent framework for multi-domain low-resource named entity recognition (NER) via in-context learning (ICL) with LLMs. It integrates Knowledge retrieval, Disambiguation, and Reflective analysis under a central planner, with the stated goal of addressing three key challenges in low-resource ICL-based NER: reliance on large annotated support sets, insufficient domain knowledge of the LLM, and entity ambiguity (Mu et al., 24 Nov 2025). The framework operates in two stages—Knowledge In-context Construction and Reflection & Correction—and combines natural-language type definitions, a static set of entity-level contrastive demonstrations, Wikipedia-derived factual knowledge, contextual disambiguation, and structured self-assessment to improve NER across multiple domains (Mu et al., 24 Nov 2025).

1. Problem Setting and Design Objective

KDR-Agent is designed for NER in low-resource, multi-domain settings using ICL with LLMs (Mu et al., 24 Nov 2025). The paper identifies three explicit issues motivating the framework: reliance on large annotated support sets (Issue 1), insufficient domain knowledge of the LLM (Issue 2), and entity ambiguity (Issue 3) (Mu et al., 24 Nov 2025). In this formulation, the framework is intended not merely to predict entity spans and labels from a sentence, but to augment the inference process with external factual evidence and post hoc diagnostic correction.

The central methodological premise is that low-resource NER can benefit from decomposing inference into specialized subtasks handled by distinct agents. This suggests a modular alternative to single-pass prompting: instead of relying exclusively on retrieved labeled exemplars or the latent knowledge of the backbone LLM, KDR-Agent injects external knowledge, explicit disambiguation rationale, and reflective error analysis into the prompt construction and revision process (Mu et al., 24 Nov 2025).

2. Multi-Agent Architecture and Coordination

KDR-Agent divides inference into two stages: A. Knowledge In-context Construction and B. Reflection & Correction (Mu et al., 24 Nov 2025). The architecture comprises four coordinated roles: a Central LLM Planner, a Knowledge Retrieval Agent, a Disambiguation Agent, and a Reflective Analysis Agent (Mu et al., 24 Nov 2025).

The Central LLM Planner scans the input sentence $x$ to identify both concepts requiring external facts and ambiguous mentions. It emits a set of Wikipedia search queries,

$Q = \{\,q_i = \mathrm{GenerateQuery}(m_i, x)\,\},$

and a prompt segment $\mathcal{P}_{\mathrm{ambig}}$ listing mentions to disambiguate (Mu et al., 24 Nov 2025). The Knowledge Retrieval Agent issues Wikipedia lookups for each query,

$k_i \;=\;\mathrm{RetrieveFromWiki}(q_i),$

and returns lead-paragraph snippets $\{k_i\}$ formatted as $\mathcal{P}_{\mathrm{know}}$ (Mu et al., 24 Nov 2025). The Disambiguation Agent consumes the input $x$ together with $\mathcal{P}_{\mathrm{ambig}}$ and produces natural-language reasoning statements,

$$\mathcal{P}_{\mathrm{disamb}} = \mathrm{Disambiguate}(x,\;\mathcal{P}_{\mathrm{ambig}}),$$

which are intended to guide the prediction toward the correct sense or entity type (Mu et al., 24 Nov 2025).

After the first-pass prediction $\hat y^{(0)}$, the Reflective Analysis Agent performs structured error analysis under a reflection guideline $Q = \{\,q_i = \mathrm{GenerateQuery}(m_i, x)\,\},$0,

$Q = \{\,q_i = \mathrm{GenerateQuery}(m_i, x)\,\},$1

and generates a diagnostic report used in a second correction pass (Mu et al., 24 Nov 2025). The paper specifies the coordination flow as follows:

1. Central planner generates $Q = \{\,q_i = \mathrm{GenerateQuery}(m_i, x)\,\},$2
2. Retrieval agent builds $Q = \{\,q_i = \mathrm{GenerateQuery}(m_i, x)\,\},$3
3. Disambiguation agent builds $Q = \{\,q_i = \mathrm{GenerateQuery}(m_i, x)\,\},$4
4. Prompt assembly in Stage 1 yields initial inference $Q = \{\,q_i = \mathrm{GenerateQuery}(m_i, x)\,\},$5
5. Reflective agent produces $Q = \{\,q_i = \mathrm{GenerateQuery}(m_i, x)\,\},$6
6. Final correction in Stage 2 yields $Q = \{\,q_i = \mathrm{GenerateQuery}(m_i, x)\,\},$7 (Mu et al., 24 Nov 2025)

A plausible implication is that the framework separates epistemic augmentation from corrective reasoning: factual completion and ambiguity resolution occur before initial decoding, whereas structured error repair is deferred to a subsequent pass.

3. Knowledge Retrieval and Disambiguation

The knowledge retrieval component is explicitly based on Wikipedia. For each flagged concept $Q = \{\,q_i = \mathrm{GenerateQuery}(m_i, x)\,\},$8, the planner formulates

$Q = \{\,q_i = \mathrm{GenerateQuery}(m_i, x)\,\},$9

and the retrieval call returns

$\mathcal{P}_{\mathrm{ambig}}$0

where $\mathcal{P}_{\mathrm{ambig}}$1 is the lead paragraph of the top-ranked Wikipedia entry (Mu et al., 24 Nov 2025). The retrieved set $\mathcal{P}_{\mathrm{ambig}}$2 is concatenated and formatted into the prompt segment

$\mathcal{P}_{\mathrm{ambig}}$3

The paper states that no additional similarity ranking or embedding-based retrieval is used; the system relies on the MediaWiki Action API, restricted to pre-May 1 2025 content (Mu et al., 24 Nov 2025).

This design is narrower than retrieval pipelines that employ dense retrieval or re-ranking. Here, the retrieval component is deliberately lightweight and factual rather than learned. For low-resource NER, the main function of the knowledge snippets is to compensate for insufficient domain knowledge in the backbone LLM, especially when domain-specific mentions are poorly represented in the model’s internal knowledge (Mu et al., 24 Nov 2025).

The disambiguation process is also formulated in natural language rather than through a learned scoring function. Given the input $\mathcal{P}_{\mathrm{ambig}}$4 and the planner’s $\mathcal{P}_{\mathrm{ambig}}$5, the Disambiguation Agent performs a “self-check” dialogue for each mention $\mathcal{P}_{\mathrm{ambig}}$6 to explain its sense in context:

$\mathcal{P}_{\mathrm{ambig}}$7

The output is a set of natural-language interpretation statements, and the paper explicitly notes that it does not introduce an explicit scoring or ranking function for this stage (Mu et al., 24 Nov 2025). This suggests that KDR-Agent treats ambiguity resolution as a prompt-mediated reasoning problem rather than a separate classification submodule.

4. Reflective Analysis and Correction

The reflection mechanism is designed to detect and correct four common error types: span errors, type errors, spurious detections, and omissions (Mu et al., 24 Nov 2025). After the initial prediction $\mathcal{P}_{\mathrm{ambig}}$8, the Reflective Analysis Agent generates a structured report

$\mathcal{P}_{\mathrm{ambig}}$9

which is then serialized into $k_i \;=\;\mathrm{RetrieveFromWiki}(q_i),$0, and a final correction prompt $k_i \;=\;\mathrm{RetrieveFromWiki}(q_i),$1 is constructed (Mu et al., 24 Nov 2025).

The paper provides a pseudocode sketch in which the agent iterates over predicted mentions in $k_i \;=\;\mathrm{RetrieveFromWiki}(q_i),$2, records boundary mismatches and type mismatches with justifications, then identifies missing gold mentions and spurious predicted mentions, aggregates the findings into $k_i \;=\;\mathrm{RetrieveFromWiki}(q_i),$3, and outputs the structured report (Mu et al., 24 Nov 2025). Although the pseudocode references gold mentions and gold boundaries, the operational description in the framework is that the agent performs structured self-assessment by comparing $k_i \;=\;\mathrm{RetrieveFromWiki}(q_i),$4 with the input sentence $k_i \;=\;\mathrm{RetrieveFromWiki}(q_i),$5 under the reflection guideline (Mu et al., 24 Nov 2025).

Within the framework, reflection is not merely explanatory. It is part of a two-pass inference protocol in which the first-pass output is treated as revisable. The ablation results indicate that this component has the largest single impact among the major modules, which supports the interpretation that explicit post hoc diagnosis is central to KDR-Agent’s effectiveness (Mu et al., 24 Nov 2025).

5. Prompt Construction: Type Definitions and Contrastive Demonstrations

KDR-Agent supplements retrieved knowledge and disambiguation rationale with two additional prompt components: natural-language type definitions and entity-level contrastive demonstrations (Mu et al., 24 Nov 2025).

For a target type set $k_i \;=\;\mathrm{RetrieveFromWiki}(q_i),$6, each type $k_i \;=\;\mathrm{RetrieveFromWiki}(q_i),$7 is associated with a textual description $k_i \;=\;\mathrm{RetrieveFromWiki}(q_i),$8 specifying scope and inclusions or exclusions. These are concatenated into

$k_i \;=\;\mathrm{RetrieveFromWiki}(q_i),$9

(Mu et al., 24 Nov 2025). This provides explicit label semantics in the prompt, reducing dependence on latent label understanding.

The framework also uses a static support set $\{k_i\}$0, where each $\{k_i\}$1 contains both positive and negative mention–type pairs:

$\{k_i\}$2

The negative examples simulate four error types: boundary shifts, wrong labels, spurious mentions, and omissions (Mu et al., 24 Nov 2025). These demonstrations are formatted into

$\{k_i\}$3

The paper states that, although no explicit loss function is trained, these contrastive pairs encourage the LLM to distinguish correct from incorrect spans and types in context (Mu et al., 24 Nov 2025).

A notable feature of this design is the use of static rather than dynamically retrieved demonstrations. This directly addresses the paper’s first stated issue—reliance on large annotated support sets—and shifts the burden from retrieval over many labeled exemplars to richer prompt structure and auxiliary reasoning (Mu et al., 24 Nov 2025).

6. Experimental Setup and Empirical Results

The experimental evaluation covers 10 datasets spanning 5 domains: Biomedical (BC5CDR, NCBI), Task-oriented Dialogue (MIT Movie, MIT Restaurant), News (CoNLL-2003, OntoNotes 5.0), Social Media (Twitter Broad, Twitter NER-7), and Open-domain (WikiANN English subset, sampled 5 K; WNUT-17) (Mu et al., 24 Nov 2025). In the low-resource setting, the shared static support set uses $\{k_i\}$4 for datasets with many types—MIT Movie, MIT Restaurant, and OntoNotes 5.0—and $\{k_i\}$5 for the remaining seven datasets (Mu et al., 24 Nov 2025). The LLM backbones evaluated are GPT-4o, Qwen-2.5-72B, and DeepSeek-V3 (Mu et al., 24 Nov 2025).

Selected GPT-4o results reported in the paper are summarized below.

Model	BC5CDR	NCBI	MIT Movie	MIT Rest.	CoNLL	OntoNotes
Zero-Shot (CMAS)	73.21	69.91	67.61	51.64	78.31	60.79
Few-Shot (CodeIE)	77.61	71.97	70.67	56.93	83.01	65.67
KDR-Agent	82.47	79.41	76.16	69.98	83.34	71.85

The paper reports absolute gains of 4–8 F1 pts over the strongest few-shot baselines (CodeIE / GPT-NER) and states that improvements are consistent across all ten datasets and all three backbones (Mu et al., 24 Nov 2025). Examples given include Qwen-2.5-72B: BC5CDR 81.45 vs 76.10 (CodeIE) and DeepSeek-V3: MIT Restaurant 65.99 vs 58.11 (Mu et al., 24 Nov 2025). It also notes that formal p-values are not reported, although the improvements are described as large and consistent (Mu et al., 24 Nov 2025).

These results position KDR-Agent as a strong low-resource ICL baseline for cross-domain NER. The empirical pattern suggests that the combination of retrieved factual context, explicit ambiguity handling, and reflective correction is beneficial even when the backbone model is already a strong general-purpose LLM.

7. Ablations, Component Contributions, and Scope

The ablation study, reported on NCBI, OntoNotes 5.0, and Twitter NER-7 with the GPT-4o backbone, evaluates the contributions of the major components (Mu et al., 24 Nov 2025).

Ablation	NCBI	OntoNotes	Twitter-7
Full KDR-Agent	79.41	71.85	60.87
– Reflection Stage	75.91	70.17	57.81
– Knowledge Retrieval (KRA)	76.21	71.70	59.34
– Disambiguation Agent (DA)	75.49	70.73	55.81
– KRA + DA removed	74.16	69.94	55.07
– No Contrastive Negatives	78.36	70.69	58.99

The paper’s stated insights are that Reflective Analysis has the largest single impact (−3–4 F1 when removed), that Biomedical and Social Media suffer most when KRA or DA are ablated, and that contrastive negative examples yield a modest but consistent +1 F1 uplift (Mu et al., 24 Nov 2025). These findings indicate that the framework’s gains are not attributable to a single prompt engineering trick; rather, they arise from the interaction of multiple targeted components.

The paper does not explicitly list limitations or future directions (Mu et al., 24 Nov 2025). It does, however, include reader-noted possible extensions: broader external knowledge sources beyond Wikipedia, more fine-grained disambiguation criteria or learned scoring functions, and automatic expansion of contrastive demonstration pools without manual annotation (Mu et al., 24 Nov 2025). Because these are explicitly marked as not being paper-authored limitations, they should be understood as plausible implications rather than claims of the work itself.

A potential misconception is that KDR-Agent is a generic “knowledgeable deep research” system. In fact, despite the acronym overlap with later work on Knowledgeable Deep Research, KDR-Agent in this context denotes a specialized NER framework whose components are tailored to low-resource, multi-domain in-context entity extraction rather than to long-form report generation (Mu et al., 24 Nov 2025).