--- title: 'PU-ADKA: Expert-Driven LLM Adaptation' url: https://www.emergentmind.com/topics/pu-adka type: topic --- # PU-ADKA: Expert-Driven LLM Adaptation PU-ADKA, short for **Positive Unlabeled Active Domain Knowledge Acquisition**, is a budget-aware, expert-in-the-loop framework for improving domain-specific large language models in highly specialized and cost-sensitive domains such as drug discovery and rare disease research. It treats expert interaction as a constrained resource and jointly optimizes **which questions to annotate** and **which expert should answer each question**, under a fixed monetary budget such as \$100. The framework combines **positive–unlabeled question–expert matching** with **multi-agent reinforcement learning** and uses the acquired expert answers as supervised fine-tuning data for a base LLM, rather than as preference data in an RLHF pipeline [2508.17202]. ## 1. Problem formulation and motivating setting PU-ADKA is designed for settings in which base LLMs, including biomedical LLMs, have a fixed knowledge cutoff and often miss dynamic, mechanistic, and tacit expert knowledge. The paper instantiates this setting with **Llama2-7B** as a base “predated” biomedical model whose knowledge stops around early 2023, while the target domain knowledge comes from **2024 PubMed papers** and expert expertise [2508.17202]. The framework targets domains where knowledge is rapidly evolving, true experts are few and busy, and each consultation is costly. The motivating examples include **cancer drug development**, including NK cells, CAR-T, and mRNA vaccines, as well as **sepsis and rare disease mechanisms**. In these regimes, standard strategies are characterized as suboptimal under tight budgets: large-corpus fine-tuning captures static textual knowledge but not nuanced expert judgment; generic human-feedback pipelines do not account for heterogeneous expert competence and cost; and naive expert querying wastes budget on easy, redundant, or poorly routed questions [2508.17202]. Formally, PU-ADKA assumes an unlabeled question pool $D_{tr} = \{q_i\}_{i=1}^n$, an expert set $\mathcal{E} = \{e_j\}_{j=1}^m$, a fixed budget $B$, and a base LLM $\theta$. It seeks a selected set of question–expert pairs $S \subseteq D_{tr} \times \mathcal{E}$ that maximizes downstream performance after fine-tuning: $$ S^* = \arg\max_{S \subseteq D_{tr} \times \mathcal{E}} F(\theta_S, D_{te}) $$ subject to $$ \sum_{(q_i, e_j) \in S} c(q_i, e_j) \le B. $$ Here, $\theta_S$ is the LLM fine-tuned on annotations from $S$, $F(\theta_S, D_{te})$ is an evaluation metric such as **Win Rate** or **LC_WR**, and $c(q_i,e_j)$ is the cost of having expert $e_j$ annotate question $q_i$ [2508.17202]. This formulation makes PU-ADKA a dual active-learning system: it performs **active question selection** and **active expert selection** simultaneously. ## 2. Expert modeling and the positive–unlabeled matching layer A central design choice in PU-ADKA is to model expert assignment as a **positive–unlabeled (PU) classification** problem. The paper’s rationale is that if a question is extracted from a paper authored by expert $e_j$, then $(q_i,e_j)$ is a **positive pair**. For other experts, however, the corresponding pairs are not reliable negatives, because another expert may still be capable of answering the question. The unlabeled region therefore represents uncertainty in competence, not absence of competence [2508.17202]. Experts are represented through several structures. The paper defines an **expert capability matrix** $A \in \{0,1\}^{Q \times N}$, where $A_{ji}=1$ if expert $e_j$ is capable of annotating question $q_i$. In simulation, the top 20 PubMed authors by publication count are treated as proxy experts, and questions are mechanism-focused QA pairs extracted from PubMed papers. Expert cost is heterogeneous: in simulation, per-question costs are proportional to cumulative journal impact factors, with example prices **$[0.5, 0.4, 0.3, 0.2, 0.1]$**; in the human study, costs are **[$0.5, $0.2, $0.1, $0.1, $0.1]** to reflect doctor versus senior and junior PhD cost ratios [2508.17202]. Question and expert representations are derived from **Llama2-7B** embeddings. The question embedding $E_q$ comes from the last hidden layer applied to question text, and the expert embedding $E_e$ is the average embedding of the expert’s publication texts. An **expert-wise attention network** then forms an expert-aware question representation: $$ c_{ij} = \sigma(W[E_q, E_e^j] + b), $$ $$ a_{ij} = \frac{\exp(\sigma(W[E_q, E_e^j] + b))}{\sum_{k \in \mathcal{E}} \exp(\sigma(W[E_q, E_e^k] + b))}, $$ $$ Z_i = \sum_{k \in \mathcal{E}} a_{ik} E_e^k. $$ The model then computes $$ P(E_q, E_e^j) = Y([Z_i, E_e^j]), $$ where $Y$ is an MLP producing the probability that expert $e_j$ is a good match for question $q_i$ [2508.17202]. The “PU” in PU-ADKA therefore refers specifically to **positive–unlabeled question–expert matching**. This differs from soft-label PU formulations that assign real-valued positivity scores to unlabeled instances; a related but distinct example is “Soft Label PU Learning” [2405.01990], which operates on instance labels rather than expert-routing decisions. ## 3. PU risk minimization and suitability scoring The question–expert matcher is trained with a **non-negative PU risk estimator**. Let $x_p = (q_i,e_j)$ denote positive pairs and $x_u$ unlabeled pairs, with $n_p$ positive samples, $n_u$ unlabeled samples, and prior probability $T_p = 0.1$. The training objective is $$ \mathrm{Risk}_{PU}(g) = \frac{1}{n_p} \sum_{i=1}^{n_p} \ell(g(x_p^i), +1) + \frac{1}{n_u} \sum_{k=1}^{n_u} \max \left(0,\, \frac{1}{n_u} \sum_{k=1}^{n_u} \ell(g(x_u^k), -1) - \frac{T_p}{n_p} \sum_{i=1}^{n_p} \ell(g(x_p^i), -1) \right). $$ The first term evaluates risk on known positives, while the second approximates the negative risk from unlabeled and positive distributions and clips it at zero to avoid negative estimates [2508.17202]. This yields a suitability score $g(q_i,e_j)$ used throughout the rest of the framework. In operational terms, the PU learner estimates which expert is likely to provide high-value supervision for a given question even when explicit negative supervision is unavailable. A plausible implication is that PU-ADKA’s matching layer is most valuable in domains where expertise overlaps partially rather than partitioning cleanly, because unlabeled question–expert pairs remain semantically ambiguous rather than truly negative. The paper reports an encoder ablation showing that **Llama2-7B** embeddings outperform **BERT-base** embeddings for this stage: with BERT-base, performance is **WR 16.3%** and **LC_WR 24.0%**, while with Llama2-7B embeddings it is **WR 18.2%** and **LC_WR 25.6%** on CKAD under the reported evaluation setup [2508.17202]. ## 4. Multi-agent reinforcement learning under a fixed budget Once the framework has question–expert suitability scores, it treats actual data acquisition as a **multi-agent reinforcement learning** problem. The RL state includes three variables: the matching score $g(q_i,e_j)$, the remaining budget $B_t$, and an expert sampling weight $$ w_j^t = \frac{B_t}{c(q_i, e_j)} \times (1 - \alpha \Gamma_j^t), $$ where $\Gamma_j^t$ is the number of times expert $e_j$ has already been selected and $\alpha$ is a decay factor encouraging less frequent expert reuse [2508.17202]. Because multiple agents may choose the same question, PU-ADKA uses a competition rule: $$ \mathrm{Compete}(q_i \mid e_1,\dots,e_h) = e_v \quad \text{where} \quad e_v = \arg\max_{e_j \in \{e_1,\dots,e_h\}} g(q_i,e_j). $$ The expert with the highest PU score receives the question; other agents must reselect until all chosen assignments are unique [2508.17202]. Reward design explicitly encodes **performance gain per unit cost** while favoring question diversity: $$ r_t = \sum_{(q_i, e_j) \in S_t} \frac{\Delta F_t \cdot D_i}{\sum_{(q_i, e_j) \in S_t} c(q_i, e_j)}, $$ with diversity score $$ D_i = \min_{q_z \in S_{