ReProCon: Scalable Few-Shot Biomedical NER

• ReProCon is a few-shot biomedical NER framework that leverages multi-prototype modeling, cosine-contrastive learning, and Reptile meta-learning to address data scarcity and class imbalance.
• It employs a lightweight fastText + BiLSTM encoder achieving macro-F1 performance close to BERT-based baselines while using significantly lower memory resources.
• The framework’s design supports scalability to increasing entity types and demonstrates robust performance with reduced label budgets, as shown in detailed ablation studies.

to=arxiv_search.search аԥсద్దict qwargs={"^^^^^^^^2query2 Scalable and Resource-Efficient Few-Shot Biomedical Named Entity Recognition\"","max_results"^^^^8submittedDate7^^^^"sort_by":"submittedDate"}【อ่านข้อความเต็มjson string to=arxiv_search.search 天天中彩票和 天天中彩票任选={"results":&&&^^^^2query2^^^^&&&)","title":"ReProCon: Scalable and Resource-Efficient Few-Shot Biomedical Named Entity Recognition","authors":["Yash Bansod","Duan Min","Alex Vitkin","James Brown","Ronald de Bruin","Jitendra Jonnagaddala"],"abstract":"Named Entity Recognition (NER) in biomedical domains faces challenges due to data scarcity and imbalanced label distributions, especially with fine-grained entity types. We propose ReProCon, a novel few-shot NER framework that combines multi-prototype modeling, cosine-contrastive learning, and Reptile meta-learning to tackle these issues. By representing each category with multiple prototypes, ReProCon captures semantic variability, such as synonyms and contextual differences, while a cosine-contrastive objective ensures strong interclass separation. Reptile meta-updates enable quick adaptation with little data. Using a lightweight fastText + BiLSTM encoder with much lower memory usage, ReProCon achieves a macro-F^^^^2ti:\2^^^^ score close to BERT-based baselines (around 99 percent of BERT performance). The model remains stable with a label budget of 3^^^^2query2^^^^ percent and only drops 7.8 percent in F^^^^2ti:\2^^^^ when expanding from ^^^^2ti:\2^^^^9 to 5^^^^2query2^^^^ categories, outperforming baselines such as SpanProto and CONTaiNER, which see ^^^^2ti:\2query2^^^^ to 32 percent degradation in Few-NERD. Ablation studies highlight the importance of multi-prototype modeling and contrastive learning in managing class imbalance. Despite difficulties with label ambiguity, ReProCon demonstrates state-of-the-art performance in resource-limited settings, making it suitable for biomedical applications.","categories":["cs.CL","cs.LG"],"published":"2^^^^2query22^^^^5-^^^^2query2 ReProCon is a few-shot biomedical named entity recognition (NER) framework designed for settings characterized by data scarcity, class imbalance, semantic variability, and constrained compute resources. It combines multi-prototype modeling, a cosine-contrastive objective, and Reptile meta-learning, and is instantiated with a lightweight fastText + BiLSTM encoder rather than a transformer-heavy default. In the reported experiments, ReProCon achieves a macro-PRESERVED_PLACEHOLDER_^^^^2query2^^^^ score close to BERT-based baselines, remains stable with a label budget of 3^^^^2query2^^^^ percent, and drops only 7.8 percent in PRESERVED_PLACEHOLDER_^^^^2ti:\2^^^^ when expanding from ^^^^2ti:\2^^^^9 to 5^^^^2query2^^^^ categories, while also addressing label ambiguity through a PageRank-based co-occurrence graph during training (&&&^^^^2ti:\2^^^^&&&).

^^^^2ti:\2^^^^. Problem setting and design rationale

Biomedical NER presents a particularly difficult few-shot learning regime because fine-grained entity types are both numerous and unevenly distributed. The framework is motivated by three linked problems identified in that regime: limited labeled samples, imbalanced label distributions, and substantial intra-class semantic variability. The latter includes both synonyms and contextual differences, which make a single centroid-style representation of a category insufficient for robust span classification.

ReProCon addresses these problems through a composite design. Multi-prototype modeling is used to represent each category with several class centroids rather than one. A cosine-contrastive learning objective is used instead of standard cross-entropy, with the stated aim of producing strong interclass separation while still aligning spans with plausible in-class representatives. Reptile provides the meta-learning component, allowing rapid adaptation with little data while avoiding the memory and compute overhead associated with second-order methods such as MAML. This combination places ReProCon within the broader lineage of prototypical few-shot learning and first-order meta-learning, but with explicit emphasis on scalability and resource efficiency in biomedical settings.

The framework is evaluated on large biomedical corpora, including MedMentions, and in settings with up to 5^^^^2query2^^^^ entity types. This suggests a design target broader than narrowly parameterized benchmark episodes: the method is intended to remain functional as label spaces expand rather than only within fixed low-way few-shot episodes.

2. Multi-prototype representation and encoder architecture

A central departure from classic prototypical networks is the use of multiple prototypes per class. ReProCon defines a prototype tensor

$\mathbf{P} \in \mathbb{R}^{(N \times M) \times D},$

where $N$ is the number of categories, $M$ is the number of prototypes per category, and $D$ is the prototype or embedding dimension, exemplified as 5^^^^2query2^^^^. The stated rationale is that biomedical concepts such as “Protein” can appear in highly heterogeneous forms, including gene symbols and multi-word mentions, so a single prototype induces representational collapse for rare or contextually atypical mentions.

To prevent prototype overlap, the method introduces a prototype repulsion loss:

$$\mathcal{L}_{\text{proto}} = \frac{1}{N M} \sum_{i=0}^{N M-1} \max_{j \ne i} \bigl(\cos(\mathbf{p}_i, \mathbf{p}_j) + 1\bigr).$$

This term penalizes the most similar other prototype for each prototype, thereby encouraging angular diversity. In the paper’s interpretation, this diversity is important not only across classes but also within a class, where multiple prototypes are expected to encode distinct semantic submodes rather than redundant copies (&&&^^^^2ti:\2^^^^&&&).

The reported lightweight backbone consists of static fastText embeddings of 3^^^^2query2query2^^^^ dimensions, sinusoidal position encodings of 2^^^^2query2query2^^^^ dimensions, and a single-layer BiLSTM with $h = 1024$, followed by projection to a 5^^^^2ti:\22^^^^-dimensional representation space. As a comparator, the study also includes a BERT encoder using BERT-base-cased with full-sequence encoding and mean-max pooling. Span representation is obtained either from the BiLSTM hidden state at the marker in the fastText + BiLSTM configuration or from a pooled BERT hidden state over the marked span.

The architectural significance of this design lies in its stated memory profile. The fastText + BiLSTM encoder is described as much smaller and less resource-hungry than BERT-based alternatives, while still achieving performance close to transformer baselines.

3. Cosine-contrastive objective and Reptile adaptation

ReProCon replaces standard cross-entropy with a supervised contrastive formulation based on cosine distance. For ^^^^^^^^2query2^^^^^^^^ spans $\{\mathbf{q}_j\}$ and prototypes $\mathbf{p}_i$, it defines

PRESERVED_PLACEHOLDER_^^^^2ti:\2query2^^^^

For each category PRESERVED_PLACEHOLDER_^^^^2ti:\2ti:\2^^^^, the distance between a ^^^^^^^^2query2^^^^^^^^ span and that category is the minimum distance to any of its PRESERVED_PLACEHOLDER_^^^^2ti:\22^^^^ prototypes:

PRESERVED_PLACEHOLDER_^^^^2ti:\2^^^^3

The span alignment loss is then

PRESERVED_PLACEHOLDER_^^^^2ti:\2^^^^4

The total loss is

PRESERVED_PLACEHOLDER_^^^^2ti:\2^^^^5

In the accompanying interpretation, this objective ensures that each span is close to some prototype of its own class while preserving separation from other categories. The category-wise pooling and minimum-distance mechanism are also presented as a way to mitigate gradient domination by frequent classes, which is especially relevant under biomedical label imbalance.

The meta-learning component uses Reptile. After inner-loop task adaptation, the meta-update is

PRESERVED_PLACEHOLDER_^^^^2ti:\2^^^^6

where PRESERVED_PLACEHOLDER_^^^^2ti:\2^^^^7 denotes meta-parameters, PRESERVED_PLACEHOLDER_^^^^2ti:\2^^^^8 the parameters before task adaptation, PRESERVED_PLACEHOLDER_^^^^2ti:\2^^^^9 the parameters after task adaptation, and $\mathbf{P} \in \mathbb{R}^{(N \times M) \times D},$^^^^2query2^^^^ the meta step size. Tasks are constructed episodically by drawing $\mathbf{P} \in \mathbb{R}^{(N \times M) \times D},$^^^^2ti:\2^^^^ categories and $\mathbf{P} \in \mathbb{R}^{(N \times M) \times D},$2 support samples per category. The explicit advantage claimed for Reptile is that it avoids second-order or Hessian computations, which in turn reduces VRAM and compute demands relative to MAML-style procedures.

4. Empirical performance, scalability, and resource profile

The empirical results are reported in macro-$\mathbf{P} \in \mathbb{R}^{(N \times M) \times D},$3 terms for ^^^^2ti:\2^^^^9-way and 5^^^^2query2^^^^-way settings. In Table ^^^^2ti:\2^^^^, the fastText configuration yields 47.95 for ^^^^2ti:\2^^^^9-way and 49.45 for 5^^^^2query2^^^^-way, while the BERT configuration yields 47.9^^^^2query2^^^^ and 52.^^^^2query2ti:\2^^^^, respectively. The abstract summarizes the lightweight model’s quality as being around 99 percent of BERT performance. At the same time, the fastText + BiLSTM system is described as operating with much lower memory usage and much lower memory or inference cost than BERT-based models (&&&^^^^2ti:\2^^^^&&&).

The framework is also presented as robust to constrained supervision. Macro-$\mathbf{P} \in \mathbb{R}^{(N \times M) \times D},$4 is reported as stable or improving as the support set shrinks from 8^^^^2query2^^^^ percent to 3^^^^2query2^^^^ percent per category, and the abstract specifically states that the model remains stable with a label budget of 3^^^^2query2^^^^ percent. This is one of the principal grounds on which the method is characterized as suitable for resource-limited biomedical applications.

Scalability across label-space size is another explicit focus. When increasing entity types from ^^^^2ti:\2^^^^9 to 5^^^^2query2^^^^, the paper reports only a 7.8 percent performance drop, corresponding to $\mathbf{P} \in \mathbb{R}^{(N \times M) \times D},$5. The comparison given in the paper is to reported degradations of 9.9 percent for SpanProto and ^^^^2ti:\2^^^^8 to 32 percent for CONTaiNER in Few-NERD. The argument advanced by the authors is that the supervised contrastive multi-prototype formulation helps preserve decision boundaries as the number of categories grows.

All experiments are reported as running on a single GPU, specifically an NVIDIA L4 with 22GB VRAM. This hardware detail is important because the paper’s central claim is not merely accuracy under few-shot conditions, but accuracy under those conditions with a comparatively lightweight computational footprint.

5. Ablations and what they indicate

The ablation study is used to isolate the contribution of prototype multiplicity, contrastive learning, and hard-negative mining. Table 3 reports the following macro-$\mathbf{P} \in \mathbb{R}^{(N \times M) \times D},$6 values: Full ReProCon, 5^^^^2query2^^^^.8^^^^2query2 single-prototype per category, 49.^^^^2ti:\2^^^^3; no hard-negative mining, 56.66; and cross-entropy loss, 2.7^^^^2query2^^^^ (&&&^^^^2ti:\2^^^^&&&).

Two conclusions are stated directly. First, using one prototype per class reduces $\mathbf{P} \in \mathbb{R}^{(N \times M) \times D},$7 by roughly ^^^^2ti:\2^^^^.7 points relative to the full model, which quantifies the value of modeling intra-class diversity. Second, replacing the contrastive formulation with plain cross-entropy leads to near-random performance, which the paper treats as evidence that contrastive, prototype-based learning is essential under severe data scarcity.

The hard-negative result is more nuanced. Disabling hard-negative mining increases performance in the reported ablation, and the authors attribute this to variance introduced by hard-negative sampling in highly limited-data regimes. Their interpretation is that the intended benefit of harder discrimination can be outweighed by destabilized gradients when examples are scarce. This constitutes an explicit caution against assuming that standard metric-learning heuristics transfer uniformly to biomedical few-shot NER.

Taken together, the ablations support a specific reading of the method: the gains are not presented as arising from meta-learning alone, but from the interaction between multi-prototype structure and the cosine-contrastive objective, with some commonly used auxiliary mechanisms proving counterproductive.

6. Label ambiguity, failure modes, and practical significance

Biomedical entity typing frequently involves ambiguity because the same span can have multiple valid UMLS types. The paper gives examples such as entities that can be labeled both “Anatomical Structure” and “Substance,” and it further notes recurring confusion between “Phenomenon or Process” and “Natural Phenomenon or Process.” ReProCon does not eliminate this ambiguity. Instead, during training, it uses a PageRank-based co-occurrence graph to assign the most central type among the possible labels for a span, treating the highest-PageRank type as the most general or representative one (&&&^^^^2ti:\2^^^^&&&).

This preprocessing step is presented as a noise-reduction strategy rather than a full solution to overlapping or hierarchical labels. The confusion matrix analysis described in the paper indicates that extremely fine-grained or hierarchically overlapping classes remain difficult even with flexible prototype representations. A plausible implication is that prototype multiplicity helps absorb some semantic variation, but cannot by itself resolve ontology-level ambiguity.

In practical terms, the framework is positioned for settings where annotation budgets are small, the number of entity types may expand, and GPU memory is limited. The paper describes ReProCon as demonstrating state-of-the-art performance in resource-limited settings and as being suitable for biomedical applications. That suitability follows directly from the conjunction of three reported properties: near-BERT macro-$\mathbf{P} \in \mathbb{R}^{(N \times M) \times D},$8, operation on a single NVIDIA L4 22GB GPU, and stability under a 3^^^^2query2^^^^ percent label budget. The broader methodological significance is that efficient few-shot biomedical NER need not be tied to large transformer encoders if prototype geometry, contrastive training, and first-order meta-learning are co-designed.