Med-Char: Biomedical NER via BiLSTM-CNN-Char

• Med-Char is a biomedical NER model that uses a modified BiLSTM-CNN-Char architecture to combine token-level and character-level features for enhanced entity recognition.
• It integrates efficiently into Spark NLP pipelines via a TensorFlow implementation, enabling distributed batch inference in production environments.
• Empirical results show state-of-the-art performance on eight biomedical benchmarks using domain-specific embeddings, outperforming heavier transformer models.

Searching arXiv for Med-Char and related biomedical NER papers to ground the article in the cited literature. Med-Char is a biomedical named entity recognition (NER) model associated with Spark NLP’s NerDLApproach, presented as a production-oriented reimplementation of the BiLSTM-CNN-Char sequence tagger architecture for biomedical text on Apache Spark. It combines pretrained word embeddings, character-level CNN features, and a bidirectional LSTM encoder to perform token-level sequence labeling without relying on heavyweight contextual transformers such as BERT (Kocaman et al., 2020). In the biomedical setting, its design is motivated by the orthographic and morphological irregularity of entity mentions, including numerals, hyphens, mixed capitalization, rare forms, and spelling variation, which make character-aware representations especially useful (Kocaman et al., 2020).

1. Definition and architectural identity

Med-Char is described as the practical biomedical NER model embodied in Spark NLP’s NerDLApproach, implemented as a modified BiLSTM-CNN-Char architecture adapted to biomedical text and integrated into Apache Spark (Kocaman et al., 2020). Its central objective is to recognize biomedical entities from tokenized sentences by combining token-level semantic information with subword morphological features.

At a high level, the model processes a sentence token by token. Each token is mapped to a pretrained word embedding and also to a character-derived vector produced by a convolutional neural network over the token’s character sequence. These two representations are concatenated and then passed through a bidirectional LSTM for contextual sequence encoding, after which token-level tag probabilities are predicted over BIO labels (Kocaman et al., 2020). The paper characterizes the architecture as “slightly modified” relative to the original BiLSTM-CNN-Char formulation, with the modifications directed toward speed, portability, and Spark deployment (Kocaman et al., 2020).

The role of character-level modeling is central rather than auxiliary. Biomedical terms often contain prefixes, suffixes, numerals, Greek letters, irregular casing, and other orthographic patterns that are informative for entity recognition. The paper states that character features are “highly useful in NER models” and provide “a level of immunity to typos and spelling errors,” a property that is particularly consequential for biomedical corpora with rare or unseen terms (Kocaman et al., 2020). This suggests that Med-Char should be understood not merely as a generic sequence tagger deployed in a biomedical setting, but as an architecture whose inductive bias is especially aligned with biomedical lexical structure.

2. Representation learning and sequence labeling pipeline

For a sentence $x = (x_1, \dots, x_n)$, each token $x_t$ is represented by a word embedding $w_t \in \mathbb{R}^{d_w}$. The paper reports biomedical embeddings of dimension 200 and GloVe embeddings of dimension 300 (Kocaman et al., 2020). Each token also has a character sequence $c_t = (c_{t,1}, \dots, c_{t,m_t})$, whose characters are embedded into a matrix of dimension 25 and processed by a 1D convolution with 25 filters of kernel size 3, followed by max pooling to obtain a fixed-size character vector $g_t$ (Kocaman et al., 2020).

The token representation is then formed as

$h_t^{(0)} = [w_t; g_t],$

with casing features also discussed in relation to the original framework. The paper notes that the original architecture used capitalization categories such as allCaps, upperInitial, lowercase, mixedCaps, and noinfo, and states that lexical features were removed while relying on pretrained embeddings, casing features, and char CNN features (Kocaman et al., 2020).

The contextual encoder is a bidirectional LSTM: $$\overrightarrow{h_t} = \mathrm{LSTM}_f(h_t^{(0)}, \overrightarrow{h_{t-1}}), \qquad \overleftarrow{h_t} = \mathrm{LSTM}_b(h_t^{(0)}, \overleftarrow{h_{t+1}})$$ with contextual token representation

$h_t = [\overrightarrow{h_t}; \overleftarrow{h_t}].$

The output of each direction is decoded by a linear layer and a log-softmax layer into tag log-probabilities, and the two directional vectors are added to produce the final output, following the original BiLSTM-CNN-Char design (Kocaman et al., 2020). In compact form, the tag distribution is written as

$p(y_t \mid x) = \mathrm{softmax}(W h_t + b).$

The paper does not describe a CRF layer; rather, decoding is presented as a linear and log-softmax classification process (Kocaman et al., 2020). This is a notable point because biomedical NER systems are often discussed in relation to BiLSTM-CRF or transformer-CRF variants. Med-Char, by contrast, is framed as a character-aware BiLSTM tagger whose empirical performance derives from representation quality and implementation efficiency rather than from structured decoding additions.

3. Spark NLP implementation and systems integration

A defining aspect of Med-Char is that its contribution is not limited to model architecture. The paper presents the implementation itself as central: a modified TensorFlow realization of the architecture is integrated into Spark NLP through Scala and the TensorFlow API (Kocaman et al., 2020). The LSTM implementation uses LSTMBlockFusedCell, described as a highly efficient implementation based on Zaremba et al., using a single TensorFlow operation for the entire LSTM and reported to be faster and more memory-efficient than LSTMBlockCell (Kocaman et al., 2020).

The training workflow is organized as a Spark ML pipeline. The paper gives a concise operational sequence: $$\text{read CoNLL data} \rightarrow \text{tokenize / sentence split} \rightarrow \text{load word embeddings} \rightarrow \text{NER training} \rightarrow \text{Spark ML pipeline fit}.$$ The illustrative configuration includes setMaxEpochs(10), setDropout(0.5), setLr(0.001), setPo(0.005), setBatchSize(8), and setValidationSplit(0.2) within NerDLApproach (Kocaman et al., 2020).

At inference time, the trained pipeline is applied to new text within the same Spark pipeline framework, allowing distributed batch inference on Spark clusters (Kocaman et al., 2020). The paper emphasizes that Spark NLP is the only NLP library among those discussed that can scale on Spark clusters, support Python, R, Scala, and Java, and be used in production-grade distributed systems (Kocaman et al., 2020). The scalability claim, however, is asymmetrical: deployment and inference are distributed, while training is not fully distributed across worker nodes.

This implementation orientation distinguishes Med-Char from work that is benchmark-centric but not deployment-centric. A plausible implication is that the model’s significance lies partly in reducing the gap between biomedical NER research benchmarks and enterprise or clinical pipeline integration.

4. Training regime, embeddings, and tagging choices

The paper trains separate models on eight biomedical benchmark datasets and tunes hyperparameters by random search (Kocaman et al., 2020). The reported best values are summarized below.

Component	Reported value
LSTM state size	200
Dropout	0.5
Batch size	8
Learning rate	0.001
Epochs	10–15
Optimizer	Adam
Learning-rate decay coefficient $x_t$0	0.005

The decayed learning rate is explicitly defined as

$x_t$1

This reflects a comparatively conventional optimization setup, with the model’s empirical performance attributed less to novel optimization heuristics than to architecture–embedding synergy and engineering implementation (Kocaman et al., 2020).

Two embedding families are compared within the same architecture. The first is Spark-Biomedical embeddings: 200-dimensional skip-gram embeddings trained on PubMed abstracts and case studies, with a vocabulary of 2.2 million. The second is Spark-GloVe6B embeddings: 300-dimensional pretrained GloVe vectors trained on Wikipedia and Gigaword (Kocaman et al., 2020). Coverage statistics strongly favor the biomedical embeddings, which average 99.5% coverage on the biomedical datasets, compared with 96.1% for GloVe (Kocaman et al., 2020). The paper further notes that the biomedical embeddings show consistently above 99% in-vocabulary coverage for both train and test sets.

The tagging scheme is also explicitly discussed. Although BIOES was found to converge quickly, it sometimes failed to generalize and got stuck in local minima, so the implementation used BIO instead (Kocaman et al., 2020). This is an important corrective to the common assumption that more expressive tagging schemes are always preferable in practice. Here, the choice of BIO is associated with better optimization stability in the reported experiments.

5. Benchmarks and empirical results

Med-Char is evaluated on eight public biomedical NER benchmarks: NCBI Disease, BC5CDR, BC4CHEMD, Linnaeus, Species800, JNLPBA, AnatEM, and BioNLP13-CG (Kocaman et al., 2020). These datasets cover entity categories including disease, chemical, species, anatomy, cell types, genes/proteins, and cancer genetics. Evaluation is performed using micro-averaged test F1 excluding O tokens (Kocaman et al., 2020).

With biomedical embeddings, the reported scores are as follows.

Dataset	F1
NCBI Disease	89.13
BC5CDR	89.73
BC4CHEMD	93.72
Linnaeus	86.26
Species800	80.91
JNLPBA	81.29
AnatEM	89.13
BioNLP13-CG	85.58

With GloVe6B embeddings, the same architecture yields 87.19 on NCBI Disease, 88.32 on BC5CDR, 92.32 on BC4CHEMD, 85.51 on Linnaeus, 79.22 on Species800, 79.78 on JNLPBA, 87.74 on AnatEM, and 84.30 on BioNLP13-CG (Kocaman et al., 2020).

The paper’s headline empirical claim is that the biomedical-embedding variant achieves state of the art on seven of the eight benchmarks and beats SciSpacy on all eight (Kocaman et al., 2020). The abstract reports explicit gains on BC4CHEMD to 93.72% with a 4.1% gain, Species800 to 80.91% with a 4.6% gain, and JNLPBA to 81.29% with a 5.2% gain (Kocaman et al., 2020). Against Stanza, Spark-Biomedical is reported to be ahead on NCBI Disease, BC5CDR, BC4CHEMD, Species800, JNLPBA, AnatEM, and BioNLP13-CG, with Stanza ahead only on Linnaeus (Kocaman et al., 2020).

A notable secondary result is that the model with general-purpose GloVe6B embeddings still exceeds Stanza on 4 out of 8 datasets and beats SciSpacy on all datasets (Kocaman et al., 2020). This suggests that the character-aware BiLSTM formulation contributes substantially to performance independently of domain-specific embeddings, although the biomedical embeddings remain consistently stronger.

6. Relation to contextual models, limitations, and interpretation

The paper explicitly situates Med-Char against contextual models such as BERT, BioBERT, ClinicalBERT, SciBERT, and PubMedBERT (Kocaman et al., 2020). Its position is not that contextual models are weak, but that they are computationally expensive to pretrain and run for inference, making them less practical under production constraints (Kocaman et al., 2020). Med-Char is therefore presented as evidence that a compact BiLSTM-CNN-Char architecture, efficiently implemented and paired with strong embeddings, can match or exceed benchmark performance without BERT-level complexity (Kocaman et al., 2020).

This positioning addresses a recurring misconception in biomedical NLP: that transformer-based contextualization is a prerequisite for competitive NER. The reported results complicate that view. In the empirical setting of the paper, a character-aware recurrent model achieves state-of-the-art results on seven of eight benchmarks without using heavy contextual embeddings (Kocaman et al., 2020). That does not imply that contextual models are generally inferior; rather, it indicates that architecture choice, embedding coverage, and deployment constraints remain decisive variables.

The principal limitation acknowledged is that training is not truly distributed across Spark worker nodes. Because of TensorFlow-in-JVM design constraints, training runs on the driver node using all available cores, which imposes speed and resource limitations (Kocaman et al., 2020). GPU support is available by adding CUDA versions of TensorFlow components, but the work does not present a full distributed training solution (Kocaman et al., 2020). The distinction between distributed inference and non-distributed training is essential for evaluating the paper’s systems claims accurately.

The broader significance of Med-Char lies in the conjunction of three properties: competitive benchmark performance, character-level robustness to biomedical orthography, and direct integration into a production-grade Spark NLP pipeline (Kocaman et al., 2020). This suggests a model class that is not only scientifically competitive on standard biomedical NER tasks but also operationally suited to downstream pipelines such as assertion status detection, entity resolution, relation extraction, and de-identification, which the paper identifies as core consumers of biomedical NER outputs (Kocaman et al., 2020).