---
title: ReCAT Model for Clinical Relation Extraction
url: https://www.emergentmind.com/topics/recat-model
type: topic
---

# ReCAT Model for Clinical Relation Extraction

The ReCAT (Relation Concept Annotation Toolkit) model is an interactive annotation, inference, and training framework for classifying entity-to-entity relations in clinical narratives, specifically those embedded in unstructured Electronic Health Records (EHR). Originating as a major extension to the CogStack MedCAT system, RelCAT addresses the complexity of clinical information spread over textual data—where relations among drugs, findings, and procedures are often diffuse and context-dependent. RelCAT provides flexible annotation, state-of-the-art model architectures, robust evaluation metrics, and reproducible code for advancing clinical relation extraction research [2501.16077].

## 1. System Architecture and Workflow

RelCAT is structured as a two-stage pipeline augmenting the established MedCAT entity recognition and linking (NER+L) system:

- **Stage I (MedCAT):** Raw clinical text is tokenized, medical entities are recognized and mapped to concept unique identifiers (CUIs, SCTIDs) using databases like SNOMED-CT/UMLS, and context meta-features such as negation and temporality are detected.
- **Stage II (RelCAT):** The system accepts pre-extracted entities with character spans and CUIs, generates all candidate entity pairs within a configurable distance, extracts context windows around each candidate pair, and applies transformer-based models (BERT or Llama) for relation classification. The output is a discrete set of relation labels for each entity pair.

Annotation is carried out through an integrated MedCATTrainer module, which allows annotators to:
- Mark entities (with CUI and TUI filtering to enforce ontology constraints).
- Specify or correct relations between entity pairs, including relation type, valid distance, and context span.
- Benefit from automatic generation of negative (non-relation) examples using TUI-based ontology priors.

## 2. Input Encoding and Preprocessing

Input text undergoes transformer-compatible tokenization, using either WordPiece (BERT) or Byte-Pair Encoding (Llama3). Special markers ([s1], [e2]) may be inserted to demarcate entities explicitly in the token sequence $\{ t_1, \ldots, t_n \}$. Context is limited to $\pm K$ tokens around entity boundaries to control inference window size.

Hidden states $H \in \mathbb{R}^{n \times d}$ are extracted from the transformer backbone. For multi-token entities, representations are constructed via max-pooling over the relevant token vectors. Optionally, the global sequence-embedding $h_{\text{cls}}$ may also be incorporated. MedCAT’s linked concept embedding underpins negative sample generation and CUI-filtered annotation.

## 3. Model Structure and Mathematical Framework

Both BERT and Llama variants implement a unified architecture:

- **Encoding:** $X = [x_1, \ldots, x_n]$, where $x_i \in \mathbb{R}^d$.
- **Transformer Forward:** $H = \text{Transformer}(X)$.
- **Entity Representation:** For entities occupying indices $E^1, E^2$:
  - $h_{e^1} = \max_{i \in E^1} H_i$
  - $h_{e^2} = \max_{i \in E^2} H_i$
  - Concatenated vector $v = [h_{e^1}; h_{e^2}] \in \mathbb{R}^{2d}$ (or $[h_{e^1}; h_{e^2}; h_{\text{cls}}] \in \mathbb{R}^{3d}$).
- **Classification:** $z = W_2 \cdot \text{ReLU}(W_1 v + b_1) + b_2$, $z \in \mathbb{R}^C$.
- **Softmax and Loss:**
  - $p_j = \frac{e^{z_j}}{\sum_k e^{z_k}}$ for class $j=1,\ldots, C$.
  - Cross-entropy: $L = -\sum_j y_j \log p_j$ (weighted: $L_{\text{weighted}} = -\sum_j \alpha_j y_j \log p_j$ for class-imbalance).

No custom attention mechanisms are employed; standard transformer attention is retained, with a two-layer MLP head for classification.

## 4. Training Regimen and Dataset Details

RelCAT is evaluated on both open and proprietary datasets:

- **n2c2 2018:** 505 discharge summaries, 8 drug-relation classes, $\sim72$k relations.
- **NHS Spatial:** 119 radiology/pathology reports, 613 spatial relations, $\sim70$ negatives.
- **NHS Physiotherapy-Mobility:** 486 physiotherapy notes, 278 single-instance relations, $\sim70$ negatives.

Preprocessing pipelines apply MedCAT NER+L using SNOMED CT for CUI linkage. Relation pairs are generated within a character distance threshold; negative sampling is ontology-driven using TUI filters.

Optimization settings:
- AdamW ($\beta_1=0.9$, $\beta_2=0.999$, $\epsilon=1e-8$).
- Learning rates: $2e-5$ (BERT), $1e-5$ (Llama).
- Batch size: $16-32$.
- Epochs: $3-5$.
- Warmup: $10\%$ steps.
- Class weighting and stratified batching mitigate imbalances.

Annotation proceeds by launching MedCATTrainer with CUI/TUI filters, marking entities and subsequently relations, and exporting the annotated corpus for model training.

## 5. Evaluation Metrics and Comparative Analysis

Performance is measured per class and averaged macro/micro:

- Precision: $\text{Precision}_j = \frac{TP_j}{TP_j + FP_j}$
- Recall: $\text{Recall}_j   = \frac{TP_j}{TP_j + FN_j}$
- F1: $F1_j = 2 \cdot \frac{\text{Precision}_j \cdot \text{Recall}_j}{\text{Precision}_j + \text{Recall}_j}$
- Macro/micro average and overall accuracy computed as described above.

### Performance Summary

| Dataset                  | Model                      | F1 Macro  | Accuracy | SOTA Reference |
|--------------------------|----------------------------|-----------|----------|----------------|
| n2c2 (gold-standard)     | BERT (unfrozen)            | 0.977     | 0.977    | 0.956 / 0.961  |
| NHS Spatial              | BERT (unfrozen)            | 0.902     | —        | —              |
| NHS Spatial              | n2c2-pretrained BERT       | 0.918     | —        | —              |
| NHS Spatial              | Llama (frozen)             | 0.933     | —        | —              |
| NHS Physio-Mobility      | BERT (unfrozen)            | 0.905     | —        | —              |
| NHS Physio-Mobility      | n2c2-pretrained BERT       | 0.938     | —        | —              |
| NHS Physio-Mobility      | Llama (frozen)             | 0.835     | —        | —              |

Minority classes on n2c2 (ADE-Drug, Duration-Drug) achieve F1 = 0.866 and 0.933. Zero/few-shot in-context LLMs (Llama 3.1, Mistral 7B) underperform at F1 = 0.29–0.49, particularly in minority classes.

## 6. Practical Guidance and Known Limitations

Deployment on a new corpus requires sequential integration with MedCAT, initial relation annotation, strategic negative sampling, transformer fine-tuning, and API-driven inference:

1. Integrate corpus into CogStack MedCAT; build/load NER+L for target ontology.
2. Annotate small seed of entity-to-entity relations.
3. Export labeled data; configure negative sampling.
4. Fine-tune transformer (BERT/Llama) using toolkit protocols.
5. Deploy via RelCAT API: text → MedCAT NER+L → RelCAT classifier → output relations as JSON.

Key limitations include heuristic, context-sensitive definition of non-relations, Llama overfitting on limited data, and a pipeline architecture that may decouple named entity recognition and relation signals. Future work aims at ontology-driven candidate relation discovery, extension to cross-sentence relations, and end-to-end architectures for integrated NER and relation extraction.

## 7. Availability and Reproducibility

Source code, annotation pipelines, and documentation are publicly available in the CogStack/MedCAT GitHub repository (https://github.com/CogStack/MedCAT). This enables full reproducibility of reported results, annotation workflows, and model evaluations.

RelCAT represents a high-fidelity, transformer-based approach for clinical relation extraction, exceeding prior state-of-the-art benchmarks and providing a comprehensive open-source toolkit for the community [2501.16077].

Source: https://www.emergentmind.com/topics/recat-model