HSMP-BERT: Hierarchical Clinical Extraction

• The paper introduces HSMP-BERT, a model that transforms unstructured Hebrew radiology reports on Crohn’s disease into 90 granular organ–finding labels using hierarchical tree-based querying.
• The methodology leverages Hebrew RoBERTa with Section Matching Prediction pretraining and prompt-based queries to align report sections with specific clinical findings.
• Key results demonstrate that hierarchical inference reduces runtime by 5.1× and achieves a mean F1 of 0.83, validating its efficiency and accuracy for clinical cohorts.

Hierarchical Structured Matching Prediction BERT (HSMP-BERT) is a prompt-based model for fine-grained extraction of structured clinical information from Hebrew free-text radiology reports in Crohn’s disease. It was developed for a low-resource-language setting in which multi-organ findings are sparsely represented, and it combines a Hebrew RoBERTa backbone, Section Matching Prediction (SMP) pretraining over report sections, prompt-based organ–finding queries, and hierarchical tree-based inference. In the reported administrative database study, the model was applied to 9,683 reports from Crohn’s patients imaged during 2010–2023 across Israeli providers, with a radiologist-annotated subset used to evaluate extraction into 90 structured labels per subject (Badash et al., 3 Sep 2025).

1. Clinical setting and label space

The stated goal of HSMP-BERT is to structure free-text Hebrew radiology reports on Crohn’s disease into granular organ–finding labels such as wall thickening in a specific gastrointestinal organ. The source corpus comprises 9,683 radiology reports, and a subset of 512 reports was manually annotated by radiologists for findings across six gastrointestinal organs and 15 pathologies, yielding 90 structured labels per subject (Badash et al., 3 Sep 2025).

The extraction problem is explicitly multi-label. Each report is assessed for all organ–finding pairs, and the reported analysis retained only two classes: presence or absence of a finding. In that formulation, “organ not visible” and “resected” were collapsed into the negative class. For evaluation, only organ–finding pairs with more than 15 positive samples were considered, resulting in 24 final targets (Badash et al., 3 Sep 2025).

This setup places HSMP-BERT at the intersection of clinical NLP, low-resource-language modeling, and structured information extraction. A plausible implication is that the model is designed not merely to classify whole reports, but to support downstream cohort analysis at the level of clinically meaningful organ–pathology combinations.

2. Backbone, section-aware pretraining, and prompt formulation

The reported base model is Hebrew RoBERTa (HeRo), which was further domain-adaptively pretrained on the radiology corpus. HSMP-BERT’s section-aware component is Section Matching Prediction (SMP), introduced to teach the model the logical structure connecting the “Findings” and “Impression” sections in radiology reports (Badash et al., 3 Sep 2025).

The SMP objective uses paired Findings and Impression sections and predicts whether they match. The prediction head is defined as

$$q_\mathcal{M}(n_k \mid x_i) = \mathrm{softmax}(W_{smp} h_{[CLS]} + b_{smp}),$$

where $n_k \in \{\text{Match}, \text{NotMatch}\}$ and $h_{[CLS]}$ is the pooled representation. The reported loss is binary cross-entropy (Badash et al., 3 Sep 2025).

Prompt-based extraction is then built on top of this section-aware representation. Each structured label is verbalized as a clinical-style natural language prompt. The prompt template is

$$\mathcal{T}(x_i) = [CLS]~x^F_i~[SEP]~\textrm{``There is \{finding\} in the \{organ\}.''}~[EOS].$$

For each report $x^F_i$ and label $y_j$, the model scores compatibility between report and prompt using prompt-based inference, without requiring explicit output layers for each class. During SMP-tuning, pairs of $(x^F_i,\text{prompt})$ are labeled 1 when correct and 0 otherwise, again optimizing a binary cross-entropy loss (Badash et al., 3 Sep 2025).

A key technical feature is that report structure is not treated as incidental metadata. Instead, the alignment between Findings and Impression is used as a pretraining signal, and the organ–finding schema is expressed through prompts rather than through a conventional fixed-output multi-label head.

3. Hierarchical inference and structured prediction

HSMP-BERT’s hierarchy appears in inference as a tree-structured query-routing procedure. The reported sequence is scan-level, then organ-level, then finding-level: first, “Is the scan normal?”; second, “Is there any abnormality in the ileum?”; third, “Is there wall thickening in the ileum?” (Badash et al., 3 Sep 2025).

The routing mechanism is conditional. If a branch is negative, its child prompts are skipped. This tree-based querying reduces unnecessary model evaluations and is the basis of the method’s efficiency claims. The paper reports that hierarchical inference cuts runtime by $5.1\times$ versus traditional inference, with average per-report time reduced from 1.77 seconds under flat inference to 0.35 seconds under hierarchical inference. Total runtime is reported as 4:46:00 for flat inference versus 0:55:36 for hierarchical inference; BERT calls are reduced from 2,180,436 to 395,750; and processed tokens are reduced from 574 M to 114 M (Badash et al., 3 Sep 2025).

This design distinguishes HSMP-BERT from flat prompt evaluation over the full Cartesian product of labels. A common misconception is to equate any “hierarchical” BERT model with hierarchy in the encoder itself. In HSMP-BERT, the reported hierarchy is operationalized through tree-based querying over clinically ordered decisions, rather than through hyperbolic geometry or layer-wise classifier attachment.

4. Evaluation protocol and reported performance

The reported split is multilabel-stratified, with 66% train+validation and 33% test, preserving label prevalence. Performance was evaluated using accuracy, F1, Cohen’s $\kappa$, AUC, PPV, NPV, and recall; the detailed evaluation framework also lists balanced accuracy and precision (Badash et al., 3 Sep 2025).

On the 24 organ–finding combinations with more than 15 positives, HSMP-BERT achieved mean F1 $0.83 \pm 0.08$ and Cohen’s $n_k \in \{\text{Match}, \text{NotMatch}\}$0 $n_k \in \{\text{Match}, \text{NotMatch}\}$1. The detailed results also report accuracy $n_k \in \{\text{Match}, \text{NotMatch}\}$2, PPV $n_k \in \{\text{Match}, \text{NotMatch}\}$3, NPV $n_k \in \{\text{Match}, \text{NotMatch}\}$4, and balanced accuracy $n_k \in \{\text{Match}, \text{NotMatch}\}$5. The paper states that performance was consistent across both common findings, such as ileum wall thickening, and rare findings, such as sigmoid comb sign (Badash et al., 3 Sep 2025).

The principal baselines were SMP zero-shot and standard fine-tuning. In the abstract, HSMP-BERT is reported to outperform the SMP zero-shot baseline, which obtained F1 $n_k \in \{\text{Match}, \text{NotMatch}\}$6 and $n_k \in \{\text{Match}, \text{NotMatch}\}$7 $n_k \in \{\text{Match}, \text{NotMatch}\}$8, and standard fine-tuning, which obtained F1 $n_k \in \{\text{Match}, \text{NotMatch}\}$9 and $h_{[CLS]}$0 $h_{[CLS]}$1, with paired $h_{[CLS]}$2-test $h_{[CLS]}$3 (Badash et al., 3 Sep 2025). The detailed comparison further reports $h_{[CLS]}$4F1 $h_{[CLS]}$5 and $h_{[CLS]}$6 versus SMP zero-shot, with $h_{[CLS]}$7, and $h_{[CLS]}$8F1 $h_{[CLS]}$9 and $$\mathcal{T}(x_i) = [CLS]~x^F_i~[SEP]~\textrm{``There is \{finding\} in the \{organ\}.''}~[EOS].$$0 versus standard fine-tuning, with $$\mathcal{T}(x_i) = [CLS]~x^F_i~[SEP]~\textrm{``There is \{finding\} in the \{organ\}.''}~[EOS].$$1 (Badash et al., 3 Sep 2025).

The paper also provides single-label examples. Ileum wall thickness reached F1 $$\mathcal{T}(x_i) = [CLS]~x^F_i~[SEP]~\textrm{``There is \{finding\} in the \{organ\}.''}~[EOS].$$2 and $$\mathcal{T}(x_i) = [CLS]~x^F_i~[SEP]~\textrm{``There is \{finding\} in the \{organ\}.''}~[EOS].$$3, while rectum wall thickness, described as rare with 6% prevalence, reached F1 $$\mathcal{T}(x_i) = [CLS]~x^F_i~[SEP]~\textrm{``There is \{finding\} in the \{organ\}.''}~[EOS].$$4, $$\mathcal{T}(x_i) = [CLS]~x^F_i~[SEP]~\textrm{``There is \{finding\} in the \{organ\}.''}~[EOS].$$5, and NPV $$\mathcal{T}(x_i) = [CLS]~x^F_i~[SEP]~\textrm{``There is \{finding\} in the \{organ\}.''}~[EOS].$$6 (Badash et al., 3 Sep 2025).

5. Population-level application in Crohn’s disease imaging

After evaluation on expert annotations, HSMP-BERT was applied to the full corpus of 9,683 reports from 7,389 patients, with stratification by sex and age. The reported large-scale output revealed associations among ileal wall thickening, stenosis, and pre-stenotic dilatation, as well as age- and sex-specific trends in inflammatory findings (Badash et al., 3 Sep 2025).

The ileum was reported as the most affected organ, more in males than females and more in adults than pediatric cases. The detailed prevalence figures are 57.7% in males versus 44.8% in females, and 52.6% in adults versus 45.1% in pediatric cases. Wall thickening and wall enhancement were the most common pathologies. Inflammation was higher in males than females, with reported frequencies of 48.4% versus 38.2%, and was also higher in adults. Stenosis and pre-stenotic dilatation were more frequent in adults, whereas DWI abnormalities were more frequent in pediatric cases (Badash et al., 3 Sep 2025).

The paper states that the model output on the full corpus closely mirrors statistics in the expert-labeled set, supporting validity and deployment suitability. In this sense, HSMP-BERT is presented not only as a per-report extraction model but also as a mechanism for population-level analysis of Crohn’s disease imaging patterns (Badash et al., 3 Sep 2025).

6. Relation to other hierarchical BERT formulations

HSMP-BERT belongs to a broader family of work that introduces hierarchy into BERT-like systems, but adjacent studies place the hierarchy at different levels of the modeling stack. In “Layer-wise Guided Training for BERT: Learning Incrementally Refined Document Representations,” different BERT layers are guided to predict labels at different depths of a label hierarchy for large-scale multilabel text classification, with the reported “last-six” configuration assigning classifiers $$\mathcal{T}(x_i) = [CLS]~x^F_i~[SEP]~\textrm{``There is \{finding\} in the \{organ\}.''}~[EOS].$$7 to $$\mathcal{T}(x_i) = [CLS]~x^F_i~[SEP]~\textrm{``There is \{finding\} in the \{organ\}.''}~[EOS].$$8 to hierarchy levels $$\mathcal{T}(x_i) = [CLS]~x^F_i~[SEP]~\textrm{``There is \{finding\} in the \{organ\}.''}~[EOS].$$9 to $x^F_i$0 and improving both classification results and parameter utilization (Manginas et al., 2020).

In “Hierarchical Multitask Learning Approach for BERT,” hierarchy is introduced during pre-training by solving tasks at different transformer layers instead of only at the last layer, transferring information from NSP to masked LM, and adding the bigram shift task to encode word order information. The reported conclusion is that imposing a task hierarchy in pre-training improves the performance of embeddings (Aksoy et al., 2020).

In “A Sentence-level Hierarchical BERT Model for Document Classification with Limited Labelled Data,” hierarchy is document-internal: a token-level RoBERTa encoder produces sentence vectors, and a sentence-level BERT encoder models document structure. That model is targeted at long-document classification with limited labeled data rather than structured clinical extraction (Lu et al., 2021). In “LLMs as Hierarchy Encoders,” hierarchy is made explicit in the embedding geometry by placing encoder outputs inside a Poincaré ball and optimizing hyperbolic clustering and centripetal losses; that work focuses on subsumption and transitive inference rather than report-to-label extraction (He et al., 2024).

These comparisons clarify the specific meaning of hierarchy in HSMP-BERT. Its reported contribution is not a generic hierarchy-aware encoder, but a combination of section-aware pretraining, prompt-based label verbalization, and scan-to-organ-to-finding inference routing for structured extraction from Hebrew radiology text (Badash et al., 3 Sep 2025).