Learning from Lost Provenance: Multiple Instance Learning for Cancer Registry Tumor Group Classification

Abstract: Modernizing cancer registries with deep learning is opening new opportunities to automate labor-intensive tasks such as the coding of pathology reports. However, progress is constrained by the scarcity of report-level human-annotated training data. Cancer registries generate substantial volumes of expert-assigned labels as a routine product of their operations, but these exist at the patient level and are not linked to the individual pathology reports that informed them, limiting their direct use for training models. We develop an efficient framework for training deep learning classifiers by leveraging these operationally-generated labels without requiring per-report human annotation, demonstrated for tumor group classification at the BC Cancer Registry. We use Attention-Based Multiple Instance Learning (ABMIL) to recover the lost link between patient-level labels and the reports that informed them, leveraging the attention the model places on each report to distil a large, noisily-labeled corpus into a compact, high-quality per-report training dataset. A classifier fine-tuned on a distilled dataset achieved a macro F1 of 0.83, outperforming established baselines across most tumor groups. By turning routine operational labels into high-quality training data without additional annotation or large-scale computing infrastructure, ABMIL offers a practical and accessible route to automating cancer registry workflows.

• The paper introduces an attention-based MIL framework that uses weak supervision to achieve high-precision tumor group classification from cancer registry data.
• It employs a two-stage pipeline combining weakly supervised classification with attention-guided data distillation to optimize performance.
• The method outperforms traditional baselines by boosting macro F1 scores from 0.68 to 0.83 while maintaining resource efficiency.

Attention-Based Multiple Instance Learning for Tumor Group Classification from Cancer Registry Reports

Introduction

Cancer registries underpin cancer surveillance and resource allocation, but the annotation bottleneck inherent in producing report-level, human-verified tumor group labels constrains the deployment of high-fidelity NLP tools for registry automation. The study presents an Attention-Based Multiple Instance Learning (ABMIL) approach for tumor group classification, leveraging only operationally-generated patient-level tumor group assignments without the need for per-report expert annotation. This is a structural match to registry operations, where human coders assign tumor group at the patient level by integrating evidence from multiple free-text pathology reports, but do not record report-to-label provenance. The paper demonstrates a two-stage pipeline: a weakly supervised ABMIL classifier, and an ABMIL-guided data distillation step yielding high-precision report-level labels.

Methodology

The dataset comprises 359,714 pathology reports linked to 82,855 unique patients in the BC Cancer Registry, after filtering to single-primary-tumor cases. Each patient (bag) has a set of pathology reports (instances) and a tumor group label. The ABMIL framework applies attention-based pooling on fixed pretrained BCCRTron embeddings ([CLS], $d=1024$), with attention weights determining the relevance of each report to the bag-level tumor group label.

The pipeline proceeds in two pathways (Figure 1):

Figure 1: Schematic of the ABMIL workflow featuring weak supervision and a dataset distillation pathway leveraging attention for instance selection.

• ABMIL-WS (Weak Supervision): Only the attention and classification head are learned, freezing all encoder parameters. Predictions are made at the bag (patient) level.
• ABMIL-DISTIL: The trained attention module from ABMIL-WS is used to rank reports, enabling percentile-based filtering to select the most relevant reports for downstream end-to-end fine-tuning of the encoder.

Label propagation (assigning each patient’s tumor label to all their reports, regardless of relevance) and concatenation (joining all reports per patient and classifying the resultant long sequence) constitute the two baseline approaches for comparison.

Results

Impact of Attention-Based Filtering

Percentile-based filtering using attention scores dramatically improved macro F1. At 0% threshold (reproducing label propagation, i.e., maximal recall but maximal label noise), macro F1 is lowest. As the threshold increases (e.g., 75%), the distillation step discards non-informative reports, increasing label precision while slightly reducing data volume.

Figure 2: Per-class performance comparison demonstrating optimal F1 at the 75th percentile threshold and diminishing returns at 99% due to over-filtering.

Figure 3: Macro and weighted F1 as a function of the attention filtering threshold, with the optimal value marked; bottom panel shows class supports, highlighting the tradeoff between minority group data volume and filtering stringency.

Comparative Performance

The ABMIL-DISTIL pipeline, trained on distilled data above the 75th percentile, attained macro F1 = 0.83 on the ODS-annotated test set—a substantial gain over the label propagation baseline (macro F1 = 0.68), and outperforming the concatenation baseline (macro F1 = 0.79) across almost all tumor groups. The only group where ABMIL-DISTIL underperformed was Sarcoma, where recall remained high but the class acted as a precision sink due to its diverse, poorly separated subtypes. The ABMIL-WS method (macro F1 = 0.72) outperformed the label propagation approach, despite not requiring GPU resources or fine-tuning of the encoder.

Figure 4: Per-class F1 comparisons across label propagation, ABMIL-DISTIL, concatenation, and ABMIL-WS, with macro-F1 summary and per-class improvements.

Analysis and Discussion

The results substantiate that attention-based instance weighting yields substantive gains in label quality for downstream supervised learning where direct, trusted provenance is absent. The attention filtering mechanism operationalizes dataset distillation, enabling exclusion of non-informative or spurious reports and affording computational tractability for fine-tuning.

The most salient finding is that nearly all tumor groups benefited from ABMIL-guided filtering, with maximum effect sizes in classes typically plagued by weak training signal (e.g., Skin, Melanoma), and a consistent outperformance relative to conventional baselines. This is a direct consequence of instance selection reducing noise from irrelevant or weakly related reports—an effect empirically observed in the declining macro F1 at ultra-stringent thresholds due to overly aggressive support reduction.

The low-resource ABMIL-WS model, which preserves the encoder frozen, provides competitive results, supporting deployment in computationally restricted registry environments. Thus, this approach democratizes NLP model deployment for PBCRs lacking large-scale annotation or GPU infrastructure.

However, applicability is contingent on the single-diagnosis per-patient filtering, limiting generalizability to multi-primary cohorts. Additionally, reliance on the assumption that each bag contains at least one diagnostically relevant report may not always be met, especially when coders integrate evidence from outside the pathology corpus.

Implications and Future Directions

Practically, this ABMIL-guided distillation paradigm enables rapid construction of high-quality per-report datasets from operational registry data programmatically, obviating the need for further expert labeling. Theoretically, the results highlight the critical role of attention in mitigating the “lost provenance” problem prevalent in health informatics and clinical NLP, functioning not only as a model mechanism but also as a data curation filter.

Future developments may target generalization to patients with multiple primaries, incorporation of non-pathology sources in the bag model, or the coupling of attention mechanisms with active or semi-supervised learning. Extension to other registry-coded attributes (primary site, histology, staging) is natural, as these are often linked to multiple, variably informative documents.

Conclusion

The application of ABMIL for weak supervision and report-level distillation enables high-fidelity tumor group classification from registry data lacking explicit instance-label provenance. The approach consistently outperforms strong baselines, is resource-efficient, and is feasible for practical deployment in real-world registrar workflows. As cancer registries expand in both size and complexity, ABMIL-guided models represent a scalable, annotation-efficient strategy to automate and augment coded case abstraction (2607.03481).