SigBERT: Temporal Survival Analysis in Oncology

• SigBERT is a framework that converts sequential, timestamped narrative oncology reports into fixed, time-aware covariates using rough path signatures.
• It employs dimensionality reduction via PCA and the SIF method to capture evolving clinical information, enhancing risk prediction over traditional static models.
• A LASSO-penalized Cox model on signature features achieves notable predictive performance (C-index 0.75), offering interpretable risk stratification in oncology.

Searching arXiv for the target paper and closely related survival-analysis work to ground the article. {"query":"(Minchella et al., 25 Jul 2025) SigBERT survival oncology rough path signature", "max_results": 5} I’ll look up the target paper directly by arXiv identifier and then a couple of related survival/NLP terms for context. SigBERT is a temporal survival-analysis framework for oncology that converts a patient’s entire history of timestamped narrative medical reports into a single, fixed set of time-aware covariates, then uses those covariates in a sparse Cox model for patient-specific risk estimation. In the formulation reported under the name “SigBERT: Combining Narrative Medical Reports and Rough Path Signature Theory for Survival Risk Estimation in Oncology,” the framework combines a domain-specific BERT-like LLM, rough path signatures, and a LASSO-penalized Cox proportional hazards model; it was trained and evaluated on a real-world oncology dataset from the Léon Bérard Center corpus and achieved a C-index score of 0.75 (sd 0.014) on the independent test cohort (Minchella et al., 25 Jul 2025).

1. Clinical motivation and conceptual formulation

SigBERT is designed for survival prediction from sequential, narrative oncology reports. The motivating problem is that classical survival-analysis methods, including Cox proportional hazards models, usually use static covariates at baseline, whereas oncology care generates long, irregular, and semantically rich trajectories consisting of repeated consultations, admissions, and evolving disease states. In this setting, much prognostic information is stored as free-text reports in the EHR rather than as structured variables. SigBERT addresses how to exploit sequential, narrative oncology reports for survival prediction, how to handle many reports per patient without exploding model complexity, and how to capture temporal dynamics of the clinical narrative rather than a bag-of-words summary over all notes (Minchella et al., 25 Jul 2025).

The framework’s central conceptual move is to treat the sequence of report embeddings as a path and then summarize that path with rough path signatures before fitting a survival model. This differs from traditional Cox, RSF, or DeepSurv pipelines on structured data, which typically use tabular covariates at a fixed time point; from text-based survival models such as BERTSurv, which largely treat each note independently or pool them in simpler ways; and from sequential models such as Dynamic-DeepHit, Survival Seq2Seq, and DySurv, which model structured time series with recurrent or seq2seq architectures. SigBERT instead uses a non-neural temporal representation—the signature transform—which yields a finite-dimensional, rotation-invariant feature map that can be plugged directly into a Cox model with LASSO.

A clinically important aspect of the problem formulation is that risk depends not only on what is stated in the reports but on how the narrative evolves over time. The paper’s illustrative contrast—“stable disease” → “progression” versus “progression” → “partial response”—captures the intended role of temporal order. A plausible implication is that SigBERT is best understood not as a document classifier, but as a trajectory encoder for longitudinal clinical text.

2. Representation pipeline from reports to patient-level features

The reported architecture follows a five-step pipeline: raw reports to word embeddings with OncoBERT, word embeddings to sentence or report embeddings with SIF, dimensionality reduction with PCA, time series of embeddings to signature features, and signature covariates to a LASSO-Cox survival model (Minchella et al., 25 Jul 2025).

For each patient, the corpus consists of French oncology reports, including consultations and hospital summaries, each associated with a timestamp. Preprocessing removes redundant metadata such as report source prefixes, drops duplicates, and standardizes date fields into a consistent datetime format. No truncation of reports is performed at the preprocessing stage. Because OncoBERT has a 512-token limitation, longer texts are handled at embedding time by chunking and aggregation. Each patient is associated with a survival time in study $T_i$ and an event indicator $\delta_i$, and the framework masks the last $\geq 100$ days of each patient’s record when building features so that prediction does not rely on immediate pre-death context.

The LLM component is OncoBERT, a CamemBERT model fine-tuned on oncology clinical notes from the Léon Bérard Center. Its embedding dimension is $p = 768$. For each token $w$, OncoBERT produces an embedding $v_w \in \mathbb{R}^p$. Rather than using the $[\mathrm{CLS}]$ token, SigBERT forms report embeddings with Smooth Inverse Frequency (SIF):

$$v_s = \frac{1}{|s|}\sum_{w \in s}\frac{a}{f(w)+a}\,v_w \in \mathbb{R}^p,$$

where $f(w)$ is the empirical frequency of word $w$ in the corpus and $\delta_i$0. The stated rationale is that SIF down-weights very frequent words and provides a robust, unsupervised sentence representation. Empirically, CLS-based document embeddings gave C-index $\delta_i$1, whereas SIF embeddings improved this to C-index $\delta_i$2, with better td-AUC and lower Brier scores. SIF also permits aggregation across all chunks of reports longer than 512 tokens.

For patient $\delta_i$3 with $\delta_i$4 reports, stacking report embeddings yields a matrix in $\delta_i$5. Because direct signature extraction in 768 dimensions is combinatorially expensive, SigBERT applies PCA over all sentence embeddings in the training set, using a compression matrix $\delta_i$6 and compressed embeddings

$\delta_i$7

The paper reports a sweep over $\delta_i$8 and states that performance stabilizes around $\delta_i$9, which is selected as a trade-off between computational cost and predictive ability. OncoBERT itself is not further fine-tuned on the survival task; it is used as a frozen feature extractor.

3. Rough path signatures as temporal encoders of clinical text

After PCA compression, each patient is represented by a time-ordered sequence of report embeddings in $\geq 100$0. SigBERT interprets this sequence as a path $\geq 100$1 and computes its truncated rough path signature. For a multi-index $\geq 100$2, the $\geq 100$3th-order signature coordinate up to time $\geq 100$4 is defined as

$\geq 100$5

Collecting all such iterated integrals of order $\geq 100$6 yields

$\geq 100$7

and the truncated signature of order $\geq 100$8 is

$\geq 100$9

The $p = 768$0th component $p = 768$1 is conventionally $p = 768$2 (Minchella et al., 25 Jul 2025).

The paper emphasizes several properties of signatures that are relevant for longitudinal clinical text. Under mild conditions, the full signature uniquely characterizes the path up to reparameterization. Signatures are invariant to time reparameterization and translation, which is useful when reports are irregularly sampled and follow-up lengths vary. They also map variable-length sequences into fixed-dimensional vectors whose dimension depends on $p = 768$3 and $p = 768$4, not on the number of reports $p = 768$5.

Computational tractability is a defining concern. The number of signature coefficients grows as

$p = 768$6

At $p = 768$7 and $p = 768$8, the paper gives approximately $p = 768$9 coefficients, which is intractable. This is the main mathematical reason for dimensionality reduction before signature extraction and for keeping the truncation level low. The chosen signature order $w$0 is not explicitly stated in the visible text, but the paper repeatedly stresses the need to keep it small.

For uniqueness, the framework uses time augmentation: time is appended as an extra monotone coordinate to the embedding path. Signature computation is implemented with the iisignature library. Chen’s identity,

$w$1

is highlighted as computationally important because it permits incremental computation over concatenated path segments. The paper does not center lead–lag transforms or log-signatures; it works with standard signatures and time augmentation.

4. Survival modeling, optimization, and evaluation protocol

Once the signature vector $w$2 has been computed for patient $w$3, SigBERT treats it as an ordinary covariate vector in a Cox proportional hazards model:

$w$4

where $w$5 is the baseline hazard and $w$6 is the coefficient vector. The resulting patient-specific risk score is

$w$7

The corresponding survival function is written as

$w$8

with $w$9 the baseline cumulative hazard (Minchella et al., 25 Jul 2025).

Estimation proceeds through the Cox partial likelihood. With observed time $v_w \in \mathbb{R}^p$0, event indicator $v_w \in \mathbb{R}^p$1, and risk set $v_w \in \mathbb{R}^p$2,

$v_w \in \mathbb{R}^p$3

To cope with high-dimensional signature features, the model uses an $v_w \in \mathbb{R}^p$4 penalty:

$v_w \in \mathbb{R}^p$5

This yields a sparse coefficient vector and therefore performs embedded feature selection over signature coordinates. For a new patient, the final predicted risk score is $v_w \in \mathbb{R}^p$6.

Censoring is handled in the standard Cox manner through risk sets in the partial likelihood. Performance is evaluated with td-AUC and Brier scores integrated over time windows such as 3, 5, and 10 years, and the Integrated Brier Score is computed up to 3, 5, and 10 years. A Kaplan–Meier estimator $v_w \in \mathbb{R}^p$7 for the censoring distribution is used to weight td-AUC and Brier scores appropriately.

The implementation stack reported in the paper consists of skglm for Cox LASSO, lifelines for baseline hazard and survival-function estimation, and sksurv for evaluation metrics. The LASSO penalty $v_w \in \mathbb{R}^p$8 is tuned by grid search in $v_w \in \mathbb{R}^p$9 with step $[\mathrm{CLS}]$0, using 5-fold cross-validation on the training set to maximize cross-validated C-index. Once embeddings are precomputed, signature extraction and Cox training are said to take only minutes.

5. Léon Bérard Center corpus, baselines, and empirical performance

SigBERT was evaluated on a large oncology corpus from the Léon Bérard Center, a comprehensive cancer center in Lyon, France. The reported study period covers diagnosis dates roughly between 1997 and 2020. The cohort contains 7,121 patients, of whom 4,983 are deceased and 2,138 are censored, with 274,420 medical reports overall. The mean number of reports per patient is approximately 39 (sd 25). Report types are approximately 63% consultation reports and approximately 32% hospital stay reports, with the remainder in other categories. The most prevalent cancer groups are breast cancer (25%), gynecological (9.7%), gastrointestinal (8.6%), lung (5.5%), prostate (7%), and endocrine tumors (4.6%). The median survival time is 1,024 days, approximately 2.8 years. Inclusion is described as patients hospitalized at least once at the center between 2000–2024, with full follow-up; the data were anonymized and approved under GDPR and MR004 (Minchella et al., 25 Jul 2025).

The data split consists of a training cohort of 3,560 patients with 136,748 reports and a test cohort of 3,561 patients with 137,672 reports. The split is described as structured and stratified to preserve temporal and distributional consistency, with 5-fold cross-validation within the training cohort for hyperparameter tuning.

The empirical comparison is against several simpler text and temporal baselines on the same data. Using only the most recent report per patient and feeding its embedding to a Cox model yields C-index $[\mathrm{CLS}]$1. Averaging all report embeddings per patient before Cox yields C-index $[\mathrm{CLS}]$2. A neural Cox-Time model yields C-index $[\mathrm{CLS}]$3. SigBERT reaches C-index $[\mathrm{CLS}]$4, which the authors interpret as evidence that naive aggregation or simple time encoding is insufficient to capture the temporal information in the notes.

Metric	Reported value
C-index	$[\mathrm{CLS}]$5
95% CI (Jackknife)	$[\mathrm{CLS}]$6
Mean td-AUC over 10 years	$[\mathrm{CLS}]$7
IBS up to 3 years	$[\mathrm{CLS}]$8 (sd $[\mathrm{CLS}]$9)
IBS up to 5 years	$$v_s = \frac{1}{|s|}\sum_{w \in s}\frac{a}{f(w)+a}\,v_w \in \mathbb{R}^p,$$0 (sd $$v_s = \frac{1}{|s|}\sum_{w \in s}\frac{a}{f(w)+a}\,v_w \in \mathbb{R}^p,$$1)
IBS up to 10 years	$$v_s = \frac{1}{|s|}\sum_{w \in s}\frac{a}{f(w)+a}\,v_w \in \mathbb{R}^p,$$2 (sd $$v_s = \frac{1}{|s|}\sum_{w \in s}\frac{a}{f(w)+a}\,v_w \in \mathbb{R}^p,$$3)

The paper notes that a naive Brier-score baseline corresponding to constant survival probability 0.5 yields BS $$v_s = \frac{1}{|s|}\sum_{w \in s}\frac{a}{f(w)+a}\,v_w \in \mathbb{R}^p,$$4, so IBS values below $$v_s = \frac{1}{|s|}\sum_{w \in s}\frac{a}{f(w)+a}\,v_w \in \mathbb{R}^p,$$5 indicate good calibration; the reported IBS remains below $$v_s = \frac{1}{|s|}\sum_{w \in s}\frac{a}{f(w)+a}\,v_w \in \mathbb{R}^p,$$6 up to 10 years. Risk-time association is quantified on uncensored patients by the correlation between $$v_s = \frac{1}{|s|}\sum_{w \in s}\frac{a}{f(w)+a}\,v_w \in \mathbb{R}^p,$$7 and the risk score $$v_s = \frac{1}{|s|}\sum_{w \in s}\frac{a}{f(w)+a}\,v_w \in \mathbb{R}^p,$$8: Pearson $$v_s = \frac{1}{|s|}\sum_{w \in s}\frac{a}{f(w)+a}\,v_w \in \mathbb{R}^p,$$9 (sd $f(w)$0) and Spearman $f(w)$1 (sd $f(w)$2), both with $f(w)$3.

The study also reports how performance changes as more reports per patient are revealed. With only 2 reports, the C-index is approximately $f(w)$4; at 28 reports, approximately $f(w)$5; beyond approximately 100 reports, performance converges around $f(w)$6. This suggests that the framework can exploit longer longitudinal histories rather than merely tolerate them.

6. Interpretability, limitations, and place in the survival-analysis literature

SigBERT’s interpretability derives primarily from its linear Cox layer over signature features and the sparsity induced by LASSO. Because many coefficients are set to zero, the remaining nonzero coefficients can be read as increasing or decreasing hazard in relation to specific multi-indices of embedding and time coordinates. The paper also reports risk stratification by quartiles of the predicted risk score: boxplots of log survival time show a clear monotone trend in which higher risk quartiles correspond to shorter survival, and both ANOVA and Kruskal–Wallis tests give $f(w)$7 (Minchella et al., 25 Jul 2025).

At the same time, the article emphasizes a major limitation of interpretation. Signature coordinates are high-order interactions of PCA components of report embeddings, and those PCA components are themselves compressed semantic dimensions derived from OncoBERT. As a result, mapping a selected coefficient back to readable clinical phrases or token-level rationales is non-trivial. The framework therefore offers feature-level interpretability, not phrase-level explanation, and it does not provide attention maps or token saliency.

The limitations acknowledged in the paper are structural rather than incidental. First, tractable signature computation requires PCA compression from 768 to 25 dimensions and a low signature order, which may discard information even though empirical performance stabilizes by 25 dimensions. Second, the LLM is fine-tuned on Léon Bérard oncology notes and may not generalize to other institutions, languages, or specialties. Third, the current system is single-modality: it uses only narrative reports and does not integrate structured covariates such as tumor stage, biomarkers, or demographics. Fourth, evaluation is confined to a single center, so external generalizability remains untested.

Within the broader literature, SigBERT is positioned at the intersection of clinical NLP for prognosis, signature-based time-series modeling, and dynamic survival analysis. The paper presents it as extending the signature-based logic of CoxSig from structured time series to unstructured text trajectories, while differing from BERTSurv and from recurrent or seq2seq survival architectures by using a fixed, mathematically grounded temporal encoding rather than a learned sequential network. A plausible implication is that SigBERT is most distinctive not because it uses BERT, but because it makes rough path signatures the central interface between longitudinal clinical text and survival modeling. Future directions proposed in the paper include multimodal integration with structured EHR variables, benchmarking alternative survival models on the same signature features, evaluating other pretrained LLMs and end-to-end fine-tuning strategies, testing external generalization across institutions and languages, and analyzing which embedding dimensions and interaction orders carry prognostic information.