Contextual Filtering in Clinical Summarization

• The paper introduces an attention-guided token filtering mechanism that compresses clinical notes while retaining key clinical context.
• It combines Context-Preserving Token Filtering with a domain-specific knowledge graph to augment summaries with structured patient information.
• Empirical results show improved BLEU, ROUGE-L, and BERT-F1 scores over baselines, demonstrating enhanced coherence and clinical fidelity.

Searching arXiv for the target paper and closely related clinical summarization context.

ConTextual is a clinical text summarization framework for LLMs that combines Context-Preserving Token Filtering (CPTF) with a Domain-Specific Knowledge Graph (KG) to compress long clinical notes while retaining clinically important context and then augment the reduced text with structured knowledge before generation. It is designed for settings in which unstructured clinical notes are long, noisy, and decision-relevant, and it addresses two limitations identified in prior work: processing all input tokens uniformly, and relying on heuristic filters that can miss nuanced clinical cues. In the reported experiments on two public benchmark datasets, ConTextual improves both linguistic coherence and clinical fidelity, and is presented as a scalable solution for long-note summarization (Piya et al., 23 Apr 2025).

1. Problem formulation and architectural overview

ConTextual treats clinical summarization as a two-stage transformation of a note $d = \{t_1,\dots,t_n\}$: first, it reduces the note to a shorter sequence $f(d)$ of length $\lfloor r \cdot |d| \rfloor$, where $r \in (0,1]$ is a retention ratio; second, it summarizes that reduced representation after augmenting it with patient-specific graph context (Piya et al., 23 Apr 2025). The motivating objective is to preserve the clinically important context of long notes while making them tractable for downstream LLM summarization.

The framework has three main components. The first is CPTF, which performs contextual token filtering over the original note. The second is a Domain-Specific Knowledge Graph, whose nodes encode diagnoses, medications, and treatments together with their relations. The third is a RAG-style summarization stage in which an LLM conditions on the reduced note plus KG context to generate the final summary. The reported average input note length is approximately 2,267 tokens, which makes the preprocessing role of CPTF central to the overall design (Piya et al., 23 Apr 2025).

This design is significant because it separates compression from generation. Rather than asking the summarizer to discover relevance entirely inside a long-context forward pass, ConTextual first constructs a compressed, ordered, context-preserving surrogate of the note and then supplements it with structured retrieval. This suggests a division of labor in which token-level saliency and graph-based factual augmentation play complementary roles.

2. Context-Preserving Token Filtering

CPTF operationalizes the reduction function $f$ by computing token importance from the internal multi-layer, multi-head attention patterns of an LLM. For each layer $l \in \{1,\dots,L\}$ and head $h \in \{1,\dots,H\}$, the model exposes an attention matrix $A_l^h \in \mathbb{R}^{n \times n}$. These are averaged across heads to obtain a per-layer attention matrix

$\bar{A}_l = \frac{1}{H}\sum_{h=1}^{H} A_l^h.$

ConTextual then assigns each layer a weight

$$w_l = \alpha + (1-\alpha)\frac{l}{L}, \qquad \alpha \in [0,1],$$

so that deeper layers receive larger weight while lower layers still contribute (Piya et al., 23 Apr 2025).

The token importance score at position $f(d)$0 is

$f(d)$1

This is a purely attention-based saliency measure: no gradients and no extra loss functions are used. The method is post-hoc and deterministic given the model’s attention maps. The interpretation supplied in the paper is that tokens with consistently strong, layer-weighted interactions across the sequence act as context-carriers or context-integrators.

Selection is then reduced to top-$f(d)$2 optimization. With $f(d)$3, CPTF selects

$f(d)$4

which in practice means taking the top-$f(d)$5 tokens by $f(d)$6 and restoring their original order. The reduced narrative is reconstructed as

$f(d)$7

The ordering constraint is not incidental: it is the mechanism by which CPTF preserves local narrative structure rather than producing a bag of medically salient words (Piya et al., 23 Apr 2025).

The paper attributes context preservation to three properties. First, token importance is aggregated globally across heads and layers, so selection is not based on frequency or stopword-like heuristics. Second, reconstruction preserves the original order and positional information. Third, the layer-weighting scheme blends lower-level structural cues with higher-level semantic cues. An example workflow in the paper shows that CPTF retains section markers such as SERVICE and CHIEF COMPLAINT, core history such as “ruptured AVM, craniotomy,” diagnosis terms such as “Klebsiella UTI,” and key labs and treatments, while removing some adjectives, modifier phrases, and repeated or less critical details. The paper describes this as preserving the “semantic skeleton” of the note.

The hyperparameter $f(d)$8 governs the balance between shallow and deep layers. The authors sweep $f(d)$9 and report that $\lfloor r \cdot |d| \rfloor$0 gives the best BLEU and ROUGE, which they interpret as balancing surface fluency and deep contextual representation (Piya et al., 23 Apr 2025). This suggests that the filtering quality depends materially on how saliency is distributed across the transformer depth.

3. Knowledge-graph augmentation and summary generation

After filtering, ConTextual retrieves patient-specific structured context from a Domain-Specific KG defined as

$\lfloor r \cdot |d| \rfloor$1

with nodes for diagnoses $\lfloor r \cdot |d| \rfloor$2, medications $\lfloor r \cdot |d| \rfloor$3, treatments $\lfloor r \cdot |d| \rfloor$4, and typed relations such as $\lfloor r \cdot |d| \rfloor$5, $\lfloor r \cdot |d| \rfloor$6, and $\lfloor r \cdot |d| \rfloor$7. For a note $\lfloor r \cdot |d| \rfloor$8, a retrieval function

$\lfloor r \cdot |d| \rfloor$9

maps the note to patient-specific entities. Contextual neighbors are then retrieved as

$r \in (0,1]$0

This retrieved graph context is concatenated with the reduced note (Piya et al., 23 Apr 2025).

The enriched representation is written as

$r \in (0,1]$1

and the summarizer conditions on

$r \in (0,1]$2

Generation is expressed as

$r \in (0,1]$3

where $r \in (0,1]$4 denotes the LLaMA parameters. In the reported system, CPTF therefore acts as a preprocessing and compression layer, while KG augmentation acts as a structured retrieval layer that can compensate for information lost during filtering (Piya et al., 23 Apr 2025).

The paper’s interpretation of the full pipeline is that token-level filtering and structured retrieval are complementary rather than interchangeable. CPTF reduces redundancy and focuses the model on context-critical tokens; KG augmentation re-injects relations among clinically relevant entities. This division explains why CPTF alone improves over plain LLaMA, but the full ConTextual configuration yields the strongest ROUGE-L and BERT-F1.

4. Empirical performance and ablation evidence

The main ablation compares three systems: plain LLaMA 3.2, LLaMA 3.2 + CPTF, and ConTextual (LLaMA + CPTF + KG). For typical settings such as max tokens 300 and $r \in (0,1]$5, the paper reports the following approximate values for plain LLaMA: BLEU-1 $r \in (0,1]$6, ROUGE-L $r \in (0,1]$7, and BERT-F1 $r \in (0,1]$8. Adding CPTF improves these values to BLEU-1 $r \in (0,1]$9–$f$0, ROUGE-L $f$1–$f$2, and BERT-F1 $f$3. The best ConTextual configuration reports BLEU-1 $f$4, BLEU-2 $f$5, ROUGE-L $f$6, and BERT-F1 $f$7 (Piya et al., 23 Apr 2025).

These numbers support two distinct conclusions. First, CPTF alone improves over the no-filtering LLaMA baseline, indicating that contextual filtering is itself useful. Second, KG augmentation on top of CPTF raises ROUGE-L and BERT-F1 further, indicating better semantic alignment and sequence-level cohesion. The paper therefore attributes the best overall behavior to the combination rather than to filtering or retrieval in isolation.

Against external baselines, ConTextual is reported to outperform Longformer, BioBART, Flan-T5, and Gemma3-Instruct (1B) on BLEU-1, ROUGE-L, and BERT-F1. The reported values are Longformer: BLEU-1 $f$8, ROUGE-L $f$9, BERT-F1 $l \in \{1,\dots,L\}$0; BioBART: $l \in \{1,\dots,L\}$1, $l \in \{1,\dots,L\}$2, $l \in \{1,\dots,L\}$3; Flan-T5: $l \in \{1,\dots,L\}$4, $l \in \{1,\dots,L\}$5, $l \in \{1,\dots,L\}$6; Gemma3-Instruct (1B): $l \in \{1,\dots,L\}$7, $l \in \{1,\dots,L\}$8, $l \in \{1,\dots,L\}$9; and ConTextual: $h \in \{1,\dots,H\}$0, $h \in \{1,\dots,H\}$1, $h \in \{1,\dots,H\}$2 (Piya et al., 23 Apr 2025). The paper interprets the ROUGE-L margin as evidence of better sequence-level coherence and the BERT-F1 margin as evidence of stronger semantic fidelity.

The experiments also show sensitivity to retained length. The paper reports token limits of 100, 200, and 300, with performance generally increasing as more tokens are retained, at the cost of higher latency. This indicates that CPTF is not a fixed compression heuristic but a tunable trade-off between compression strength and summary quality.

5. Efficiency profile, benefits, and limitations

The reported efficiency profile distinguishes between throughput, latency, and effective sequence length. Plain LLaMA has very high throughput, reported as more than 2000 tokens/s, and low latency of approximately 2.7–5 s while processing the full note. LLaMA + CPTF has throughput around 200–500 tokens/s and latency around 6–17 s depending on token limit. ConTextual, which adds KG augmentation, has throughput around 90–220 tokens/s and latency around 6–18 s (Piya et al., 23 Apr 2025).

Although this introduces overhead, the paper argues that efficiency should be evaluated at the level of long documents. Because CPTF reduces the sequence length seen by the summarizer, it lowers the quadratic attention cost and makes long notes tractable on smaller models such as LLaMA 1B. This suggests that the relevant efficiency comparison is not raw decoder speed, but the cost of obtaining clinically faithful summaries from long clinical notes.

The benefits identified in the paper fall into four categories. Clinical fidelity and factuality are reflected in higher BERT-F1 and ROUGE-L. Linguistic coherence is reflected in ROUGE-L improvements and in the preserved note ordering. Precision and recall of clinical information are reflected in BLEU gains together with KG-based reinjection of relations. Computational efficiency comes from the tunable retention ratio $h \in \{1,\dots,H\}$3, which reduces the number of tokens sent to the summarization model (Piya et al., 23 Apr 2025).

The limitations are equally explicit. The method depends on a chosen retention ratio $h \in \{1,\dots,H\}$4; very low $h \in \{1,\dots,H\}$5 may discard subtle but clinically important details. Performance is sensitive to the layer-weight parameter $h \in \{1,\dots,H\}$6; extreme values such as 0.2 or 0.9 degrade BLEU and ROUGE. CPTF requires an additional forward pass for attention extraction, adding overhead relative to naive tokenization. The method also has no explicit training objective for filtering: it is described as a heuristic but principled use of attention statistics. Finally, it depends on the assumption that attention weights reflect salience; the paper notes that if attention patterns are misaligned with true token importance, important tokens may be underweighted (Piya et al., 23 Apr 2025).

6. Conceptual position and broader applicability

ConTextual is explicitly positioned against two alternative strategies: heuristic filtering and no filtering. Heuristic token filtering, such as stopword removal or frequency-based selection, lacks a model-based notion of contextual importance and can be unsafe in clinical text because tokens relevant to negation, temporality, dosage, or common clinical patterns may be removed despite being semantically critical. No filtering preserves all information in principle, but for long notes it imposes high computational cost and can dilute attention across redundant or low-value content (Piya et al., 23 Apr 2025).

The paper’s proposed alternative is attention-guided compression before generation, with subsequent structured retrieval. The authors emphasize that ConTextual is tested on MIMIC-IV discharge notes and BHC summaries, but also state that the modular design should, in principle, extend to other long, jargon-heavy domains such as legal contracts and scientific articles, with the KG and prompt templates being the domain-specific components (Piya et al., 23 Apr 2025). This suggests that the transferable contribution is not a clinical vocabulary artifact, but the pairing of token-level saliency filtering with structured augmentation.

In adjacent literature, contextual filtering has also been studied as fine-grained sentence selection in retrieval-augmented generation and as embedding-based segment pruning in LLM pipelines. “Learning to Filter Context for Retrieval-Augmented Generation” (Wang et al., 2023) treats filtering as sentence-level context compression between retriever and generator, while “Semantic Decomposition and Selective Context Filtering -- Text Processing Techniques for Context-Aware NLP-Based Systems” (Villardar, 19 Feb 2025) formulates filtering as embedding-based removal of irrelevant context segments. ConTextual differs in operating at the token level before summarization and in pairing that filtering with a domain-specific KG, but it belongs to the same broader family of methods that attempt to improve generation by restricting context to what is task-relevant.

Taken together, ConTextual defines contextual filtering in clinical summarization as an attention-guided, order-preserving, top-$h \in \{1,\dots,H\}$7 token selection mechanism coupled to structured clinical retrieval. Its central claim is not merely that fewer tokens are better, but that the right tokens, selected from multi-layer attention structure and then enriched with graph context, can improve both linguistic and clinical integrity in long-note summarization (Piya et al., 23 Apr 2025).