Information Bottleneck-based Causal Attention

• The paper introduces IBCA, a framework that combines an information bottleneck, Gaussian mixture spatial attention, and causal intervention to filter out noise and spurious patterns in medical images.
• It employs a Gaussian mixture variational bottleneck to derive label-specific attention maps that isolate causal signals from confounders in complex multi-label tasks.
• Empirical evaluations on colonoscopy and retinal datasets show significant improvements in class-wise recall, overall recall, and mean average precision compared to existing causal methods.

Information Bottleneck-based Causal Attention (IBCA) is a framework for learning class-specific, causally meaningful attention for multi-label medical image recognition. It combines an information bottleneck objective, a Gaussian mixture model for spatial attention, and a causal intervention mechanism implemented via contrastive alignment. In the formulation introduced for “Information Bottleneck-based Causal Attention for Multi-label Medical Image Recognition,” class-specific attention is treated as a mixture of causal, spurious, and noisy factors, with the stated aim of learning discriminative class-specific attention for multi-label classification of medical images such as colonoscopy and retinal fundus images (Cui et al., 11 Aug 2025).

1. Problem setting and motivation

IBCA is defined in the setting of multi-label medical image classification, where each image can contain multiple co-occurring diseases or findings and the model must predict a set of labels $Y \subset \{1,\dots,N_c\}$ for each image $X$ (Cui et al., 11 Aug 2025). The method is motivated by two requirements that are emphasized as crucial in this setting: performance, because the model must disentangle overlapping visual evidence for different diseases, and interpretability, because clinicians want class-specific attention maps that highlight causal lesion regions rather than irrelevant context.

The method is positioned against two limitations of existing approaches. First, recent causal attention and causal debiasing methods can still produce spurious attention, meaning attention may lock onto class-irrelevant patterns that correlate with labels in training data but are not causal, including lighting, anatomical background, or scanner artifacts. Second, even when attention is “causal” in a deconfounding sense, it may include redundant or noisy pixels that are not discriminative for the specific disease. The paper contrasts this with Information Bottleneck-based attention methods, which can compress attention to retain only label-relevant information, but which do not explicitly separate causal and spurious factors and are hampered in multi-label settings by the use of a single spherical Gaussian prior.

IBCA is introduced to close this gap by combining three ideas. It applies a Variational Information Bottleneck to a Gaussian mixture spatial attention so that class-irrelevant and noisy information is filtered out while multiple label-specific attention patterns are captured. It also integrates this bottlenecked attention into a causal intervention mechanism based on multi-head self-attention. Finally, it aligns these two views through a contrastive procedure intended to pull causal attention toward denoised, label-specific Gaussian mixture attention while reducing spurious attention induced by confounders.

2. Structural causal model and causal semantics

The structural causal model underlying IBCA uses the variables $X$ for image features, $A$ for class-specific attention, $Z$ for latent class-specific features, $Y$ for multi-label disease indicators, $C$ for confounders, and $\mathbf{N}$ for noise (Cui et al., 11 Aug 2025). The causal perspective is developed in three stages. In the Information Bottleneck view, the chain is $X \rightarrow A \rightarrow Z \rightarrow Y$, where the goal is to compress $(A,X)$ into $X$0 while preserving information about $X$1. In the causal attention view, a confounder $X$2 is added through the path $X$3, which injects spurious correlations into the desired path $X$4. In the final IB-based causal attention SCM, noise $X$5 affects attention $X$6 but is not causally related to $X$7, and attention is decomposed into a causal component $X$8, a spurious component driven by $X$9, and noise from $X$0.

The intervention target is the causal effect of $X$1 on $X$2, written as $X$3. The paper gives a sampling-based expression

$X$4

where $X$5 is a class-specific attention for label $X$6 in sample $X$7, $X$8 is a class-specific classifier head, and $X$9 is the inverse propensity weighting for attention given confounder $A$0. In practice, because multi-head attention uniformly samples from various contexts, this weighting is approximated by a constant $A$1.

Within this SCM, the paper assigns distinct roles to the two principal components of IBCA. The Information Bottleneck with Gaussian mixture is said to remove noise and enforce compact, label-discriminative attention patterns, while causal intervention and contrastive alignment are used to remove spurious effects due to confounders. This yields the paper’s central interpretation of class-specific attention as a mixture of causal, spurious, and noisy factors rather than as a single undifferentiated attention map.

3. Gaussian mixture variational bottleneck

The first technical component of IBCA is Gaussian mixture multi-label spatial attention with a variational information bottleneck. Given an input image $A$2, a hybrid ViT backbone, specifically ResNet-50 plus ViT-B/16 with multiple class tokens, produces patch tokens $A$3 and class tokens $A$4 (Cui et al., 11 Aug 2025). Multi-label spatial attention maps over patches are then modeled as a Gaussian mixture in which the number of components $A$5 equals the number of classes $A$6. For label $A$7, the attention map is modeled by a Gaussian component with mean vector $A$8, covariance or variance $A$9, and mixture coefficient $Z$0, giving

$Z$1

with $Z$2 and $Z$3.

The paper describes sampling in three steps. A new set of mixture weights $Z$4 is sampled from a Gamma distribution for each class. For each component, a vector is sampled from the Gaussian prior $Z$5. These samples form a spatial attention sample $Z$6 for all classes over patches. The resulting process is described as adaptive token grouping, because patches are probabilistically assigned to class-specific Gaussian components, producing label-conditioned spatial attention. Class-aware patch features are then computed as

$Z$7

followed by Global Average Pooling and a multi-label classifier supervised by the Multi-Label Soft Margin loss.

The variational bottleneck begins from

$Z$8

and is made tractable by approximating the first term with the prediction loss and the second term with a KL divergence between an approximate posterior and a prior over Gaussian components. The prior is a fixed spherical Gaussian,

$Z$9

and the posterior for each class is approximated by $Y$0, with

$Y$1

under the derivation’s simplifying assumption that the dimension $Y$2.

The stated effect of this construction is twofold. Minimizing the KL term limits how much information from $Y$3 can flow into $Y$4, while the classifier loss forces attention to retain the spatial patterns most informative for predicting $Y$5. The paper therefore attributes to this branch the removal of noise and class-irrelevant information. It also argues that using one Gaussian component per class overcomes the limitation of a single spherical Gaussian in multi-label scenarios, because multiple label-specific attention patterns can then be represented explicitly.

4. Contrastive causal intervention and overall architecture

The second technical component is Contrastive Enhancement-based Causal Intervention (CECI), which links multi-head self-attention in the Transformer to the Gaussian mixture spatial attention learned by the bottleneck branch (Cui et al., 11 Aug 2025). From the last Transformer block, each head $Y$6 has an attention matrix

$Y$7

which is decomposed into class-to-patch attention

$Y$8

and patch-to-patch affinity

$Y$9

These are multiplied to form a class-specific attention sample $C$0.

The intervention branch does not use these maps directly to attend features. Following MCTFormer+, it uses all tokens for classification. In parallel to the patch-token VIB branch, it applies Global Average Pooling to class tokens $C$1 to obtain class-specific features $C$2 and supervises them with a Multi-Label Soft Margin loss $C$3. The paper characterizes this multi-head attention sampling as an implicit implementation of the do-calculus-based intervention, because it aggregates multiple class-specific attention samples across heads and uses class tokens as causal features.

To inject denoised, label-specific knowledge from the Gaussian mixture bottleneck into this branch, IBCA introduces the Contrastive Enhancement Attention loss. After sigmoid activation is applied to both the Gaussian mixture spatial attention $C$4 and each head’s class-specific attention $C$5, the cosine similarity loss for head $C$6 is

$C$7

and the average across heads is

$C$8

This alignment is described as contrastive because it encourages each multi-head self-attention map to align with the Information Bottleneck-cleaned spatial attention. The paper attributes to this loss a progressive mitigation of spurious and noisy parts of attention.

The full network therefore consists of a hybrid ViT backbone with patch tokens and multiple class tokens, a GM-VIB patch-token branch, and a causal intervention class-token branch. The total loss is

$C$9

with $\mathbf{N}$0 and $\mathbf{N}$1. Training is end-to-end with Adam, learning rate $\mathbf{N}$2, batch size $\mathbf{N}$3, and $\mathbf{N}$4 epochs. During inference, the final multi-label prediction is the mean of the patch-token prediction from the GM-VIB branch and the class-token prediction from the causal branch.

5. Empirical evaluation and interpretability

The empirical study uses two datasets: Endo, with 3,865 colonoscopy images and 4 lesion types, and MuReD, with 2,208 retinal fundus images from ARIA, STARE, and RFMiD and 20 label types; both are split 8:1:1 into train, validation, and test sets (Cui et al., 11 Aug 2025). Evaluation uses Class-wise Recall, Class-wise F1, Overall Recall, Overall F1, and mean Average Precision. Baselines include general multi-label methods TA-DCL, TS-Former, MLSL-Net, C-Tran, and Q2L, together with causal methods CCD, IDA, and ML-C.

For MuReD, the detailed results description reports that the best previous causal method, ML-C, achieved CR 59.69, OR 64.91, and mAP 68.41, whereas IBCA achieved CR 66.74, OR 72.63, and mAP 73.43. The same description states improvements over ML-C of CR +7.05%, OR +7.72%, and mAP +5.02%, and notes that “the text mentions 4.95%, but table shows 73.43 vs 68.41.” The abstract separately reports that, compared to the second-best results for each metric, IBCA achieves improvements of 6.35% in CR, 7.72% in OR, and 5.02% in mAP for MuReD. For Endo, the detailed description reports that IDA obtained CR 64.71, CF1 70.04, and mAP 74.96, while IBCA obtained CR 66.18, CF1 72.05, and mAP 76.38, corresponding to improvements of CR +1.47%–1.74%, CF1 +2.01%, and mAP +1.42%; the abstract reports 1.47% in CR, 1.65% in CF1, and 1.42% in mAP. These reporting differences are part of the record rather than a separate interpretation.

The ablation study compares four variants: Basic, with no Gaussian mixture, no VIB, and no CAE; Single VIB, with a single Gaussian and no CAE; GMM VIB, with Gaussian mixture plus VIB but no CAE; and the full model, GM-VIB plus CAE. The reported observations are consistent across the two datasets. Moving from Basic to Single VIB or GMM VIB increases CR and OR notably, which is presented as evidence that IB alone improves attention quality and prediction by filtering task-irrelevant information. Moving from Single VIB to GMM VIB improves most metrics, including Endo gains of CR +2.05 and CF1 +1.99 relative to Single VIB, which is used to support the claim that one Gaussian component per label is better than a single Gaussian for all labels. Adding CAE on top of GMM VIB yields further improvements in CR, OR, and some F1 metrics, which the paper interprets as evidence that aligning multi-head self-attention with Gaussian-mixture attention enhances causal, discriminative attention.

The qualitative analysis is consistent with the same interpretation. Figure 1 is described as showing that baseline causal methods such as IDA and CCD often include task-irrelevant regions and spurious context, and that ML-C still shows non-lesion regions, whereas IBCA’s attention maps more cleanly highlight lesion areas, including fine structures such as retinal veins. Patch-token-based class-specific features under IBCA also show clearer class separation in t-SNE visualizations on both Endo and MuReD.

6. Interpretation, limitations, and relation to adjacent research

The paper presents IBCA as the first integration of Information Bottleneck and causality for multi-label classification of medical images, with explicit modeling of attention as a mixture of causal, spurious, and noisy factors (Cui et al., 11 Aug 2025). Within that framing, the Gaussian mixture branch is the mechanism for noise filtering and label-specific separation, while the intervention branch plus contrastive alignment is the mechanism for spurious reduction. The method is therefore positioned simultaneously as an attention model, a causal attention model, and a bottlenecked latent-variable model. The paper does not present formal theorems, but it states that the causal and information-theoretic rationale is supported by empirical ablations.

The paper does not deeply emphasize limitations, but several are explicitly presented as inferred challenges. These include hyperparameter sensitivity, because $\mathbf{N}$5 and $\mathbf{N}$6 must balance compression against predictive power and alignment strength; assumptions in the SCM, because confounders $\mathbf{N}$7 and noise $\mathbf{N}$8 are implicit and mostly conceptual rather than explicit or observable; Gaussian-mixture complexity, because one Gaussian per class scales linearly with the number of labels; and domain specificity, because the empirical study is restricted to colonoscopy and retinal fundus images. A plausible implication is that IBCA’s causal claims are strongest as a structured modeling interpretation of attention rather than as a fully identified causal model with observed confounders.

In the broader literature represented here, several adjacent strands pursue related combinations of compression and causality. Markov-blanket-restricted IB for domain adaptation defines a “mechanism-stable representation” by compressing only variables in the target’s Markov blanket (Javidian, 7 Jan 2026). “The Causal Information Bottleneck and Optimal Causal Variable Abstractions” formalizes a Causal Information Bottleneck objective that replaces predictive information with causal information gain under interventions (Simoes et al., 2024). “Towards Minimal Causal Representations for Human Multimodal Language Understanding” combines unimodal Information Bottleneck filtering, a mask generator that splits causal and shortcut subrepresentations, instrumental variables, and backdoor-style recombination (Jiang et al., 26 Sep 2025). “Counterfactual Supervision-based Information Bottleneck for Out-of-Distribution Generalization” shows that IB alone can fail in classification and uses counterfactual supervision to distinguish causal from spurious subspaces (Deng et al., 2022). These neighboring lines of work suggest that IBCA belongs to a larger research program in which compression is used not only for regularization but also for isolating causally reliable structure.

Within that landscape, IBCA is specialized to class-specific attention in multi-label medical image recognition. Its distinctive contribution is the coupling of Gaussian mixture spatial attention, variational bottlenecking, and multi-head causal intervention with contrastive alignment. This suggests a design pattern for later work: learn a compact, label-specific attention representation that is explicitly denoised before it is used to regularize a separate attention mechanism that is meant to support causal interpretation.