Spatial-Aware Correlated MIL

• Spatial-Aware Correlated MIL (SAC-MIL) is a framework that relaxes i.i.d. assumptions by incorporating spatial structure such as patch coordinates and adjacency graphs into MIL models.
• It employs various techniques—including spatial encoders, relative distance biases, and graph regularizers—to capture and leverage inter-instance correlations effectively.
• The 2025 SAC-MIL architecture achieves linear time and memory efficiency while significantly improving classification metrics in computational pathology.

Spatial-Aware Correlated Multiple Instance Learning (SAC-MIL) denotes a class of multiple instance learning methods that relax the independent-and-identically-distributed assumption on instances within a bag and explicitly model spatial or structural dependence among those instances. In computational pathology, where a whole-slide image (WSI) is partitioned into many patches but only slide-level labels are available, SAC-MIL replaces an unordered-bag view with formulations that encode patch coordinates, local neighborhoods, relative distances, adjacency graphs, or segmentation-defined groups. In the literature, the term is used in both a broad sense, as a correlated-MIL framework for spatially organized bags (Shao et al., 2021), and in a narrow sense, as a 2025 architecture combining a positional encoding module with an MLP-based SAC block for full instance correlation in linear time (Bai et al., 4 Sep 2025).

1. Problem setting and motivation

In binary MIL, one observes bags $X=\{x_1,\dots,x_n\}$ of instances $x_i\in\mathbb R^{d_x}$ with only a bag label $Y\in\{0,1\}$. Two equivalent formulations recur in the literature: the classical rule $Y=1\iff \exists j:y_j=1$, and the equivalent statement $Y=1\iff \sum_{i=1}^n y_i>0$, where the instance labels $y_i$ are unobserved (Shao et al., 2021, Wölflein et al., 2023). Standard embedding-level MIL maps each $x_i$ to an embedding, aggregates the set by mean, max, or attention, and then classifies.

The spatial problem arises because many medically important MIL bags are not arbitrary sets. In WSI analysis, gigapixel slides are cut into patches; in volumetric CT, scans are decomposed into ordered slices. Treating these instances as independent discards neighborhood structure, tissue architecture, and serial continuity. The pathology-oriented formulations explicitly note that patch arrangement is dropped under i.i.d. assumptions, while the CT formulation observes that neighboring slices should have similar diagnostic importance (Wölflein et al., 2023, Wu et al., 2023).

This motivates SAC-MIL: spatial context is promoted from a nuisance variable to a first-class inductive bias. The surveyed methods do this in materially different ways. Some add spatial encoders to token embeddings, some alter the attention kernel to depend on pairwise distance, some regularize attention logits by graph smoothness, some reconstruct spatial adjacency, and some use segmentation masks to define groups or pseudo-bags. A central theme is that absolute positional encodings alone are often insufficient. In particular, attention-based MIL with absolute positional encodings may still fail to compare relative distances and may not be rotationally invariant, which is consequential in settings such as tumor–immune cell proximity (Wölflein et al., 2023).

2. Correlated MIL as a formal framework

A canonical formalization appears in TransMIL, which introduces “correlated MIL” and treats bag prediction as approximation of a continuous scoring set-function $S:\mathcal X\to\mathbb R$ on bags, with continuity measured in the Hausdorff metric $d_H$. The model family is written as

$$\hat Y \;=\; g\bigl(P\,\{\,f(x_j)+h(x_j)\colon j=1,\dots,n\}\bigr),$$

where $x_i\in\mathbb R^{d_x}$0 extracts morphological features, $x_i\in\mathbb R^{d_x}$1 encodes spatial or contextual features, $x_i\in\mathbb R^{d_x}$2 is a pooling matrix whose off-diagonal entries capture learned correlations among instances, and $x_i\in\mathbb R^{d_x}$3 is the final classifier (Shao et al., 2021).

Two theoretical claims organize much of the subsequent SAC-MIL literature. First, TransMIL states a universal approximation result: every continuous set function can be approximated arbitrarily well by a model of the form $x_i\in\mathbb R^{d_x}$4, and by taking $x_i\in\mathbb R^{d_x}$5, the same universality extends to the decomposition $x_i\in\mathbb R^{d_x}$6. Second, an entropy argument shows that when bag instances are not i.i.d., the joint Shannon entropy is strictly less than the sum of marginals, implying reduced uncertainty when inter-instance dependence is modeled (Shao et al., 2021).

These results are significant because they elevate spatial context from an empirical heuristic to a mathematically motivated component of MIL. They also clarify that “correlation” in SAC-MIL is not restricted to self-attention. The formal role of $x_i\in\mathbb R^{d_x}$7 is broader: any operator whose off-diagonal structure captures dependence can instantiate correlated MIL. This interpretation is consistent with later methods that use sparse convolutions, graph regularizers, smoothness penalties, Gaussian-process couplings, or segmentation-guided grouping rather than Transformer attention.

3. Spatial encoders and correlation operators

Several distinct mechanism families instantiate SAC-MIL.

TransMIL encodes spatial structure through the Pyramid Position Encoding Generator (PPEG). Patch embeddings are reshaped into a padded 2D grid, processed by parallel depthwise or grouped convolutions with kernels $x_i\in\mathbb R^{d_x}$8, $x_i\in\mathbb R^{d_x}$9, and $Y\in\{0,1\}$0, fused with the original grid, and flattened back to a sequence before self-attention. This realizes the contextual term $Y\in\{0,1\}$1 and interleaves convolutional spatial context with long-range Transformer correlation (Shao et al., 2021).

DAS-MIL introduces distance-aware self-attention for MIL and is described as the first application of relative position representations in the MIL setting. Instead of discrete relative-position buckets, it defines learned bias vectors $Y\in\{0,1\}$2 as continuous functions of the Euclidean centroid distance $Y\in\{0,1\}$3 between instances. A representative parameterization is

$Y\in\{0,1\}$4

with analogous forms for $Y\in\{0,1\}$5 and $Y\in\{0,1\}$6. This replaces discrete distance bins with a learned continuous interpolation that generalizes to unseen distances and naturally preserves ordering (Wölflein et al., 2023).

SA-DMIL addresses ordered instances by regularizing the latent attention function over a graph. For a bag $Y\in\{0,1\}$7 with attention logits $Y\in\{0,1\}$8 and graph Laplacian $Y\in\{0,1\}$9, it introduces first- and second-order penalties

$Y=1\iff \exists j:y_j=1$0

The global objective interpolates between bag-level binary cross-entropy and one of these smoothness terms by a coefficient $Y=1\iff \exists j:y_j=1$1. This constrains discrete first and second derivatives of the attention logits along the spatial chain, encouraging smoother slice importance profiles (Wu et al., 2023).

SparseConvMIL realizes correlation through sparse convolution rather than attention. Tile embeddings are placed onto a sparse 2D grid using quantized tile-center coordinates, and a sparse-input CNN performs local mixing only at active sites. Under the submanifold sparse-convolution rule, neighboring embeddings within a $Y=1\iff \exists j:y_j=1$2 window are convolved on the sparse map and then globally pooled for slide classification. The correlation operator is therefore explicitly local and translation-structured (Lerousseau et al., 2021).

CARMIL moves correlation into regularization. It builds a $Y=1\iff \exists j:y_j=1$3-nearest-neighbor spatial graph on tile coordinates, uses a GCN-based spatial encoder $Y=1\iff \exists j:y_j=1$4 and a decoder $Y=1\iff \exists j:y_j=1$5 to reconstruct the adjacency matrix, and trains the resulting embedding jointly with the MIL objective via a binary-cross-entropy reconstruction loss

$Y=1\iff \exists j:y_j=1$6

The same work also proposes DeltaCon as a context-awareness metric by comparing the induced embedding graph to the true spatial adjacency (Saada et al., 2024).

A probabilistic variant appears in VGPMIL-PR-I, which augments a sparse Gaussian-process MIL model with an Ising-inspired quadratic coupling term

$Y=1\iff \exists j:y_j=1$7

where $Y=1\iff \exists j:y_j=1$8 are latent continuous variables and $Y=1\iff \exists j:y_j=1$9 is the patch-adjacency Laplacian within a bag. When $Y=1\iff \sum_{i=1}^n y_i>0$0, the model reduces to standard VGPMIL-PR. This is a direct instance-label correlation model rather than a feature-correlation model (Morales-Álvarez et al., 2023).

SAM-MIL uses segmentation masks from the Segment Anything Model (SAM) to define spatial groups. Patch features within each segment are average-pooled into group tokens, redundant patches are removed by SAM-Guided Group Masking, the remaining instances are split into pseudo-bags, and a spatial consistency loss is imposed on attention weights relative to SAM segment categories. This gives SAC-MIL a segmentation-conditioned notion of context rather than a purely geometric one (Fang et al., 2024).

4. SAC-MIL as a specific 2025 architecture

In its narrow, model-specific sense, SAC-MIL refers to “SAC-MIL: Spatial-Aware Correlated Multiple Instance Learning for Histopathology Whole Slide Image Classification” (Bai et al., 4 Sep 2025). The pipeline begins with feature extraction for all patches in a WSI, followed by division of the $Y=1\iff \sum_{i=1}^n y_i>0$1 instances into $Y=1\iff \sum_{i=1}^n y_i>0$2 spatially localized regions via Farthest Point Sampling on 2D patch coordinates plus K-Nearest Neighbors. Within each region, instances are reordered into contiguous sub-sequences of length $Y=1\iff \sum_{i=1}^n y_i>0$3. Features are projected by an input MLP to dimension $Y=1\iff \sum_{i=1}^n y_i>0$4, positional embeddings are added, $Y=1\iff \sum_{i=1}^n y_i>0$5 SAC blocks are stacked, and the final instance embeddings are globally averaged and classified by another MLP.

Its positional encoding module, PROPE, departs from token-index encodings and instead uses normalized polar coordinates derived from patch positions within each slide. Raw coordinates are normalized to $Y=1\iff \sum_{i=1}^n y_i>0$6 on each axis, transformed into radius and angle, scaled by a factor $Y=1\iff \sum_{i=1}^n y_i>0$7, and then injected through a ROPE-style complex rotation. The stated motivation is twofold: token index does not reflect real 2D layout in WSIs, and coordinate normalization addresses the length extrapolation issue where training and testing sequences have different lengths (Bai et al., 4 Sep 2025).

The SAC block is designed to perform full instance correlation in linear time and space. After positional embedding, the input matrix $Y=1\iff \sum_{i=1}^n y_i>0$8 is arranged so each block of $Y=1\iff \sum_{i=1}^n y_i>0$9 rows corresponds to one region. Each instance embedding is split into folds, channels are shifted across instances by a deterministic roll schedule that depends on fold index and layer depth, a channel-wise MLP $y_i$0 is applied, the shift is reversed so that information is redistributed back to original positions, and a second channel-wise MLP $y_i$1 is applied. With exponentially growing region sizes across layers, the effective context length becomes the full region size after only $y_i$2 layers, reported in practice as 3 layers (Bai et al., 4 Sep 2025).

A major distinction from Transformer-based MIL is computational. The paper contrasts Transformer self-attention at $y_i$3 time and $y_i$4 memory with SAC-MIL’s $y_i$5 time and $y_i$6 memory per layer. Because the architecture uses only channel-wise MLPs and tensor rolls, it does not require custom CUDA kernels and is presented as easier to deploy than attention-based WSI classifiers (Bai et al., 4 Sep 2025).

5. Empirical results, ablations, and interpretability

Reported evaluations indicate that spatially aware correlated MIL improves several distinct problem classes rather than a single benchmark.

TransMIL reports AUC $y_i$7 on CAMELYON16, $y_i$8 on TCGA-NSCLC, and $y_i$9 on TCGA-RCC. Its spatial-encoding ablation on CAMELYON16 lists AUC $x_i$0 with no position encoding, $x_i$1 with sinusoidal encoding, $x_i$2 with a single $x_i$3 convolution, and $x_i$4 with multi-kernel PPEG. Shuffling token order causes a 0.9 point AUC drop on CAMELYON16 and a 0.6 point drop on NSCLC, supporting the claim that the network exploits spatial locality (Shao et al., 2021).

DAS-MIL reports test balanced accuracy $x_i$5 on MNIST-COLLAGE and $x_i$6 on the inverted variant, together with CAMELYON16 test AUROC $x_i$7 and balanced accuracy $x_i$8. In the same comparison, AB-MIL yields $x_i$9, vanilla self-attention $S:\mathcal X\to\mathbb R$0, discrete relative self-attention $S:\mathcal X\to\mathbb R$1, and TransMIL $S:\mathcal X\to\mathbb R$2, while DAS-MIL uses 412 K parameters against 2.5 M for TransMIL. Its qualitative attention maps highlight exactly the “0”–“1” pair within threshold $S:\mathcal X\to\mathbb R$3, while ignoring identical digits placed farther away (Wölflein et al., 2023).

SA-DMIL, on the 2019 RSNA ICH detection challenge dataset, reports scan-level Accuracy $S:\mathcal X\to\mathbb R$4 versus $S:\mathcal X\to\mathbb R$5, F1 $S:\mathcal X\to\mathbb R$6 versus $S:\mathcal X\to\mathbb R$7, and AUC $S:\mathcal X\to\mathbb R$8 versus $S:\mathcal X\to\mathbb R$9 for SA-DMIL-S1 against Att-MIL. At slice level, the same setting yields Accuracy $d_H$0 versus $d_H$1 and F1 $d_H$2 versus $d_H$3. The paper attributes these gains to smoother attention curves and emphasizes that they are achieved without adding model parameters (Wu et al., 2023).

SparseConvMIL reports $d_H$4 accuracy and F1 $d_H$5 on CRCHistoPhenotype, surpassing max, mean, log-sum-exp, and attention MIL baselines. On TCGA pan-cancer subtype classification with 32 classes, SparseConvMIL-c32,c32 attains balanced accuracy $d_H$6, F1 $d_H$7, AUC $d_H$8, and SparseConvMIL-c128,c128 reaches balanced accuracy $d_H$9, F1 $$\hat Y \;=\; g\bigl(P\,\{\,f(x_j)+h(x_j)\colon j=1,\dots,n\}\bigr),$$0, AUC $$\hat Y \;=\; g\bigl(P\,\{\,f(x_j)+h(x_j)\colon j=1,\dots,n\}\bigr),$$1, while the reported MIL baselines plateau near balanced accuracy $$\hat Y \;=\; g\bigl(P\,\{\,f(x_j)+h(x_j)\colon j=1,\dots,n\}\bigr),$$2–$$\hat Y \;=\; g\bigl(P\,\{\,f(x_j)+h(x_j)\colon j=1,\dots,n\}\bigr),$$3 and AUC approximately $$\hat Y \;=\; g\bigl(P\,\{\,f(x_j)+h(x_j)\colon j=1,\dots,n\}\bigr),$$4–$$\hat Y \;=\; g\bigl(P\,\{\,f(x_j)+h(x_j)\colon j=1,\dots,n\}\bigr),$$5 (Lerousseau et al., 2021).

SAM-MIL reports CAMELYON-16 AUC $$\hat Y \;=\; g\bigl(P\,\{\,f(x_j)+h(x_j)\colon j=1,\dots,n\}\bigr),$$6, Accuracy $$\hat Y \;=\; g\bigl(P\,\{\,f(x_j)+h(x_j)\colon j=1,\dots,n\}\bigr),$$7, and F1 $$\hat Y \;=\; g\bigl(P\,\{\,f(x_j)+h(x_j)\colon j=1,\dots,n\}\bigr),$$8, improving over MHIM-MIL at $$\hat Y \;=\; g\bigl(P\,\{\,f(x_j)+h(x_j)\colon j=1,\dots,n\}\bigr),$$9 AUC. On TCGA Lung Cancer, it reports AUC $x_i\in\mathbb R^{d_x}$00, Accuracy $x_i\in\mathbb R^{d_x}$01, and F1 $x_i\in\mathbb R^{d_x}$02, compared with $x_i\in\mathbb R^{d_x}$03 AUC for MHIM-MIL. The reported gains are statistically significant under paired $x_i\in\mathbb R^{d_x}$04-test with $x_i\in\mathbb R^{d_x}$05, and ablations attribute 1–2 AUC points to each of instance-level masking, pseudo-bag consistency, and group tokens (Fang et al., 2024).

CARMIL reports average C-index $x_i\in\mathbb R^{d_x}$06 versus $x_i\in\mathbb R^{d_x}$07 on TCGA-COAD and $x_i\in\mathbb R^{d_x}$08 versus $x_i\in\mathbb R^{d_x}$09 on TCGA-GBM, with best single-model improvement up to $x_i\in\mathbb R^{d_x}$10 percentage points on COAD. Shuffling adjacency at test time causes a systematic C-index drop, and DeltaCon increases from frozen Phikon features at approximately $x_i\in\mathbb R^{d_x}$11 to $x_i\in\mathbb R^{d_x}$12–$x_i\in\mathbb R^{d_x}$13 on COAD and $x_i\in\mathbb R^{d_x}$14–$x_i\in\mathbb R^{d_x}$15 on GBM, indicating increased spatial coherence in the learned embedding (Saada et al., 2024).

VGPMIL-PR-I reports improvement on SICAPv2 from instance-level F1 $x_i\in\mathbb R^{d_x}$16 for VGPMIL-PR up to $x_i\in\mathbb R^{d_x}$17, and bag-level F1 from $x_i\in\mathbb R^{d_x}$18 to approximately $x_i\in\mathbb R^{d_x}$19 at $x_i\in\mathbb R^{d_x}$20. On the PANDA subset it reports $x_i\in\mathbb R^{d_x}$21 accuracy and $x_i\in\mathbb R^{d_x}$22 F1 at bag level, exceeding VGPMIL-PR at $x_i\in\mathbb R^{d_x}$23. Visualizations show fewer spurious high-probability patches on negative WSIs and more spatially coherent posterior cancer-probability maps (Morales-Álvarez et al., 2023).

The 2025 SAC-MIL model reports, with ResNet50 features, CAMELYON-16 ACC $x_i\in\mathbb R^{d_x}$24, AUC $x_i\in\mathbb R^{d_x}$25, F1 $x_i\in\mathbb R^{d_x}$26; TCGA-LUNG ACC $x_i\in\mathbb R^{d_x}$27, AUC $x_i\in\mathbb R^{d_x}$28, F1 $x_i\in\mathbb R^{d_x}$29; and TCGA-BRAC ACC $x_i\in\mathbb R^{d_x}$30, AUC $x_i\in\mathbb R^{d_x}$31, F1 $x_i\in\mathbb R^{d_x}$32. With UNI features, the corresponding numbers are CAMELYON-16 ACC $x_i\in\mathbb R^{d_x}$33, AUC $x_i\in\mathbb R^{d_x}$34, F1 $x_i\in\mathbb R^{d_x}$35; TCGA-LUNG ACC $x_i\in\mathbb R^{d_x}$36, AUC $x_i\in\mathbb R^{d_x}$37, F1 $x_i\in\mathbb R^{d_x}$38; and TCGA-BRAC ACC $x_i\in\mathbb R^{d_x}$39, AUC $x_i\in\mathbb R^{d_x}$40, F1 $x_i\in\mathbb R^{d_x}$41. Its ablations state that PROPE raises CAMELYON-16 AUC by $x_i\in\mathbb R^{d_x}$42 over the next best positional encoding and that SAC blocks improve AUC by approximately $x_i\in\mathbb R^{d_x}$43 over Cycle-MLP, Chord-Mixer, Nyström, and FlashAttn alternatives (Bai et al., 4 Sep 2025).

6. Scope, limitations, and conceptual distinctions

A recurrent conceptual distinction is that spatial awareness in MIL is not synonymous with adding absolute positional encodings. The surveyed methods encode spatial structure through Euclidean distance, graph Laplacians, sparse coordinate maps, segmentation masks, or reconstructed adjacency, and several papers explicitly position their contribution as a correction to the assumption that bag instances are independent or merely index-ordered (Wölflein et al., 2023, Wu et al., 2023, Saada et al., 2024). This suggests that SAC-MIL is best understood as a modeling principle—explicit correlation under known or inferred structure—rather than a single architecture.

Another distinction is that correlation can be imposed at different levels. DAS-MIL modifies attention scores with continuous distance-aware biases; SA-DMIL regularizes attention logits; TransMIL and the 2025 SAC-MIL correlate enriched embeddings; SparseConvMIL correlates spatially placed embeddings through convolution; CARMIL regularizes embeddings by adjacency reconstruction; VGPMIL-PR-I correlates latent instance-label variables; and SAM-MIL correlates instances through segmentation-conditioned grouping and consistency constraints. A plausible implication is that “correlation” in SAC-MIL names a family of operators with different inductive biases and failure modes, not a unique computational primitive.

Several limitations are explicit. In VGPMIL-PR-I, large $x_i\in\mathbb R^{d_x}$44 over-smooths and can hurt recall on positive WSIs, and the diagonal approximation of the covariance in the truncated-Gaussian update limits full use of the spatial prior (Morales-Álvarez et al., 2023). In DAS-MIL, fixing the continuous weighting function $x_i\in\mathbb R^{d_x}$45 or its scalars $x_i\in\mathbb R^{d_x}$46 cripples performance, indicating that spatial weighting must remain learned rather than hard-coded (Wölflein et al., 2023). In CARMIL, the average gain on TCGA-GBM is $x_i\in\mathbb R^{d_x}$47 percentage points, which is consistent but modest, and the decoder is discarded at inference time (Saada et al., 2024). In the 2025 SAC-MIL, the authors note that the method focuses on slide-level binary classification, that its behavior under extremely small sample sizes or highly irregular tissue shapes merits further study, and that region partitioning by FPS+KNN may be suboptimal for disconnected tissue (Bai et al., 4 Sep 2025).

Future directions are already foreshadowed by the cited work. Proposed extensions include multi-scale distance functions, direction-aware biases based on $x_i\in\mathbb R^{d_x}$48, hierarchical or graph-based aggregation, survival prediction, segmentation, biomarker discovery, multimodal fusion with genomics, and hybrid SAC/Transformer stacks (Wölflein et al., 2023, Bai et al., 4 Sep 2025). More generally, the same spatially correlated MIL principle is presented as applicable beyond histopathology to ecology, materials microscopy, spatial transcriptomics, temporally ordered frames, and multi-view imaging when instances carry real-valued coordinates or known adjacency (Wölflein et al., 2023, Wu et al., 2023).