Semantic-guided Retrieval State Space Module (SRSM)

• SRSM is a query-conditioned module that selects diagnostically important patches to adapt state space models, addressing challenges in weakly ordered WSI data.
• It integrates semantic reordering with selective retrieval to mitigate noise and capture vital global dependencies while preserving linear computational complexity.
• Empirical evaluations show that SRSM boosts classification accuracy and efficiency by dynamically calibrating state-space dynamics based on semantically informed queries.

Searching arXiv for the cited papers to ground the article in the current literature. Semantic-guided Retrieval State Space Module (SRSM) is a query-conditioned state space component introduced within SemaMIL for whole slide image (WSI) classification under multiple instance learning (MIL). In that framework, Semantic Reordering (SR) first clusters and arranges semantically similar patches in sequence through a reversible permutation, and SRSM then chooses a representative subset of queries to adjust state space parameters for improved global modeling (Gan et al., 30 Aug 2025). The module is designed for extremely long WSI-derived patch sequences, where linear-complexity state space models (SSMs) are attractive but uniform sequence modeling can dilute diagnostically salient structure.

1. Conceptual position in SemaMIL

SemaMIL combines two mechanisms: Semantic Reordering and the Semantic-guided Retrieval State Space Module. SR aligns semantically similar patches in sequence, facilitating stronger local and regional interactions, while SRSM operates after semantic reordering and augments the sequence modeling step to better capture long-range and global dependencies (Gan et al., 30 Aug 2025).

The design is motivated by a specific limitation of standard MIL backbones for WSI analysis. Attention-based MIL methods can identify key patches but tend to overlook contextual relationships. Transformer models can model interactions but require quadratic computational cost and are prone to overfitting. SSMs offer linear complexity, yet shuffling patch order disrupts histological meaning and reduces interpretability. Within this setting, SRSM is not a generic retrieval layer over an external corpus; it is an internal mechanism that retrieves a semantically informed subset of patches from the reordered bag and uses that subset to modulate the subsequent state-space scan (Gan et al., 30 Aug 2025).

2. Problem formulation and design rationale

WSI-derived patch sequences are extremely long, and plain SSM or Mamba-style processing can lead to two failure modes: redundancy due to background or noise patches, and overlooking morphologically critical but spatially dispersed patches. SRSM addresses these issues by selecting a semantic-informed subset of patches, denoted as queries, that are important for the slide’s diagnosis, and by employing those queries to dynamically configure the state-space propagation parameters during the main sequence scan (Gan et al., 30 Aug 2025).

The central idea is therefore selective modulation rather than uniform propagation. Semantic guidance ensures that parameter modulation is informed by the most diagnostically relevant tissue structures rather than by background, noise, or arbitrary positions. The paper further states that this enhances efficiency and model performance with minimal overhead (Gan et al., 30 Aug 2025).

A concise way to characterize the module is that it separates the reordered sequence into a small set of semantically important conditioning tokens and a larger context sequence to be scanned. This division allows the SSM dynamics to become slide-specific and bag-specific rather than fixed across all inputs.

3. Mathematical construction

SRSM begins with the semantically reordered sequence $\{x_i\}_{i=1}^N$. Each patch $x_i$ receives an importance score via a lightweight linear projection, and the top-$K$ patches form the query set $Q = \{x_{i_j}\}_{j=1}^K$. The remaining patches form the context sequence $C = \{c_k\}_{k=1}^{N-K}$ (Gan et al., 30 Aug 2025).

The underlying model is a discrete-time state-space system derived from the continuous linear time-invariant form:

$$\dot{h}(t) = Ah(t) + Bx(t), \quad y(t) = Ch(t) + Dx(t)$$

After zero-order-hold discretization, the recurrence becomes:

$h_k = A_d h_{k-1} + B_d x_k$

$y_k = C h_k + D x_k$

SRSM’s distinctive step is to derive the discrete-time parameters adaptively from the semantic query set $Q$. The query set is concatenated and projected to obtain the step size $\Delta$ and an intermediate input matrix $x_i$0:

$x_i$1

$x_i$2

Using a fixed base transition matrix $x_i$3, the module computes:

$x_i$4

$x_i$5

The context sequence is then causally scanned with these adapted parameters:

$x_i$6

$x_i$7

This construction makes the propagation dynamics conditional on a compact semantic summary of the slide rather than solely on the local token stream (Gan et al., 30 Aug 2025).

Element	Role	Definition
$x_i$8	Query set	Top-$x_i$9 important patches
$K$0	Context sequence	Remaining reordered patches
$K$1	Adaptive step size	$K$2
$K$3	Adaptive input term	$K$4
$K$5	Discrete transition	$K$6
$K$7	Discrete input map	Query-conditioned discretization term

4. Sequence processing, multi-directionality, and implementation scope

The paper describes SRSM as a causal scan over the context sequence using the query-conditioned parameters. In this interpretation, $K$8 encodes local tissue continuity tailored to the current slide, while $K$9 gates the passage of context information, enabling suppression of noise and integration of high-level structure as specified by $Q = \{x_{i_j}\}_{j=1}^K$0 (Gan et al., 30 Aug 2025).

The process can also be executed in parallel along four scan directions for robust two-dimensional context, producing output sequences $Q = \{x_{i_j}\}_{j=1}^K$1 for $Q = \{x_{i_j}\}_{j=1}^K$2. These outputs are globally pooled to form a fixed-dimensional bag representation for final classification (Gan et al., 30 Aug 2025). This multi-directional scan is important because WSI data are inherently two-dimensional even when processed as sequences.

Implementation-wise, the number of query patches $Q = \{x_{i_j}\}_{j=1}^K$3 is a hyperparameter controlling selectivity, and the sizes of $Q = \{x_{i_j}\}_{j=1}^K$4 and $Q = \{x_{i_j}\}_{j=1}^K$5 depend on how $Q = \{x_{i_j}\}_{j=1}^K$6 is vectorized. Other hyperparameters are inherited from the main SSM backbone, including sequence length, embedding size, and the number of SSM blocks or layers (Gan et al., 30 Aug 2025). The summary table in the paper further characterizes SRSM as dynamic, per-slide, per-bag, and as adding only a small parameter head over the base SSM.

5. Empirical behavior in ablation and efficiency analysis

The empirical results reported for SemaMIL indicate that SRSM contributes both to accuracy and to efficiency. Evaluation on four WSI subtype datasets shows that, compared to strong baselines, SemaMIL achieves state-of-the-art accuracy with fewer FLOPs and parameters (Gan et al., 30 Aug 2025).

The ablation reported for EBRAINS shows additive benefit from SR and SRSM:

Configuration	EBRAINS ACC	Note
Baseline (no SR, no SRSM)	72.2%	Reference
+SRSM only	73.5%	SRSM gain
+SR only	74.0%	SR gain
+SR + SRSM	75.1%	Best ablation

The same section states that similar gains are seen on BRACS and that both modules are additively beneficial (Gan et al., 30 Aug 2025). In the computational efficiency table, SemaMIL is reported with FLOPs of 0.248 G, parameters of 0.464 M, and ACC of 75.1%, described as best among all methods compared for FLOPs and Params and highest for ACC (Gan et al., 30 Aug 2025).

These results situate SRSM as more than an interpretability or routing device. Within the reported experiments, it is a functional contributor to slide-level classification quality while preserving the linear-cost advantages associated with SSM backbones.

6. Relation to broader SSM retrieval research and terminological boundaries

SRSM belongs to a broader line of work that asks how SSMs handle retrieval and long-range structure, but its use of “retrieval” is specialized. In SemaMIL, retrieval refers to selecting a representative subset of semantically important patches from an internal reordered sequence and using that subset to modulate state-space dynamics (Gan et al., 30 Aug 2025). This differs from retrieval-state formulations in retrieval-augmented generation, where the retrieval state may be defined explicitly as $Q = \{x_{i_j}\}_{j=1}^K$7 over query plans, traversal paths, evidence, verification signals, and reusable artifacts (Niu et al., 25 May 2026). The lexical overlap can therefore be misleading: SRSM is a sequence-modeling module for bag-level WSI classification, not a document-level RAG state manager.

Related SSM analysis in code understanding provides a useful comparative backdrop. One study reports that SSMs can outperform Transformers at capturing code syntax and semantics during pretraining, but may forget certain syntactic and semantic relations during fine-tuning, especially when the task emphasizes short-range dependencies; it introduces SSM-Interpret to expose a spectral shift toward short-range dependencies during fine-tuning (Wu et al., 6 Feb 2026). This suggests a general concern for SSM design: retaining global structure while remaining responsive to local evidence. SRSM addresses that balance by letting semantically important patches determine the effective state transition and input mapping for the context scan.

A separate study on in-context retrieval reports that Transformers excel at retrieval while SSMs offer efficient linear-time processing but have limited retrieval capabilities, and that hybrid architectures often improve content-based retrieval. The same work finds that SSM-based models develop locality-aware embeddings where adjacent positions become neighbors in embedding space, whereas Transformers remain superior on position retrieval tasks (Pantazopoulos et al., 3 Mar 2026). A plausible implication is that SRSM’s query-conditioned parameterization is one strategy for increasing the selectivity of an SSM without incurring full quadratic attention, particularly when the target signal is globally relevant but sparsely distributed across a long sequence.

Taken together, these adjacent results frame SRSM as a semantically conditioned SSM module tailored to long, noisy, weakly ordered medical image sequences: it preserves linear-complexity scanning, but avoids treating all patches as equally informative by letting a small, diagnostically important subset control the effective dynamics of the scan.