Cross-Swin State Block in TSANet

• Cross-Swin State Block is a lightweight dual-branch module combining local cross-modality attention with global state-space modeling for efficient demosaicing.
• It fuses positional priors with image features through parallel QCSA and RVSS branches, enhancing color propagation and robustness on Quad Bayer CFA sensors.
• Implemented in TSANet, CSSB achieves high performance with lower computational cost compared to traditional full self-attention methods.

Cross-Swin State Block (CSSB) denotes a lightweight architectural block introduced in TSANet for Quad Bayer HybridEVS demosaicing. In that setting, it is the core building unit of the encoder in the demosaicing stage and is designed to fuse positional prior with image features while enhancing global dependency modeling under linear complexity (Zhou et al., 8 Aug 2025). The term is specific to that proposal rather than to the original Swin Transformer family at large: Swin Transformer introduced shifted-window self-attention for cross-window connection, but it did not define any explicit module called a “Cross-Swin State Block” (Liu et al., 2021).

1. Terminology and architectural lineage

CSSB is best understood against the background of Swin Transformer’s shifted-window hierarchy. Swin Transformer was proposed as a hierarchical vision Transformer whose representation is computed with shifted windows, combining non-overlapping local windows for efficiency with alternating shifted partitions for cross-window connection (Liu et al., 2021). In that formulation, global multi-head self-attention over an input with $h \times w$ patches has complexity $\mathcal{O}((hw)^2 C)$, whereas window-based self-attention over fixed $M \times M$ windows has complexity $\mathcal{O}(M^2 \cdot hw \cdot C)$, yielding linear complexity with respect to image size when $M$ is fixed (Liu et al., 2021).

The key mechanism in Swin Transformer is the alternation between regular window-based multi-head self-attention and shifted-window multi-head self-attention. Using the notation given for adjacent blocks,

$$\hat{\mathbf{z}}^{l}=\mathrm{W\text{-}MSA}(\mathrm{LN}(\mathbf{z}^{l-1}))+\mathbf{z}^{l-1}, \qquad \mathbf{z}^{l}=\mathrm{MLP}(\mathrm{LN}(\hat{\mathbf{z}}^{l}))+\hat{\mathbf{z}}^{l},$$

$$\hat{\mathbf{z}}^{l+1}=\mathrm{SW\text{-}MSA}(\mathrm{LN}(\mathbf{z}^{l}))+\mathbf{z}^{l}, \qquad \mathbf{z}^{l+1}=\mathrm{MLP}(\mathrm{LN}(\hat{\mathbf{z}}^{l+1}))+\hat{\mathbf{z}}^{l+1}.$$

This alternating pattern enables cross-window state exchange, but the original paper explicitly does so without introducing a dedicated “cross-Swin” state layer (Liu et al., 2021). A common misconception is therefore to treat any cross-window interaction in Swin as a CSSB. The historical record in the cited papers does not support that interpretation.

2. Structural definition of the CSSB

In TSANet, the Cross-Swin State Block is a dual-branch, parallel module that combines local cross-modality attention with global state-space modeling. The two branches are the Quad Bayer Cross Swin Attention (QCSA) branch and the Residual Vision State Space (RVSS) branch. Input image features $\mathbf{F}_I$ are first projected by a $1 \times 1$ convolution and split into two groups; one split is processed jointly with position features $\mathbf{F}_P$ through QCSA, while the other is processed by RVSS. The two outputs are then concatenated, projected, and merged with the input through a residual connection (Zhou et al., 8 Aug 2025).

The block-level computation is summarized in the reported formulation:

$\mathcal{O}((hw)^2 C)$0

$\mathcal{O}((hw)^2 C)$1

$\mathcal{O}((hw)^2 C)$2

Component	Realization	Reported role
Local branch	QCSA	Local cross-modality fusion
Global branch	RVSS	Long-range dependency aggregation
Output fusion	Concat + $\mathcal{O}((hw)^2 C)$3 conv + residual	Recombination and refinement

This composition gives CSSB a character distinct from a plain Swin block. It is not merely a renamed shifted-window attention layer; it is a hybrid block in which a Swin-derived local attention path and a state-space path operate in parallel (Zhou et al., 8 Aug 2025).

3. Positional prior and local cross-attention

A defining feature of CSSB is its explicit use of positional prior. The demosaicing problem considered in TSANet arises from a Quad Bayer CFA sensor paired with event pixels that do not encode color information. The reported motivation is that the Quad Bayer CFA has a non-trivial spatial arrangement of color filters, while event pixel locations introduce structured missing-color regions. CSSB addresses this by using an explicit position or geometry branch carrying CFA type and event pixel positions, then fusing those priors with image features through QCSA (Zhou et al., 8 Aug 2025).

Within QCSA, the cross-attention relation is asymmetric in source: query vectors are derived from position encoding, while key and value vectors come from image features. Attention is computed locally within windows, and the module uses window-relative positional bias. The pseudocode summary given for QCSA is: $M \times M$1 where $\mathcal{O}((hw)^2 C)$4 are learned matrices and $\mathcal{O}((hw)^2 C)$5 is window-relative positional bias (Zhou et al., 8 Aug 2025).

This design makes the local branch a form of cross-modality attention, but the modalities are not PET/CT or multi-view anatomical planes; they are image features and positional priors specific to the sensor layout. A plausible implication is that CSSB should be read less as a generic Swin variant and more as a sensor-aware fusion block built around Swin-style local attention.

4. State-space augmentation and complexity regime

The second branch of CSSB is RVSS, described as a Residual Vision State Space module that adapts Mamba-style state space models to vision. Its role is to process the complementary split of the feature tensor and to aggregate long-range dependencies through efficient scan-style operations rather than through quadratic global attention (Zhou et al., 8 Aug 2025). In the reported rationale, this branch is intended to learn global structural cues that are important for color propagation, large-region reconstruction, and the resolution of long-range ambiguities in demosaicing.

The computational argument follows the same general efficiency trajectory as Swin Transformer, but with a different mechanism for global context. QCSA is confined to windows of size $\mathcal{O}((hw)^2 C)$6, so its cost is stated as $\mathcal{O}((hw)^2 C)$7, which is linear when $\mathcal{O}((hw)^2 C)$8 is small and fixed. RVSS is also described as linear in sequence or image size. By contrast, full self-attention remains quadratic in the number of tokens (Zhou et al., 8 Aug 2025). This places CSSB in a broader line of work that seeks to preserve the deployment advantages of local attention while compensating for its limited receptive field.

The distinction from canonical Swin is important. Swin Transformer achieves cross-window information flow by alternating regular and shifted local windows, implemented with cyclic shift and masking (Liu et al., 2021). CSSB instead couples a local cross-attention path with a state-space path, so that global dependency modeling is not delegated to repeated shifted-window interaction alone.

5. Role inside TSANet and reported empirical position

TSANet is described as a lightweight two-stage network via state space augmented cross-attention, designed to handle event pixels inpainting and demosaicing separately (Zhou et al., 8 Aug 2025). Within that framework, CSSB serves as the core building unit of the encoder for the demosaicing stage, also referred to as Q2R in the detailed summary. Its stated goals are to capture global dependencies efficiently, fuse rich positional priors from the CFA or quasi-CFA layout and event pixels, and maintain linear computational complexity with respect to image size (Zhou et al., 8 Aug 2025).

The abstract reports that TSANet demonstrates excellent demosaicing performance on both simulated and real data while remaining lightweight, and that it averages better results than DemosaicFormer across seven diverse datasets in both PSNR and SSIM, while reducing parameter and computation costs by $\mathcal{O}((hw)^2 C)$9 and $M \times M$0, respectively (Zhou et al., 8 Aug 2025). The detailed summary further attributes qualitative gains to fewer color errors and mosaicking artifacts, sharper textures, and improved robustness to spatially complex CFA or event layouts. It also reports that TSANet-l has 16.26M parameters and 149.4G FLOPs, compared with DemosaicFormer’s 30.28M parameters and 491.1G FLOPs, with “Average” values of PSNR 37.73 and SSIM 0.982 for TSANet-l versus PSNR 37.76 and SSIM 0.979 for DemosaicFormer (Zhou et al., 8 Aug 2025). This suggests that the paper positions CSSB not as an isolated theoretical block, but as an efficiency-quality trade-off mechanism for mobile or edge deployment.

6. Relation to adjacent Swin-based “cross” mechanisms

The phrase “Cross-Swin State Block” has acquired some ambiguity because multiple Swin-derived papers use “cross” to denote different kinds of interaction. SwinCross introduces a Cross-Modal Swin Transformer for PET/CT head-and-neck tumor segmentation, using a dual-branch encoder and Cross-Modal Attention modules at multiple resolutions; the relevant block there is a Cross-Modal Shifted Window Block rather than a CSSB in the TSANet sense (Li et al., 2023). SwinMM uses a cross-view decoder with a cross-attention block at the bottleneck to integrate different anatomical views during fine-tuning; the detailed summary explicitly notes a connection to a “Cross-Swin State Block,” but presents the actual mechanism as cross-view attention between view representations (Wang et al., 2023). In the adenocarcinoma segmentation framework, the reported enhancement is a parallel cross-attention module added to a pre-trained Swin-UNet, and the detailed summary states that the term “Cross-Swin State Block” is not explicitly mentioned there (Qayyum et al., 2024).

A separate but related confusion concerns cross-window communication itself. Swin-Free argues that Swin Transformer’s shifted windows achieve cross-connection among local windows but incur memory copy overhead; it replaces shifting with size-varying windows across stages to obtain cross-window attention more efficiently (Koo et al., 2023). This alternative again underscores that cross-window interaction is an architectural function, not a uniquely defined block name. In the original Swin Transformer, cross-window information flow is intrinsic to alternating regular and shifted windows, and the paper does not define a state block for that purpose (Liu et al., 2021).

Taken together, these papers support a narrow definition of CSSB. Properly used, the term refers to the dual-path block in TSANet that combines QCSA and RVSS for HybridEVS demosaicing (Zhou et al., 8 Aug 2025). Used more loosely, it can blur together distinct constructs such as shifted-window cross-window exchange, cross-modal attention, cross-view attention, or parallel attention augmentation in Swin-derived architectures.