---
title: Omni Selective Scan (OSS) for Vision SSMs
url: https://www.emergentmind.com/topics/omni-selective-scan-oss
type: topic
---

# Omni Selective Scan (OSS) for Vision SSMs

Omni Selective Scan (OSS) is a mechanism for enhancing the spatial modeling capacity of visual state space models (SSMs). OSS addresses the critical limitation of unidirectional or causally sequenced SSMs by enabling efficient, bidirectional, and multi-directional information flow across two-dimensional grid structures and channel dimensions, while maintaining linear computational complexity. OSS underpins recent vision architectures such as VmambaIR and OCTOPUS, facilitating strong global and local feature propagation in a computationally efficient manner and resulting in state-of-the-art performance across various low-level and high-level vision tasks [2403.11423][2602.00904].

## 1. Foundations: State Space Models and Visual Sequence Modeling

State space models (SSMs) are rooted in control theory and are defined by continuous or discrete time dynamics that map sequences of inputs $x(t)\in\mathbb{R}^D$ through a hidden state $h(t)\in\mathbb{R}^N$ to output $y(t)\in\mathbb{R}^D$. The discretized evolution for standard SSMs is governed by:
\[
h_t = \bar{A} h_{t-1} + \bar{B} x_t,\quad y_t = C h_t + D x_t,
\]
where matrices $\bar{A}, \bar{B}, C, D$ are learned. Although SSMs such as S4, S5, and Mamba provide efficient long-range sequence modeling with linear time and memory complexity, naïve application to images via rasterization undermines local spatial relationships and fails to propagate information isotropically across the 2D grid. This causal, 1D formulation links non-adjacent pixels while simultaneously ignoring direct neighbors, impeding spatial coherence crucial for vision tasks [2403.11423][2602.00904].

## 2. Multi-Directional and Omni-Directional Feature Propagation

Omni Selective Scan (OSS) generalizes the recurrence mechanism of SSMs by performing independent, discrete scans in multiple directions. In VmambaIR, OSS performs six bidirectional scans: horizontal forward/backward, vertical forward/backward, and channel-wise forward/backward [2403.11423]. In OCTOPUS, OSS extends this further to eight principal spatial orientations: right ($\rightarrow$), left ($\leftarrow$), down ($\downarrow$), up ($\uparrow$), southeast ($\searrow$), northwest ($\nwarrow$), southwest ($\swarrow$), and northeast ($\nearrow$) [2602.00904].

Each scan processes a set of independent 1D lines (rows, columns, or diagonals for spatial dimensions; channels for depth), applying SSM recurrences:
\[
\begin{aligned}
f_t^{(k,\ell)} &= \sigma(W_f[h_{t-1}^{(k,\ell)}, x_t^{(k,\ell)}] + b_f),\\
h_t^{(k,\ell)} &= f_t^{(k,\ell)} \odot \big(A_k h_{t-1}^{(k,\ell)} + B_k x_t^{(k,\ell)}\big),\\
y_t^{(k,\ell)} &= C_k h_t^{(k,\ell)},
\end{aligned}
\]
where $k$ indexes direction, $\ell$ indexes scan-line, and $f_t$ is a learned gate [2602.00904]. All directions are processed independently in parallel, preserving strict $\mathcal{O}(HW)$ complexity.

## 3. OSS Block Structure, Traversal Selection, and Fusion

The OSS block comprises a directional scan module and an efficient feature fusion scheme. Each directional scan outputs a set of features aligned to 2D pixel locations. After all directions are processed, a traversal selection (O-Attention) mechanism fuses the multi-directional context at each spatial location. Specifically, for each pixel $p$:
1. The outputs from all scanned directions $Y^{(k)}(p)$ are stacked into $X_\mathrm{stack}(p) \in \mathbb{R}^{C\times D}$, where $D$ is the number of directions.
2. Two $1\times1$ convolutions (or linear layers) compute scores $u^{(k)}(p)$, followed by a softmax normalization over directions, yielding attention weights $w^{(k)}(p)$.
3. The fused output is $Y(p) = \sum_{k=1}^D w^{(k)}(p) Y^{(k)}(p)$ [2602.00904].

In VmambaIR, additional channel-wise SSM scans are incorporated after spatial fusion, followed by a $1\times1$ convolutional projection [2403.11423].

Alongside the OSS module, the Efficient Feed-Forward Network (EFFN) operates on the output, comprising a $1\times1$ expansion, depthwise convolution, gated linear unit, and final $1\times1$ projection. This structure enables nonlinear and cross-channel mixing at low computational cost [2403.11423].

## 4. Computational Complexity and Efficiency

Unlike the quadratic complexity of transformer self-attention ($\mathcal{O}((HW)^2)$ for an $H\times W$ image), OSS’s total complexity is linear in the number of patches:
\[
\mathcal{O}(D\cdot N \cdot N_\text{state}),
\]
with $N=HW$ and $D$ constant (6 for VmambaIR, 8 for OCTOPUS). All components (scan, gating, O-Attention) scale as $\mathcal{O}(N)$ [2403.11423][2602.00904]. Empirically, OSS in VmambaIR reported only a $\sim7\%$ FLOP increase over single-direction SSM while substantially expanding the model’s 2D and channel context [2403.11423].

## 5. Architectural Integration and Practical Deployment

OSS blocks are modular and readily integrated into hierarchical architectures. In VmambaIR, a four-stage U-Net variant is used:
- Encoder: sequential OSS blocks at progressively reduced spatial resolutions.
- Decoder: upsampling, additional OSS blocks, and skip concatenations.
- Refinement: multiple OSS blocks at full resolution followed by a pixel-shuffle or convolutional output module, depending on the task [2403.11423].

In OCTOPUS, OSS is the foundational layer for vision SSMs, replacing standard raster-scan or unidirectional recurrence with true multi-directional propagation. Traversal selection is key to adaptively fusing the multi-orientation outputs at each pixel [2602.00904].

## 6. Empirical Performance and Analysis

OSS enables state-of-the-art results in both image restoration and semantic segmentation:
- VmambaIR achieves 29.99 dB (Urban100, 4× SR), outperforming BebyGAN (29.19 dB), with LPIPS 0.0496 vs 0.0529, and demonstrates significant efficiency gains: 27.06 dB (NTIRE2020, 4× real SR) using 10.5 M parameters and 20.5 G FLOPs, compared to MM-RealSR’s 25.19 dB/26.13 M/78.6 G [2403.11423].
- On Rain100H deraining, VmambaIR attains 31.66 dB/0.909 SSIM, exceeding Restormer’s 31.46 dB/0.904 with lower computational cost [2403.11423].
- Ablations confirm the importance of both planar and channel-wise OSS scanning; removing planar or channel scanning reduces PSNR by 0.43 dB and 0.14 dB, respectively [2403.11423].
- OCTOPUS demonstrates substantial improvements on segmentation (ADE20K single-scale mIoU: 37.93% for Octopus-T vs 22.77% for VMamba-T), cleaner object boundaries, and improved region consistency. Classification accuracy on miniImageNet also increases compared to previous vision SSMs (Octopus-T: 86.60% Top-1 vs 85.82% for VMamba-T) [2602.00904].

An analysis of the effective receptive field in OCTOPUS indicates the emergence of isotropic, eight-spoked coverage, superior to the window-based localities of Swin transformer and anisotropy of VMamba, reflecting OSS’s enhancement of 2D spatial awareness [2602.00904].

## 7. Significance and Perspectives

By overcoming the causality and locality constraints of standard SSMs, OSS establishes a path for scalable, spatially-aware, and efficient vision architectures. Its ability to tightly couple global context modeling and local spatial coherence, while maintaining strict linear complexity and plug-and-play architectural integration, positions OSS as a foundational operator for next-generation visual SSMs. The demonstrated empirical gains in restoration and segmentation, together with interpretability through effective receptive field analyses, underscore OSS’s impact in both theoretical modeling and practical system performance [2403.11423][2602.00904].

| Aspect                        | VmambaIR (6 directions)        | OCTOPUS (8 directions)              |
|-------------------------------|-------------------------------|-------------------------------------|
| Spatial scan directions       | H/W ±, Channels ±              | All axes ±, diagonals ±             |
| Fusion mechanism              | Addition and projection         | Traversal selection (O-Attention)   |
| Core SSM type                 | Mamba                          | Mamba                               |
| Complexity per pass           | $\mathcal{O}(HW)$              | $\mathcal{O}(8HW)$                  |
| Empirical improvement         | SR/Derain SOTA, efficient      | Segmentation/classification boost   |

Source: https://www.emergentmind.com/topics/omni-selective-scan-oss