MambaOcc: Efficient BEV Occupancy Prediction

• MambaOcc is a visual occupancy prediction framework that delivers fine-grained 3D semantic mapping using BEV representations, linear state-space models, and a local adaptive reordering mechanism.
• It leverages innovations such as BEV-centric design, Mamba SSMs, and LAR to reduce memory bottlenecks and quadratic compute complexity, leading to improved performance over transformer-based approaches.
• Empirical results demonstrate that MambaOcc achieves state-of-the-art mIoU while reducing parameters by up to 42% and FLOPs by 39% compared to traditional deep learning baselines.

MambaOcc is a visual occupancy prediction framework for bird’s-eye-view (BEV) scene understanding, designed to deliver fine-grained 3D semantic and geometric interpretation of driving environments with strong computational efficiency. It addresses the primary challenges of voxel-based dense occupancy prediction—memory bottlenecks and quadratic compute scaling—by leveraging BEV representations, Mamba linear state-space models (SSM), and a novel local adaptive reordering mechanism. MambaOcc demonstrates state-of-the-art performance on large-scale autonomous driving datasets, offering a substantial reduction in both parameter count and floating-point operations compared to transformer- and heavy 3D CNN-based baselines (Tian et al., 2024).

1. Problem Formulation and Motivation

Occupancy prediction for autonomous driving infers, for each voxel within a 3D scene, whether it is occupied and, frequently, its semantic class. Unlike bounding box detection or per-pixel segmentation, occupancy prediction yields dense volumetric or BEV grids, critical for representing open-world geometry and complex traffic participants. However, explicit 3D voxelization quickly incurs prohibitive memory and FLOP costs, and standard transformer-based fusion models scale quadratically with the number of voxels, making real-time deployment infeasible for fine-grained grids (Tian et al., 2024).

MambaOcc mitigates these obstacles with:

• BEV-centric representation, collapsing height into feature channels to operate on a 2D spatial grid, thus dramatically reducing sequence length.
• Mamba S6-based linear attention, replacing quadratic transformer calculations with $O(N)$ complexity state-space modeling while preserving long-range context.

2. Mamba State-Space Modeling and Linear Attention

Central to MambaOcc is the use of Mamba-style state-space sequence modeling, which provides an efficient alternative to softmax-based attention. Attention is formulated as: $$\mathrm{Attention}(Q,K,V) = \Phi(Q) \, (\Phi(K)^\top V)$$ for $Q, K, V \in \mathbb{R}^{N \times d}$ (query, key, value) and a feature map $\Phi:\mathbb{R}^d \rightarrow \mathbb{R}^{d'}$ such that

$$\Phi(Q)\,\Phi(K)^{\!\top} \approx \mathrm{softmax}(Q K^\top)$$

without explicitly instantiating the $N \times N$ attention matrix. In MambaOcc, the projection uses $1 \times 1$ convolutions, and the typical nonlinearity is $\mathrm{ELU}(x) + 1$ applied element-wise.

This mechanism is underpinned by an SSM recurrence: $$h_t = \bar{A} h_{t-1} + \bar{B} x_t,\quad y_t = C h_t$$ where the update matrices are parameterized as

$$\bar{A} = \exp(\Delta A),\quad \bar{B} = (\Delta A)^{-1}(\exp(\Delta A)-I)\Delta B,\quad B=s_B(x_t),\quad C=s_C(x_t),\quad \Delta = \tau_\Delta (W_\Delta+s_\Delta(x_t))$$

with $$\mathrm{Attention}(Q,K,V) = \Phi(Q) \, (\Phi(K)^\top V)$$0 as learnable projections. This discretized continuous-time SSM allows fast, large-context processing for BEV sequences with linear complexity.

3. Local Adaptive Reordering (LAR) Mechanism

Mamba SSM layers are sensitive to token sequence order. A fixed raster scan over BEV grids can compromise adaptation to local scene structure, since spatially adjacent but semantically linked elements may be far apart in sequence space.

To address this, MambaOcc introduces the Local Adaptive Reordering (LAR) module. For each BEV grid position $$\mathrm{Attention}(Q,K,V) = \Phi(Q) \, (\Phi(K)^\top V)$$1, a local offset $$\mathrm{Attention}(Q,K,V) = \Phi(Q) \, (\Phi(K)^\top V)$$2 is learned via a deformable convolutional layer: $$\mathrm{Attention}(Q,K,V) = \Phi(Q) \, (\Phi(K)^\top V)$$3 yielding a pseudo-permutation

$$\mathrm{Attention}(Q,K,V) = \Phi(Q) \, (\Phi(K)^\top V)$$4

for the token sequence. Multiple original locations can map to a single output location, with aggregation over pre-images $$\mathrm{Attention}(Q,K,V) = \Phi(Q) \, (\Phi(K)^\top V)$$5 performed by a local attention: $$\mathrm{Attention}(Q,K,V) = \Phi(Q) \, (\Phi(K)^\top V)$$6 for positional embeddings $$\mathrm{Attention}(Q,K,V) = \Phi(Q) \, (\Phi(K)^\top V)$$7. Since the deformable offsets and fusion weights are jointly optimized, LAR dynamically reorders the input to group semantically related tokens together, improving SSM-based modeling of local context.

4. Hybrid Encoder Architecture

The information processing flow in MambaOcc consists of:

• Multi-view images are encoded with a VMamba backbone and lifted to BEV features using depth-aware LSS pooling (optionally fusing temporally).
• The BEV encoder applies three stages, each composed of:
  • A LAR group for learned local reordering and aggregation,
  • An SS2D (Mamba S6) group for linear global context fusion,
  • Optionally, a $$\mathrm{Attention}(Q,K,V) = \Phi(Q) \, (\Phi(K)^\top V)$$8 pointwise convolution for channel mixing.

Empirical studies highlight the efficacy of this hybrid design. Replacing an SS2D block with a LAR block yields a +0.48 mIoU gain over pure CNN–SS2D stacks, and +0.69 mIoU over dual SS2D. Channel widths scale as [128, 256, 512] for the base and [256, 512, 1024] for MambaOcc-Large (Tian et al., 2024).

5. Computational Efficiency

MambaOcc attains high accuracy at substantially reduced resource usage, primarily due to its BEV-centric design and the replacement of quadratic attention and expensive 3D convolutions with linear-complexity Mamba modules. A comparison with FlashOcc and PanoOcc reveals:

Method	FLOPs (G)	Params (M)	mIoU
FlashOcc	1467.5	137.1	43.3
MambaOcc	893.8	79.5	43.4
MambaOcc-Large	1002	119	44.1
PanoOcc	–	–	–

Relative to FlashOcc, base MambaOcc reduces parameter count by 42% and computational operations by 39%, while MambaOcc-Large improves mIoU by 0.8 points with a 14% parameter reduction (Tian et al., 2024).

6. Empirical Results

On the Occ3D-nuScenes dataset (17 classes, 0.4m voxels, 700/150 splits), MambaOcc achieves strong results:

• Base: 43.4 mIoU, superior to FlashOcc’s 43.3 with lower compute.
• Large: 44.1 mIoU, a 0.8 mIoU gain on FlashOcc.
• VMamba backbone alone improves mIoU by +3.96 over ResNet-50.
• Adding LAR-SS2D: +1.12 mIoU; positional encoding: +0.13.
• Pure SS2D vs. CNN-SS2D vs. LAR-SS2D: 34.72, 34.93, 35.41 mIoU respectively.

Larger many-to-one LAR kernels (3×3, 5×5) further improve aggregation, with temporal (4D) fusion boosting base results by +4.7 mIoU (35.41 → 39.78) (Tian et al., 2024).

7. Extensions, Limitations, and Future Research

MambaOcc is the first BEV-based occupancy prediction model to combine linear Mamba SSMs with a learnable token reordering mechanism. This facilitates real-time, high-fidelity occupancy mapping with substantially reduced computational expense. Open problems identified for future research include fully bidirectional Mamba modeling in BEV space, joint depth-adaptive reordering for improved coupling of geometric and sequential context, and integration of LAR within end-to-end planning pipelines for autonomous navigation (Tian et al., 2024).