HybridTM: Transformer-Mamba 3D Segmentation

• HybridTM is a 3D semantic segmentation architecture that combines Transformer attention with Mamba state-space modeling in a unified hybrid layer.
• It employs an inner-layer hybrid strategy by placing attention (group size 1024) before Mamba (group size 4096) to preserve local details and capture long-range dependencies.
• The UNet-like encoder-decoder structure, with sequential processing of xCPE, attention, Mamba, and FFN, achieves state-of-the-art mIoU on benchmarks like ScanNet and nuScenes.

HybridTM is a 3D semantic segmentation architecture that combines Transformer attention and Mamba state-space modeling in a unified backbone. It is presented as the first hybrid architecture that integrates Transformer and Mamba for 3D semantic segmentation, and its defining design choice is the “Inner Layer Hybrid Strategy,” which places attention and Mamba inside the same Hybrid Layer rather than allocating separate stages to each operator. The architecture is motivated by a specific complementarity: attention preserves relative spatial relationships and fine-grained local features, whereas Mamba models long-range dependencies with linear complexity, which is especially relevant for large-scale, sparse, and irregular point clouds (Wang et al., 24 Jul 2025).

1. Problem setting and conceptual basis

HybridTM is designed for 3D semantic segmentation, where every point in a 3D point cloud is assigned a semantic class label. The paper frames the task around three properties of point clouds: they are large-scale, sparse and irregular, and they require both fine local geometry understanding and global contextual reasoning. This combination creates a tension between representational fidelity and scalable context modeling (Wang et al., 24 Jul 2025).

The paper’s motivation is explicitly comparative. Transformer-based 3D segmentation methods benefit from attention because it models pairwise relationships and preserves spatial detail, but global self-attention is limited by quadratic complexity, so prior methods typically apply attention only within local groups or windows. Mamba-based methods offer linear-complexity sequence processing and are therefore attractive for long-range modeling, but the paper argues that they struggle with feature representation in 3D, especially with fine-grained local features and relative spatial relationships in serialized sparse voxels. HybridTM addresses that tradeoff by placing attention before Mamba inside each block, so that Mamba receives locally enriched features rather than raw serialized tokens.

A plausible implication is that the architecture is not merely a hybrid in the sense of mixing two operators, but a hybridization strategy tailored to the failure modes of sparse 3D data: local relational preservation is treated as a prerequisite for effective long-range sequence modeling.

2. Backbone organization and data flow

HybridTM uses a UNet-like encoder-decoder architecture operating on voxelized point clouds. The overall network contains HybridTM encoders, HybridTM decoders, down-sampling modules, up-sampling modules, and a classification head. The encoder has five stages with layer counts $[2,2,2,6,2]$, and the decoder has four stages with layer counts $[2,2,2,2]$ (Wang et al., 24 Jul 2025).

Within the backbone, the basic computational unit is the Hybrid Layer. Each Hybrid Layer contains an xCPE module, an attention layer, a Mamba layer, and an FFN layer. The architecture therefore does not separate Transformer and Mamba at the stage level. Instead, both operators appear within every Hybrid Layer.

The information flow is sequential. The input point cloud is first voxelized. Features then pass through encoder stages composed of Hybrid Layers, are downsampled across scales, and are subsequently refined in decoder stages that also use Hybrid Layers before final prediction by the classification head. Inside each Hybrid Layer, the sequence is:

1. xCPE positional enhancement
2. attention on small groups
3. restoration to the original token order
4. bidirectional Mamba on larger groups
5. restoration again
6. FFN-based feature integration

This ordering is central to the method. The paper explicitly states that the design is not parallel, does not use explicit addition of separate attention and Mamba branches, and does not introduce learned gating or cross-attention between them.

3. Inner Layer Hybrid Strategy

The paper identifies the Inner Layer Hybrid Strategy as the key technical contribution. Rather than assigning attention to some stages and Mamba to others, HybridTM combines them inside each Hybrid Layer in the order

$$\text{xCPE} \rightarrow \text{Attention} \rightarrow \text{Mamba} \rightarrow \text{FFN}.$$

This is described as a finer-grained hybridization than the “outer hybrid” strategies discussed in relation to prior 2D hybrid models (Wang et al., 24 Jul 2025).

Attention operates first on small non-overlapping groups. Its role is to preserve relative spatial relationships and extract fine-grained local features. Mamba then operates on larger groups, using those improved features to model broader dependencies. The FFN is described as enabling “thorough integration of local and global information.” The paper’s argument is that this order matters: if Mamba is applied before local relational refinement, performance suffers because serialized point-cloud representations do not preserve spatial relations as directly as attention does.

The ablation study supports that claim. On the ScanNet validation set, the paper reports:

Hybrid strategy	mIoU
Outer strategy, Mamba before Attention	77.1
Outer strategy, Attention before Mamba	77.4
IL, Mamba before Attention	77.5
IL, Attention before Mamba	77.8

These values show two trends. First, attention before Mamba is better than the reverse. Second, inner-layer integration is better than outer-layer allocation. The table itself gives a difference of $77.8$ versus $77.5$ between the two IL orderings; this suggests that the ordering effect is real but smaller than the difference between inner-layer and outer-stage hybridization.

4. Mathematical formulation and implementation details

The paper denotes the input to a Hybrid Layer by

$F \in \mathbb{R}^{N \times C},$

where $N$ is the number of input voxels or tokens and $C$ is the feature dimension. Tokens are partitioned into groups using space-filling curves, following Point Transformer V3-style serialization (Wang et al., 24 Jul 2025).

For the attention stage, with local group size $L$, the paper gives

$$\begin{aligned} & F_g = \mathrm{Parition}(F, L), \ & F_{g}^{'} = F_{g} + \mathrm{MSA}(F_{g}), \ & F^{'} = \mathrm{Restore}(F_{g}^{'}, L). \end{aligned}$$

For the Mamba stage, with larger group size $[2,2,2,2]$0,

$[2,2,2,2]$1

The implementation constants reported in the paper are specific. Attention uses group size 1024 throughout the encoder and decoder, and Mamba uses group size 4096 throughout the network. Residual additions are explicit in both sublayers. The model also uses xCPE for positional enhancement, but the paper does not restate xCPE’s mathematical form, nor does it specify normalization type, FFN expansion ratio, activation, dropout, voxel size, sparse tensor format, or the detailed upsampling and downsampling operators.

Training follows Point Transformer V3 for loss and data augmentation. The reported settings are: AdamW, One-Cycle policy, batch size 12, and 4 NVIDIA RTX 3090 GPUs. Training epochs are 800 for ScanNet, 800 for ScanNet200, 3000 for S3DIS, and 50 for nuScenes.

A concise summary of the reported architectural and training parameters is useful:

Component	Reported setting
Encoder depths	$[2,2,2,2]$2
Decoder depths	$[2,2,2,2]$3
Attention group size	1024
Mamba group size	4096
Optimizer	AdamW
LR schedule	One-Cycle policy

5. Empirical performance and ablation evidence

The paper evaluates HybridTM on ScanNet, ScanNet200, S3DIS, and nuScenes, using mIoU as the main metric. It reports state-of-the-art results on ScanNet, ScanNet200, and nuScenes, while presenting S3DIS as evidence of generalization rather than a best reported score (Wang et al., 24 Jul 2025).

Dataset	HybridTM mIoU	Comparison noted in paper
ScanNet	77.8	Point Transformer v3: 77.5
ScanNet200	36.5	Point Transformer v3: 35.2
nuScenes	80.9	Point Transformer v3: 80.2
S3DIS Area 5	72.1	Swin3D: 72.5

On ScanNet, HybridTM reaches 77.8% mIoU, improving over Point Transformer v3 by 0.3, Swin3D by 2.3, and Serialized Point Mamba by 1.0. On ScanNet200, it reaches 36.5% mIoU, improving over Point Transformer v3 by 1.3 and OctFormer by 3.9. On nuScenes, it reaches 80.9% mIoU, improving over Point Transformer v3 by 0.7 and SphereFormer by 1.4. On S3DIS Area 5, it reaches 72.1% mIoU, which is below Swin3D’s 72.5, but above Serialized Point Mamba by 1.5.

The paper’s operator-level ablation on ScanNet is equally direct:

Configuration	mIoU
Attention only	77.1
Mamba only	76.9
Attention + Mamba with IL	77.8

These results support the paper’s central complementarity claim. Attention-only exceeds Mamba-only by 0.2, while the combined IL design improves by 0.7 over attention-only and 0.9 over Mamba-only. The paper interprets the Mamba-only deficit as feature degradation in 3D feature extraction.

The paper also reports qualitative results, claiming more precise segmentation for large objects on ScanNet, better segmentation of small objects on nuScenes, and a better balance between local detail and broad context. By contrast, it does not report explicit runtime, FLOPs, memory, parameter count, throughput, or scaling curves. The efficiency claim therefore remains qualitative: attention has quadratic complexity, Mamba has linear complexity, and the architecture is intended to exploit that complementarity without formal efficiency tables.

6. Position in the literature, scope, and limitations

HybridTM is positioned against three families of methods. The first is Transformer-based 3D segmentation, including PCT, Point Transformer, PatchFormer, Point Transformer v2, Point Transformer v3, Swin3D, Fast Point Transformer, Stratified Transformer, OctFormer, and SphereFormer. The second is Mamba-based 3D segmentation, including Point Cloud Mamba, Point Mamba, LION-Mamba, and Serialized Point Mamba. The third is hybrid Transformer-Mamba modeling in NLP and 2D vision, represented by Jamba, MambaVision, MAP, and MaskMamba (Wang et al., 24 Jul 2025).

The paper’s distinction from those hybrid designs is precise. It claims that prior hybrid Transformer-Mamba designs mainly use outer hybrid strategies, whereas HybridTM introduces block-level inner-layer integration for sparse, irregular, serialized 3D voxels. This suggests that the contribution is not only the use of both operators, but the claim that 3D segmentation requires a different granularity of hybridization from dense 2D settings.

The paper does not provide a dedicated limitations section. It also does not state detailed future-work directions beyond the general implication that hybrid architectures are promising for 3D semantic segmentation. Several constraints are nonetheless explicit. The method omits many implementation details needed for exact replication, including normalization type, hidden widths, voxel size, FFN expansion ratio, and classification-head design. It does not ablate the number of hybrid layers, alternative group sizes, alternative FFN designs, or parameter/FLOP trade-offs. It does not provide formal efficiency measurements. These omissions matter because HybridTM is motivated partly by the complexity contrast between attention and Mamba, yet the paper substantiates that contrast only qualitatively.

Within those boundaries, HybridTM is best understood as a task-specific hybrid architecture for 3D semantic segmentation whose defining principle is sequential inner-layer integration: local relational modeling by attention first, broad dependency modeling by bidirectional Mamba second, and feature integration through an FFN. The empirical record reported in the paper supports that design on ScanNet, ScanNet200, and nuScenes, while the ablations indicate that neither operator alone reproduces the full effect of the hybrid configuration.