LaSSM: Efficient 3D Instance Segmentation

• LaSSM is a query-based 3D instance segmentation method that constructs queries from superpoints and refines them using state space models.
• It employs hierarchical semantic-spatial query initialization to balance semantic discriminativeness and spatial coverage in voxelized point clouds.
• The framework achieves state-of-the-art performance on indoor benchmarks while reducing computational complexity compared to attention-based methods.

LaSSM most commonly denotes a query-based 3D instance segmentation framework for point clouds that is designed to be both simple and highly efficient while still reaching or surpassing state of the art on challenging large-scale indoor benchmarks (Yao et al., 11 Feb 2026). In that usage, it addresses two core issues in current query-based 3D scene instance segmentation: query initialization in sparse point clouds and efficient query refinement without quadratic-cost attention over many queries and points. The acronym also appears in adjacent multimodal literature in two potentially confusing ways: the similarly spelled LLaSM refers to a “Large Language and Speech Model” for speech-and-text instruction following (Shu et al., 2023), and “LaSSM” is used informally in one paper to denote a state-space Large Audio LLM (Bhati et al., 2024). This polysemy makes disambiguation necessary in technical discussion.

1. Nomenclature and disambiguation

The primary arXiv paper whose title explicitly uses LaSSM is “LaSSM: Efficient Semantic-Spatial Query Decoding via Local Aggregation and State Space Models for 3D Instance Segmentation” (Yao et al., 11 Feb 2026). In that work, LaSSM is a 3D scene instance segmentation framework operating on voxelized point clouds, superpoints, and query decoders.

Two neighboring usages require explicit separation. First, LLaSM—with a double “L” as in “LLaMA”—stands for Large Language and Speech Model and is a speech–language multimodal LLM designed to follow speech-and-text instructions (Shu et al., 2023). The source explicitly states that the spelling “LaSSM” does not appear in that paper and that the model is consistently named LLaSM. Second, “State-Space Large Audio LLMs” introduces the first fully state-space LALM and notes that one can denote it informally as a LaSSM, but the paper’s formal terms are Hybrid-LALM and ssLALM rather than a canonical model name “LaSSM” (Bhati et al., 2024).

Term	Meaning	Primary domain
LaSSM	Efficient semantic-spatial query decoding for 3D instance segmentation	3D point clouds
LLaSM	Large Language and Speech Model	Speech + text
ssLALM / informal “LaSSM”	State-space large audio LLM	Audio + language

A common misconception is therefore to treat all three as variants of the same model family. The available evidence indicates instead that they belong to different research lines: query-based 3D scene instance segmentation, speech–language instruction following, and audio–language modeling with state-space backbones.

2. Problem setting and design motivation

In the 3D vision usage, LaSSM is situated within query-based 3D scene instance segmentation from point clouds, where the goal is to predict, for each object instance in a scene, a binary mask over points or voxels and a semantic label for that instance (Yao et al., 11 Feb 2026). Query-based methods follow the DETR paradigm: a fixed set of queries is fed into a transformer-like decoder, which interacts with scene features and outputs a set of instance predictions.

The LaSSM paper identifies two bottlenecks. The first is the query initialization dilemma in sparse 3D data. Geometry-based strategies such as farthest point sampling provide coverage without discriminativeness, while semantic-confidence selection provides discriminativeness without coverage. Learnable query embeddings also ignore scene content and are the same for all scenes. The second bottleneck is the computational burden of attention-based decoders. In 3D, hundreds of queries, multiple decoder layers, and large scene features make cross-attention and self-attention expensive; cross-attention scales roughly as $O(q \times n)$ and self-attention as $O(q^2)$ (Yao et al., 11 Feb 2026).

LaSSM is designed to resolve these issues by constructing queries directly from scene-adaptive superpoints, balancing semantic confidence and spatial distribution, and by replacing heavy attention with a more efficient structure based on state space models and local aggregation (Yao et al., 11 Feb 2026). The design goals are to resolve the initialization dilemma, replace heavy attention with a more efficient structure, explicitly integrate coordinates into query refinement, and stay simple and practical while delivering state-of-the-art performance, particularly on large, high-resolution indoor scenes.

A plausible implication is that LaSSM should be read less as a purely architectural novelty in decoder design and more as a coordinated proposal about where queries come from, how they are spatially updated, and how refinement complexity is controlled.

3. Core architecture

LaSSM is organized into three main parts: a Sparse 3D U-Net backbone, a hierarchical semantic-spatial query initializer, and a coordinate-guided SSM decoder (Yao et al., 11 Feb 2026). The input is a voxelized point cloud

$X \in \mathbb{R}^{n \times 6},$

where each voxel has coordinates $(x,y,z)$ and color $(r,g,b)$. The sparse 3D U-Net extracts voxel features

$F = Backbone(X) \in \mathbb{R}^{n \times d_o}.$

Superpoint pooling then groups voxels into superpoints based on geometry, using Felzenszwalb-Huttenlocher graph segmentation / superpoint pooling:

$F_s = SpPool(F), \quad C_s = SpPool([x,y,z]).$

Here, $F_s \in \mathbb{R}^{s \times d_o}$ are superpoint features and $C_s \in \mathbb{R}^{s \times 3}$ are their coordinates (Yao et al., 11 Feb 2026). These superpoints are the basis both for query initialization and for local aggregation in the decoder.

From these superpoint features, the initializer selects a subset of superpoints driven by semantic confidence and then applies farthest point sampling to ensure spatial coverage. It outputs query contents

$Q \in \mathbb{R}^{q \times d}$

and query coordinates

$O(q^2)$0

These queries are then refined by the coordinate-guided SSM decoder, which applies local aggregation, a spatial dual-path SSM block, and prediction heads for masks, classes, and center offsets (Yao et al., 11 Feb 2026).

The mask head $O(q^2)$1 generates a per-query mask embedding, with masks computed as

$O(q^2)$2

while the center offset head updates coordinates according to

$O(q^2)$3

Query refinement is repeated across $O(q^2)$4 layers, with $O(q^2)$5 in the experiments (Yao et al., 11 Feb 2026).

This architecture combines superpoint-level scene abstraction with query-level instance prediction. The paper positions this combination as the core novelty: a superpoint-based, semantic-spatial query initializer, a local aggregation scheme, and a spatial dual-path SSM block (Yao et al., 11 Feb 2026).

4. Hierarchical semantic-spatial query initialization

The query initializer is intended to resolve the tension between semantic discriminativeness and spatial coverage. It operates on superpoints, which are small, geometrically coherent regions of the scene obtained via graph-based segmentation on the voxel grid (Yao et al., 11 Feb 2026).

LaSSM first applies a lightweight MLP classifier $O(q^2)$6 to superpoint features:

$O(q^2)$7

where $O(q^2)$8 is the number of instance classes and the extra dimension is background. After softmax, each superpoint receives a semantic activation score for being any non-background class:

$O(q^2)$9

This yields a scalar confidence of “being part of some instance” for each superpoint (Yao et al., 11 Feb 2026).

Instead of applying a fixed threshold, LaSSM keeps the top-$X \in \mathbb{R}^{n \times 6},$0 superpoints according to an adaptive ratio $X \in \mathbb{R}^{n \times 6},$1:

$X \in \mathbb{R}^{n \times 6},$2

This allows the number of semantic candidates to vary with scene complexity (Yao et al., 11 Feb 2026). To prevent those candidates from concentrating in a small region, farthest point sampling is then applied over the selected superpoints:

$X \in \mathbb{R}^{n \times 6},$3

and query coordinates are taken as

$X \in \mathbb{R}^{n \times 6},$4

The paper describes this as hierarchical semantic-spatial selection: semantic hierarchy through ranking and filtering by semantic activation, followed by spatial hierarchy through FPS among semantic candidates (Yao et al., 11 Feb 2026). Empirically, the authors report that query distributions demonstrate better coverage than semantic-only or FPS-only methods, with fewer redundant queries on walls or background, and that training curves show faster convergence than FPS-based and purely semantic-based initialization, given the same decoder (Yao et al., 11 Feb 2026).

The initializer is therefore not merely a preprocessing step. The ablations indicate that it materially influences convergence speed, query placement, and final segmentation quality.

5. Coordinate-guided SSM decoding and local aggregation

The decoder replaces heavy attention with two coupled mechanisms: local aggregation and a spatial dual-path SSM block (Yao et al., 11 Feb 2026). The local aggregation module restricts each query’s interaction to a small neighborhood of superpoints defined by geometry, replacing global cross-attention from queries to all superpoints.

Given query contents $X \in \mathbb{R}^{n \times 6},$5 and coordinates $X \in \mathbb{R}^{n \times 6},$6, together with superpoint features and coordinates, the local aggregation proceeds by selecting $X \in \mathbb{R}^{n \times 6},$7 nearest superpoints for each query, expanding query features, learning importance weights, and aggregating element-wise query–neighbor interactions (Yao et al., 11 Feb 2026). For each query $X \in \mathbb{R}^{n \times 6},$8, the aggregated feature is

$X \in \mathbb{R}^{n \times 6},$9

followed by

$(x,y,z)$0

Because $(x,y,z)$1, the complexity per query is $(x,y,z)$2 and is independent of the total number of superpoints $(x,y,z)$3 (Yao et al., 11 Feb 2026).

The SSM component begins from a continuous-time linear state space model:

$(x,y,z)$4

which is then discretized by zero-order hold into

$(x,y,z)$5

The paper uses the Mamba-2 variant, which simplifies state space parameters and enables efficient sequence modeling (Yao et al., 11 Feb 2026).

Because queries do not have a natural ordering, LaSSM serializes them according to 3D Hilbert space-filling curves and a transposed Hilbert variant. Queries are sorted by those indices, processed by shared SSMs, rearranged back to the original order, averaged across both paths, and added residually:

$(x,y,z)$6

An FFN then produces

$(x,y,z)$7

The paper argues that this design yields linear self-interaction complexity in the number of queries rather than quadratic $(x,y,z)$8, while also using positional information explicitly through coordinate-based ordering rather than only through positional encodings (Yao et al., 11 Feb 2026). It additionally reports that dual-path Hilbert ordering performs better than single-path or Z-order, and that updating query coordinates after each layer is important for keeping the Hilbert ordering consistent with evolving instance centers (Yao et al., 11 Feb 2026).

A plausible implication is that LaSSM’s efficiency does not arise from replacing one global operator with another; it arises from a division of labor in which local geometry is handled by k-NN aggregation and longer-range query dependencies are handled by serialized SSMs.

6. Training objectives, empirical results, and efficiency profile

LaSSM is trained with a multi-task loss consisting of superpoint semantic activation supervision and instance-level losses for classification, mask prediction, and center regression across decoder layers (Yao et al., 11 Feb 2026). The semantic activation loss is

$(x,y,z)$9

where superpoint labels are obtained by majority voting within each superpoint. For instance prediction, LaSSM uses Hungarian matching with a cost combining BCE and Dice mask losses, classification cross-entropy, and $(r,g,b)$0 center regression. The total loss is

$(r,g,b)$1

with $(r,g,b)$2, $(r,g,b)$3, $(r,g,b)$4, and $(r,g,b)$5 (Yao et al., 11 Feb 2026).

The reported benchmark results establish LaSSM’s standing most clearly on ScanNet++ V2. The hybrid LaSSM smpro variant reaches validation mAP 29.1, AP$(r,g,b)$6 43.5, and AP$(r,g,b)$7 51.6, and test mAP 32.4, AP$(r,g,b)$8 48.0, and AP$(r,g,b)$9 54.8 (Yao et al., 11 Feb 2026). The paper states that the best previous method, SGIFormer, achieved test mAP 29.9, AP$F = Backbone(X) \in \mathbb{R}^{n \times d_o}.$0 45.7, and AP$F = Backbone(X) \in \mathbb{R}^{n \times d_o}.$1 54.4, so LaSSM improves by +2.5 mAP and +2.3 AP$F = Backbone(X) \in \mathbb{R}^{n \times d_o}.$2 on test with approximately 1/3 FLOPs (Yao et al., 11 Feb 2026). It further states that LaSSM ranks first place on the ScanNet++ V2 leaderboard at submission time.

The framework also reports competitive performance on ScanNet++ V1, ScanNet V2, ScanNet200, and S3DIS Area 5 (Yao et al., 11 Feb 2026). On ScanNet V2, for example, smpro LaSSM reaches validation mAP 58.4, AP$F = Backbone(X) \in \mathbb{R}^{n \times d_o}.$3 78.1, and AP$F = Backbone(X) \in \mathbb{R}^{n \times d_o}.$4 86.1, and test mAP 57.9. On ScanNet200, smpro LaSSM reports mAP 29.3, AP$F = Backbone(X) \in \mathbb{R}^{n \times d_o}.$5 39.2, and AP$F = Backbone(X) \in \mathbb{R}^{n \times d_o}.$6 44.5. On S3DIS Area 5, smpro LaSSM reports mAP 56.5 and AP$F = Backbone(X) \in \mathbb{R}^{n \times d_o}.$7 69.4 (Yao et al., 11 Feb 2026).

The efficiency analysis in the same paper compares LaSSM against attention-heavy alternatives. On ScanNet V2, LaSSM (smpro) reports AP$F = Backbone(X) \in \mathbb{R}^{n \times d_o}.$8 78.1 with 3.711G FLOPs, compared with SGIFormer at AP$F = Backbone(X) \in \mathbb{R}^{n \times d_o}.$9 78.4 and 4.456G FLOPs, and OneFormer3D at AP$F_s = SpPool(F), \quad C_s = SpPool([x,y,z]).$0 76.3 and 4.082G FLOPs (Yao et al., 11 Feb 2026). On ScanNet++ V2, SGIFormer reports AP$F_s = SpPool(F), \quad C_s = SpPool([x,y,z]).$1 41.1, 13.513G FLOPs, 15.96M parameters, and 355.86 ms, while LaSSM (smpro) reports AP$F_s = SpPool(F), \quad C_s = SpPool([x,y,z]).$2 43.5, 4.777G FLOPs, 18.38M parameters, and 379.94 ms (Yao et al., 11 Feb 2026). The paper additionally notes that OneFormer3D runs out of memory on ScanNet++ V2 in their setting, whereas LaSSM remains feasible.

Ablation studies report that the LaSSM initializer outperforms FPS-based and semantic-only initialization, that best around 0.8 is observed for the selection ratio $F_s = SpPool(F), \quad C_s = SpPool([x,y,z]).$3, that best at 8 is observed for the number of local neighbors $F_s = SpPool(F), \quad C_s = SpPool([x,y,z]).$4, that best performance at 400 queries is obtained, and that performance saturates at 6 layers (Yao et al., 11 Feb 2026). The paper also states that even with zero decoder layers, the initializer alone gives respectable AP$F_s = SpPool(F), \quad C_s = SpPool([x,y,z]).$5 71.4, highlighting its strength.

7. Related usages in multimodal audio and speech research

Outside 3D segmentation, the most directly confusable name is LLaSM, which stands for Large Language and Speech Model (Shu et al., 2023). LLaSM combines a frozen Whisper speech encoder, a trainable modal adaptor, and Chinese-LLAMA2-7B as the LLM backbone. Audio is represented by continuous embeddings rather than discrete speech tokens, inserted into a single unified sequence with text by replacing '<au_patch>' placeholders between '<au_start>' and '<au_end>' tokens (Shu et al., 2023). Training proceeds in two stages: modality adaptation on ASR-style data with the LLM frozen, and cross-modal instruction fine-tuning on LLaSM-Audio-Instructions, a released bilingual speech–text instruction-following dataset with 199k conversations, 508k total samples, 428k English samples, and 80k Chinese samples (Shu et al., 2023). The system’s generative modality is text rather than speech.

A separate but conceptually related usage appears in “State-Space Large Audio LLMs,” which introduces the first fully state-space LALM and states that one can denote it informally as a LaSSM (Bhati et al., 2024). In that work, a transformer-based audio encoder is replaced by DASS, a state-space audio encoder, and the transformer-based LLM is replaced by a Mamba-based state-space LLM-2.8B. The fully state-space system is called ssLALM, while the mixed system is called Hybrid-LALM (Bhati et al., 2024). The paper emphasizes linear-time sequence complexity for SSMs, reports 2.8B parameters total and ~43–62M trainable parameters for ssLALM, and shows competitive performance with transformer-based LALMs on close-ended audio tasks (Bhati et al., 2024).

These neighboring usages are relevant because they share state-space or multimodal design themes while belonging to different application domains. This suggests that the acronym “LaSSM” has become a point of lexical convergence rather than a single established family name.

8. Limitations, failure modes, and broader significance

The LaSSM 3D segmentation paper explicitly identifies several limitations (Yao et al., 11 Feb 2026). All scenes use the same fixed number of queries, 400, which the authors describe as suboptimal because small scenes may have fewer instances while large scenes may need more. Hilbert curve ordering and transposed variants add serialization overhead, and experiments are confined to indoor datasets such as ScanNet, S3DIS, and ScanNet++. The paper also presents failure cases in which a large blackboard with similar color and geometry to the wall is missed, and a door is over-segmented into multiple instances due to texture or structural ambiguities.

The paper’s stated future directions include adaptive query allocation, more direct ways to incorporate positional information with lower latency, extension to outdoor scenes, more advanced or specialized SSM designs for 3D, and application of LaSSM-style decoders to other query-based 3D tasks such as detection and panoptic segmentation (Yao et al., 11 Feb 2026). It also positions LaSSM at the intersection of query-based 3D instance segmentation, efficient decoder design, and state space models in vision and point clouds.

In the broader technical landscape, LaSSM is best understood as a response to the practical scaling problems of query decoders in sparse 3D scenes. Its significance lies not only in the reported first-place ScanNet++ V2 leaderboard result, but also in the specific decomposition of the segmentation problem into superpoint-based query construction, geometry-constrained local refinement, and coordinate-guided linear-time sequence modeling (Yao et al., 11 Feb 2026). A plausible implication is that later work may treat these components as modular design patterns even when not adopting the full LaSSM framework.