VeloxSeg: Lightweight 3D Segmentation

• The paper introduces VeloxSeg, a lightweight 3D segmentation framework that merges dual-stream CNN and Transformer architectures with Paired Window Attention, JL-guided convolution, and Spatially Decoupled Knowledge Transfer.
• VeloxSeg is a segmentation method designed for high-dimensional medical images that effectively balances efficient computation with robust handling of complex anatomical structures and heterogeneous modalities.
• Empirical results on datasets such as AutoPET-II and BraTS2021 demonstrate improved Dice scores and significant reductions in parameters and GFLOPs compared to conventional heavy-weight segmentation models.

Searching arXiv for the cited papers to ground the article and confirm identifiers. VeloxSeg is a lightweight 3D medical image segmentation framework introduced as a response to the “efficiency / robustness conflict” in volumetric segmentation: aggressive reductions in parameters and FLOPs often degrade robustness on complex anatomical structures and heterogeneous modalities. The method combines a dual-stream CNN–Transformer encoder–decoder, Paired Window Attention (PWA), Johnson–Lindenstrauss lemma-guided convolution (JLC), and Spatially Decoupled Knowledge Transfer (SDKT) via Gram matrices. Its stated design goal is to preserve robust representation learning under stringent computational budgets while supporting multimodal inputs such as PET/CT and multi-sequence MRI (Lu et al., 26 Sep 2025).

1. Problem formulation and scope

VeloxSeg addresses lightweight 3D medical segmentation under conditions where volumetric inputs are high-dimensional, anatomical and lesion morphology are complex, and modalities are heterogeneous. The reported problem setting includes whole-body PET/CT, head-and-neck PET/CT, and four-sequence brain MRI, all of which require both long-range contextual reasoning and fine local delineation (Lu et al., 26 Sep 2025).

The framework is motivated by several specific limitations of prior lightweight designs. Depthwise separable convolutions reduce compute but may decouple channels so aggressively that semantic and geometric adjacency in feature space becomes fragile. Windowed or axial attention reduces the cost of volumetric self-attention, but can sacrifice either global context or local fidelity. Pruning-based lightweighting is described as dataset-specific, dependent on importance metrics and retraining, and prone to harming generalization. VeloxSeg is therefore positioned as a redesign of the segmentation backbone itself rather than a post hoc compression pipeline (Lu et al., 26 Sep 2025).

The segmentation setting is explicitly multimodal. PET/CT is described as combining functional and structural signals with different regions of interest and different roles at different scales, while multi-sequence MRI combines T1, T1ce, T2, and FLAIR with distinct contrast behavior. Missing or noisy modalities in deployment are also identified as a practical concern. This motivates architectural separation between local fusion and cross-modal cooperation rather than a single undifferentiated encoder (Lu et al., 26 Sep 2025).

2. Architectural organization

VeloxSeg uses a 3D encoder–decoder architecture with two 4-stage encoders: a modal-fusion convolution encoder and a modal-cooperative Transformer encoder. A segmentation decoder merges multi-scale features from both streams and produces the final segmentation volume, while a separate self-supervised “texture teacher” network is used only during training for SDKT (Lu et al., 26 Sep 2025).

The architectural principle is termed “glance-and-focus.” In this decomposition, the Transformer stream provides the “glance” function by using PWA to retrieve multi-scale and long-range contextual information at low cost, including multimodal interaction. The CNN stream provides the “focus” function by using JLC blocks to extract robust local features, fuse modalities, and preserve spatial adjacency in feature space with minimal parameters. The decoder then combines contextual and local-detail information and progressively upsamples to full resolution (Lu et al., 26 Sep 2025).

The two streams are kept separate to avoid parameter growth as the number of modalities increases. Where fusion is required, a lightweight $1\times1\times1$ convolution serves as a modal mixer. The upsampling path is also optimized for efficiency: the reported final design replaces transposed convolution with a cheaper unified upsampling strategy to reduce GFLOPs and increase throughput. A plausible implication is that the architecture allocates computation preferentially to modules that expand effective receptive field or preserve local geometry, while minimizing expensive dense volumetric transformations.

3. Paired Window Attention

PWA is the Transformer-side primitive for efficient multi-scale and multimodal context aggregation. At stage $k$, for each modality $m$, encoder features are denoted

$$\mathbf{E}_m^k \in \mathbb{R}^{C^k \times H^k \times W^k \times D^k}, \quad m=1,\dots,M.$$

Queries, keys, and values are produced by point-wise convolutions and layer normalization: $$\mathbf{Q}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{K}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{V}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)).$$ Unlike standard self-attention, the projected channel dimension is chosen using a minimum head size $C_{\min}^k$ derived from JL-guided considerations and the number of heads and window pairs (Lu et al., 26 Sep 2025).

PWA defines ordered pairs of big and small windows,

$$\{ \mathrm{Win}_i^k \}_{i=1}^{N_{\text{win}^k}} = \{(B_i^k, S_i^k)\}_{i=1}^{N_{\text{win}^k}},$$

with synchronous expansion by rate $r$, typically $r=2$. Expansion continues until the largest big window matches the full stage feature size, so only $\log(\text{size})$ windows are required to span local to global coverage. Within each big window, 3D max-pooling with kernel and stride $k$0 yields a constant token length $k$1 across scales, enabling parallel attention computation over multiple scales and modalities (Lu et al., 26 Sep 2025).

Gathering concatenates sequences from all modalities and scales: $k$2 Attention is then computed jointly: $k$3 After scattering back into volumetric tensors, a $k$4 pointwise convolution fuses the attended features: $k$5 This yields grouped multimodal attention over sequences encoding multi-scale, multi-modal patches (Lu et al., 26 Sep 2025).

The reported complexity is

$k$6

which is described as near-linear in $k$7 for fixed window and channel sizes. The paper states that the linear coefficient is about $k$8 of Swin Transformer. Empirically, mean attention distance analysis shows lower stages emphasizing local attention and higher stages emphasizing longer-range dependencies, supporting the intended multi-scale behavior (Lu et al., 26 Sep 2025).

4. Johnson–Lindenstrauss lemma-guided convolution

JLC is the convolutional primitive used in the CNN stream. Its construction is motivated by the Johnson–Lindenstrauss lemma, stated in the paper as follows: for a finite set $k$9 with $m$0 and $m$1, there exists a linear map $m$2 with

$m$3

such that pairwise distances are approximately preserved (Lu et al., 26 Sep 2025).

The authors interpret each group of channels in grouped 3D convolution as a low-dimensional embedding space. Let $m$4 denote the number of samples needed to cover the segmentation-related manifold for an input patch from $m$5 modalities, given that each feature voxel aggregates information from $m$6 input voxels. To preserve spatial adjacency or geometry of features, each group should satisfy

$m$7

Depthwise convolution corresponds to $m$8, which is argued to be too small relative to $m$9, suggesting that depthwise factorization is particularly fragile in high-dimensional 3D medical settings (Lu et al., 26 Sep 2025).

Because exact $$\mathbf{E}_m^k \in \mathbb{R}^{C^k \times H^k \times W^k \times D^k}, \quad m=1,\dots,M.$$0 is unknown, VeloxSeg approximates

$$\mathbf{E}_m^k \in \mathbb{R}^{C^k \times H^k \times W^k \times D^k}, \quad m=1,\dots,M.$$1

where $$\mathbf{E}_m^k \in \mathbb{R}^{C^k \times H^k \times W^k \times D^k}, \quad m=1,\dots,M.$$2 encodes task difficulty and is calibrated on AutoPET-II. This yields the stagewise recommendation

$$\mathbf{E}_m^k \in \mathbb{R}^{C^k \times H^k \times W^k \times D^k}, \quad m=1,\dots,M.$$3

which is simplified through $$\mathbf{E}_m^k \in \mathbb{R}^{C^k \times H^k \times W^k \times D^k}, \quad m=1,\dots,M.$$4 to

$$\mathbf{E}_m^k \in \mathbb{R}^{C^k \times H^k \times W^k \times D^k}, \quad m=1,\dots,M.$$5

This sequence increases with depth and is presented as the JL-guided minimum sufficient group size for lightweight 3D segmentation (Lu et al., 26 Sep 2025).

The JLC block replaces full or depthwise 3D convolution with grouped 3D convolutions whose group size follows the JL-derived lower bound. It also uses parallel multi-scale kernels $$\mathbf{E}_m^k \in \mathbb{R}^{C^k \times H^k \times W^k \times D^k}, \quad m=1,\dots,M.$$6 in 3D, i.e. $$\mathbf{E}_m^k \in \mathbb{R}^{C^k \times H^k \times W^k \times D^k}, \quad m=1,\dots,M.$$7, followed by aggregation and optional pointwise mixing. In the convolution encoder, three JLC branches operate in parallel per stage. Reported ablations show that multi-kernel JLC improves Dice by $$\mathbf{E}_m^k \in \mathbb{R}^{C^k \times H^k \times W^k \times D^k}, \quad m=1,\dots,M.$$8 while reducing parameters and FLOPs from $$\mathbf{E}_m^k \in \mathbb{R}^{C^k \times H^k \times W^k \times D^k}, \quad m=1,\dots,M.$$9M/$$\mathbf{Q}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{K}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{V}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)).$$0G to $$\mathbf{Q}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{K}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{V}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)).$$1M/$$\mathbf{Q}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{K}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{V}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)).$$2G in a pure CNN variant (Lu et al., 26 Sep 2025).

The empirical case for JLC is central to VeloxSeg. On Hecktor2022 and BraTS2021, JLC outperforms depthwise by $$\mathbf{Q}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{K}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{V}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)).$$3 and $$\mathbf{Q}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{K}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{V}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)).$$4 Dice, respectively, with only approximately $$\mathbf{Q}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{K}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{V}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)).$$5M extra parameters. On Hecktor2022, it reportedly surpasses full convolution with $$\mathbf{Q}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{K}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{V}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)).$$6M fewer parameters. In a BraTS$$\mathbf{Q}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{K}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{V}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)).$$7MSD generalization setting, an $$\mathbf{Q}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{K}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{V}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)).$$8-pruned model with $$\mathbf{Q}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{K}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)),\quad \mathbf{V}_m^k = \mathrm{PWC}(\mathrm{LN}(\mathbf{E}_m^k)).$$9M parameters achieves $C_{\min}^k$0 Dice, whereas JLC with $C_{\min}^k$1M parameters achieves $C_{\min}^k$2 Dice, supporting the claim that JL-guided grouping improves robustness rather than only efficiency (Lu et al., 26 Sep 2025).

5. Multimodal interaction and Spatially Decoupled Knowledge Transfer

VeloxSeg is explicitly designed for heterogeneous modalities. In the CNN stream, stacked modalities are fused locally by JLC. In the Transformer stream, PWA performs modal cooperation by concatenating tokens from all modalities during paired-window gathering and computing attention in the joint token space. The total number of PWA parameters is described as almost independent of the number of modalities $C_{\min}^k$3, aside from minor growth due to input channel dimension (Lu et al., 26 Sep 2025).

An ablation on AutoPET-II distinguishes early fusion from explicit modal interaction. CT only reaches $C_{\min}^k$4 Dice and PET only reaches $C_{\min}^k$5. Early fusion of PET+CT reaches $C_{\min}^k$6, while VeloxSeg multi-modal PWA using $C_{\min}^k$7 reaches $C_{\min}^k$8, a reported $C_{\min}^k$9 Dice gain over early fusion with only $$\{ \mathrm{Win}_i^k \}_{i=1}^{N_{\text{win}^k}} = \{(B_i^k, S_i^k)\}_{i=1}^{N_{\text{win}^k}},$$0M parameters and $$\{ \mathrm{Win}_i^k \}_{i=1}^{N_{\text{win}^k}} = \{(B_i^k, S_i^k)\}_{i=1}^{N_{\text{win}^k}},$$1 GFLOPs. This suggests that the model benefits not merely from combined modality content, but from multi-scale cross-modal alignment performed inside attention (Lu et al., 26 Sep 2025).

SDKT is the training-only mechanism used to inject texture priors from self-supervised reconstruction. For each modality $$\{ \mathrm{Win}_i^k \}_{i=1}^{N_{\text{win}^k}} = \{(B_i^k, S_i^k)\}_{i=1}^{N_{\text{win}^k}},$$2, a self-supervised reconstruction teacher $$\{ \mathrm{Win}_i^k \}_{i=1}^{N_{\text{win}^k}} = \{(B_i^k, S_i^k)\}_{i=1}^{N_{\text{win}^k}},$$3 produces decoder feature maps $$\{ \mathrm{Win}_i^k \}_{i=1}^{N_{\text{win}^k}} = \{(B_i^k, S_i^k)\}_{i=1}^{N_{\text{win}^k}},$$4. For a feature map $$\{ \mathrm{Win}_i^k \}_{i=1}^{N_{\text{win}^k}} = \{(B_i^k, S_i^k)\}_{i=1}^{N_{\text{win}^k}},$$5, the Gram matrix is defined as

$$\{ \mathrm{Win}_i^k \}_{i=1}^{N_{\text{win}^k}} = \{(B_i^k, S_i^k)\}_{i=1}^{N_{\text{win}^k}},$$6

VeloxSeg then minimizes

$$\{ \mathrm{Win}_i^k \}_{i=1}^{N_{\text{win}^k}} = \{(B_i^k, S_i^k)\}_{i=1}^{N_{\text{win}^k}},$$7

The paper states that this is mathematically equivalent to minimizing an MMD with a second-order polynomial kernel. Because Gram matrices encode channel-channel correlations rather than exact spatial layout, the method is intended to transfer texture and style statistics without forcing the segmentation network to reconstruct irrelevant spatial regions (Lu et al., 26 Sep 2025).

The final training loss is

$$\{ \mathrm{Win}_i^k \}_{i=1}^{N_{\text{win}^k}} = \{(B_i^k, S_i^k)\}_{i=1}^{N_{\text{win}^k}},$$8

with $$\{ \mathrm{Win}_i^k \}_{i=1}^{N_{\text{win}^k}} = \{(B_i^k, S_i^k)\}_{i=1}^{N_{\text{win}^k}},$$9 and $r$0. In comparison with other transfer objectives on AutoPET-II, the paper reports: $r$1 Dice without SDKT; $r$2 Dice with $r$3 feature loss; $r$4 Dice with affinity loss; $r$5 Dice with shared-ROI loss; and $r$6 Dice with SDKT. The text characterizes SDKT as the only approach providing consistent positive transfer in this setup (Lu et al., 26 Sep 2025).

6. Experimental evaluation

VeloxSeg is evaluated on four public datasets: AutoPET-II, Hecktor2022, BraTS2021, and MSD2019 Task-01. AutoPET-II contains 1014 FDG PET/CT scans with patch size $r$7. Hecktor2022 contains 524 PET/CT scans with patch size $r$8. BraTS2021 uses four MRI sequences registered to $r$9 and cropped to $r=2$0 patches. Images are normalized, standardized, and split $r=2$1 into train, validation, and test. Training uses PyTorch 2.4.1, AdamW with learning rate $r=2$2, weight decay $r=2$3, linear warmup plus cosine annealing, batch size $r=2$4 with balanced positive:negative patches $r=2$5, and $r=2$6 epochs. Data augmentation consists of random flipping along the z-axis with probability $r=2$7 (Lu et al., 26 Sep 2025).

The main quantitative results reported for PET/CT are summarized below.

Dataset	Best baseline cited in paper	VeloxSeg
AutoPET-II	VSmTrans: 62.46% Dice	62.51% Dice
Hecktor2022	U-KAN: 55.89% Dice	56.48% Dice

On AutoPET-II, VeloxSeg is reported to use only $r=2$8 of VSmTrans parameters and $r=2$9 of its GFLOPs. Under the nnU-Net training pipeline, the contrast is much stronger: on AutoPET-II, nnUNet achieves $\log(\text{size})$0 Dice with $\log(\text{size})$1M parameters and $\log(\text{size})$2 GFLOPs, while VeloxSeg reaches $\log(\text{size})$3 Dice with $\log(\text{size})$4M parameters and $\log(\text{size})$5 GFLOPs; GPU throughput is $\log(\text{size})$6 versus $\log(\text{size})$7 patches/s, and CPU throughput is $\log(\text{size})$8 versus $\log(\text{size})$9, corresponding to $k$00 GPU and $k$01 CPU speedups. On Hecktor2022 under the same framework, nnUNet reaches $k$02 Dice and VeloxSeg reaches $k$03 Dice with $k$04M parameters and $k$05 GFLOPs; GPU throughput is $k$06 versus $k$07, and CPU throughput is $k$08 versus $k$09 (Lu et al., 26 Sep 2025).

On BraTS2021, the early-fusion variant VeloxSeg-C reaches $k$10 average Dice over ET, TC, and WT, improving on the best cited baseline SuperLightNet at $k$11. It also reports HD95 of $k$12 mm. The corresponding computational profile is $k$13M parameters, $k$14 GFLOPs, GPU throughput $k$15 images/s, and CPU throughput $k$16 images/s (Lu et al., 26 Sep 2025).

Ablation studies attribute gains to each major component. A CNN-only JLC variant with width $k$17, kernels $k$18, and JL groups $k$19 reaches $k$20 Dice with $k$21M parameters and $k$22 GFLOPs. Adding PWA lifts performance to $k$23 Dice with shallow attention depth $k$24, while reducing FFN expansion to $k$25 further improves Dice to $k$26. Replacing transposed convolution with unified upsampling reduces FLOPs from $k$27G to $k$28G and increases GPU throughput from $k$29 to $k$30 patches/s. Adding SDKT then raises Dice from $k$31 to $k$32 (Lu et al., 26 Sep 2025).

7. Interpretation, relation to other “VeloxSeg”-style ideas, and limitations

Within the provided literature, the name “VeloxSeg” is used explicitly for the 3D medical framework above (Lu et al., 26 Sep 2025). Two contemporaneous papers use the term only interpretively: one describes a “VeloxSeg-style” paradigm for velocity-guided level-set evolution in amodal segmentation, derived from VELA, and another discusses how a “VeloxSeg-like” vehicle segmentation system might borrow ideas from prototype-guided SAM-based modeling (Li et al., 3 Aug 2025, Wang et al., 6 Aug 2025). These uses do not define a separate established method with the same technical content; rather, they treat “VeloxSeg” as shorthand for a segmentation approach emphasizing speed, robustness, or evolution-based reasoning. This suggests that the actual VeloxSeg paper is distinct in scope and domain, despite superficial overlap in nomenclature.

Conceptually, VeloxSeg’s contribution lies in combining three orthogonal strategies within a very small computational envelope. PWA allocates attention capacity to multi-scale and multimodal context with near-linear complexity. JLC provides a theory-guided lower bound on channel interactions for grouped 3D convolution, directly targeting the representational fragility of depthwise factorization in volumetric data. SDKT adds training-time texture priors without inference-time overhead. The reported results indicate that the model’s efficiency gains are not purchased at the expense of robustness, which is the central claim of the paper (Lu et al., 26 Sep 2025).

The limitations described are mostly implicit rather than extensive. Group size scaling $k$33 for JLC and minimum windows for PWA are tuned empirically on AutoPET-II and then reused across tasks, so some settings may benefit from retuning. Even with PWA, 3D attention retains non-trivial memory cost, so very large volumes may still require patching or sliding-window inference. Training self-supervised texture teachers adds cost and complexity, although no teacher is needed at inference. The authors suggest that the design can be extended to other 3D tasks such as detection, instance segmentation, registration, and radiomics modeling, and that PWA and JLC may also be useful in other high-dimensional domains such as natural videos, 3D point clouds, LiDAR, or hyperspectral imaging (Lu et al., 26 Sep 2025).