Compound-Scaled 3D ConvNeXt for Medical Segmentation

• The paper introduces MedNeXt blocks that use 3D depthwise convolutions with inverted bottlenecks to enhance volumetric segmentation performance.
• It systematically employs compound scaling by jointly adjusting depth, width, and receptive fields, optimizing outcomes on diverse CT and MR benchmarks.
• Micro-architectural enhancements like 3D Global Response Normalization and progressive kernel upscaling stabilize feature maps and boost segmentation accuracy.

Compound-scaled 3D ConvNeXt architectures represent a modern class of fully convolutional encoder–decoder neural networks for volumetric tasks, with a particular emphasis on medical image segmentation. These models systematically apply simultaneous scaling along depth, width, and spatial context axes—inspired by EfficientNet’s “compound scaling” principle—within a framework derived from ConvNeXt, itself a convolutional analog of Vision Transformers. Recent implementations, typified by the MedNeXt and MedNeXt-v2 backbones, integrate residual inverted bottleneck blocks, large 3D kernels, representation stabilization layers, and progressive upscaling strategies to achieve state-of-the-art segmentation outcomes across heterogeneous CT and MR benchmarks (Roy et al., 2023, Roy et al., 19 Dec 2025).

1. Network Architecture and Block Composition

At the core of compound-scaled 3D ConvNeXt is the MedNeXt block, an “inverted bottleneck” structure with strictly convolutional operations. For input $x \in \mathbb{R}^{C \times D \times H \times W}$, the block consists of:

$$\begin{aligned} u &= \mathrm{DWConv}_{k\times k\times k}(x) \ u' &= \mathrm{GN}(u) \ v &= \mathrm{Conv}_{1\times1\times1}^{C \rightarrow C \cdot R}(u') \ v' &= \mathrm{GELU}(v) \ w &= \mathrm{Conv}_{1\times1\times1}^{C \cdot R \rightarrow C}(v') \ y &= x + w \end{aligned}$$

Key attributes:

• Depthwise convolution with 3D kernel (typically $k=3$ or $5$), followed by GroupNorm (GN) for batch-agnostic normalization.
• Channel expansion/compression via $1\times1\times1$ convolutions, with expansion ratio $R$ dictating block width.
• Residual connection for improved gradient flow.

All encoder, bottleneck, upsampling, and decoder blocks in the UNet-like macro-architecture are formulated from these MedNeXt units (Roy et al., 2023).

2. Compound Scaling: Depth, Width, and Receptive Field

MedNeXt and MedNeXt-v2 operationalize compound scaling by jointly adjusting:

• Depth: Total number of blocks per stage (UNet-like hierarchy).
• Width: Channel expansion ratio $R$ or base channel count $C$.
• Kernel size (Receptive Field): Choice of $k=3$ or $5$ for the depthwise convolution.
• Context (Editor's term): Patch or crop size (e.g., $$\begin{aligned} u &= \mathrm{DWConv}_{k\times k\times k}(x) \ u' &= \mathrm{GN}(u) \ v &= \mathrm{Conv}_{1\times1\times1}^{C \rightarrow C \cdot R}(u') \ v' &= \mathrm{GELU}(v) \ w &= \mathrm{Conv}_{1\times1\times1}^{C \cdot R \rightarrow C}(v') \ y &= x + w \end{aligned}$$0 or $$\begin{aligned} u &= \mathrm{DWConv}_{k\times k\times k}(x) \ u' &= \mathrm{GN}(u) \ v &= \mathrm{Conv}_{1\times1\times1}^{C \rightarrow C \cdot R}(u') \ v' &= \mathrm{GELU}(v) \ w &= \mathrm{Conv}_{1\times1\times1}^{C \cdot R \rightarrow C}(v') \ y &= x + w \end{aligned}$$1 voxels).

MedNeXt quantizes compound scaling into discrete configurations (S, B, M, L) with corresponding block counts and widths:

Config.	Blocks per Stage	Expansion Ratio $$\begin{aligned} u &= \mathrm{DWConv}_{k\times k\times k}(x) \ u' &= \mathrm{GN}(u) \ v &= \mathrm{Conv}_{1\times1\times1}^{C \rightarrow C \cdot R}(u') \ v' &= \mathrm{GELU}(v) \ w &= \mathrm{Conv}_{1\times1\times1}^{C \cdot R \rightarrow C}(v') \ y &= x + w \end{aligned}$$2
S	$$\begin{aligned} u &= \mathrm{DWConv}_{k\times k\times k}(x) \ u' &= \mathrm{GN}(u) \ v &= \mathrm{Conv}_{1\times1\times1}^{C \rightarrow C \cdot R}(u') \ v' &= \mathrm{GELU}(v) \ w &= \mathrm{Conv}_{1\times1\times1}^{C \cdot R \rightarrow C}(v') \ y &= x + w \end{aligned}$$3	$$\begin{aligned} u &= \mathrm{DWConv}_{k\times k\times k}(x) \ u' &= \mathrm{GN}(u) \ v &= \mathrm{Conv}_{1\times1\times1}^{C \rightarrow C \cdot R}(u') \ v' &= \mathrm{GELU}(v) \ w &= \mathrm{Conv}_{1\times1\times1}^{C \cdot R \rightarrow C}(v') \ y &= x + w \end{aligned}$$4
B/M	$$\begin{aligned} u &= \mathrm{DWConv}_{k\times k\times k}(x) \ u' &= \mathrm{GN}(u) \ v &= \mathrm{Conv}_{1\times1\times1}^{C \rightarrow C \cdot R}(u') \ v' &= \mathrm{GELU}(v) \ w &= \mathrm{Conv}_{1\times1\times1}^{C \cdot R \rightarrow C}(v') \ y &= x + w \end{aligned}$$5, $$\begin{aligned} u &= \mathrm{DWConv}_{k\times k\times k}(x) \ u' &= \mathrm{GN}(u) \ v &= \mathrm{Conv}_{1\times1\times1}^{C \rightarrow C \cdot R}(u') \ v' &= \mathrm{GELU}(v) \ w &= \mathrm{Conv}_{1\times1\times1}^{C \cdot R \rightarrow C}(v') \ y &= x + w \end{aligned}$$6	$$\begin{aligned} u &= \mathrm{DWConv}_{k\times k\times k}(x) \ u' &= \mathrm{GN}(u) \ v &= \mathrm{Conv}_{1\times1\times1}^{C \rightarrow C \cdot R}(u') \ v' &= \mathrm{GELU}(v) \ w &= \mathrm{Conv}_{1\times1\times1}^{C \cdot R \rightarrow C}(v') \ y &= x + w \end{aligned}$$7, $$\begin{aligned} u &= \mathrm{DWConv}_{k\times k\times k}(x) \ u' &= \mathrm{GN}(u) \ v &= \mathrm{Conv}_{1\times1\times1}^{C \rightarrow C \cdot R}(u') \ v' &= \mathrm{GELU}(v) \ w &= \mathrm{Conv}_{1\times1\times1}^{C \cdot R \rightarrow C}(v') \ y &= x + w \end{aligned}$$8, $$\begin{aligned} u &= \mathrm{DWConv}_{k\times k\times k}(x) \ u' &= \mathrm{GN}(u) \ v &= \mathrm{Conv}_{1\times1\times1}^{C \rightarrow C \cdot R}(u') \ v' &= \mathrm{GELU}(v) \ w &= \mathrm{Conv}_{1\times1\times1}^{C \cdot R \rightarrow C}(v') \ y &= x + w \end{aligned}$$9
L	$k=3$0, $k=3$1, $k=3$2	$k=3$3, $k=3$4, $k=3$5

In MedNeXt-v2, “anchor” scaling factors are fixed at depth $k=3$6 (52 blocks), with width $k=3$7 and context $k=3$8 set to practical values for GPU scalability: width $k=3$9 (base or doubled channels), and context $5$0 (patches of $5$1 or $5$2) (Roy et al., 2023, Roy et al., 19 Dec 2025).

3. Micro-architectural Enhancements: 3D Global Response Normalization (GRN)

MedNeXt-v2 introduces 3D Global Response Normalization (GRN) to address feature-map collapse and stabilize learning. Placed after the channel-expansion and GELU nonlinearity in each block, GRN re-weights responses:

Given $5$3,

$5$4

where $5$5 are per-channel feature maps, $5$6 are learnable parameters.

Empirical ablation demonstrates a consistent 0.3–0.4 DSC gain across BTCV, AMOS, KiTS, ACDC datasets with GRN relative to vanilla ConvNeXt variants (Roy et al., 19 Dec 2025). This suggests that activation stabilization is necessary for deep compound-scaled 3D ConvNet performance.

4. Progressive Kernel Upscaling (UpKern)

Training large 3D-kernel models ($5$7) from scratch on limited data results in performance saturation. UpKern is introduced as a mechanism for iterative kernel upscaling:

• Start with a small-kernel model $5$8, fully trained.
• Initialize the weights of the larger kernel $5$9 model by trilinear interpolation of $1\times1\times1$0 kernel weights in the depthwise convolution.
• All other weights (pointwise convolutions and normalizations) are copied directly.

This strategy achieves higher segmentation accuracy than training from scratch or simply increasing the training schedule, and is essential for capitalizing on large-kernel capacity in medical datasets (Roy et al., 2023).

5. Residual Upsampling and Downsampling Blocks

MedNeXt subsumes spatial resampling (down-/up-sampling) within residual inverted bottleneck blocks:

• Downsampling: A stride-2 depthwise convolution is employed, with a parallel stride-2 $1\times1\times1$1 convolution for the skip connection.
• Upsampling: A stride-2 transposed depthwise convolution is used, with a stride-2 transposed $1\times1\times1$2 convolution as skip.

This design contrasts with naïve strided or transposed convolutions, yielding approximately 0.9 percentage point DSC gain over standard resampling (Roy et al., 2023). It uniformly preserves semantic richness and gradient flow at resolution boundaries.

6. Model Variants, Training Protocols, and Downstream Performance

MedNeXt-v2 models maintain a constant depth (52 blocks) but vary width and context:

Variant	Depth (Blocks)	Base Channels	Patch Size	Parameters
Base	52	$1\times1\times1$3	128×128×128	≈ 62M
Width × 2.0	52	2$1\times1\times1$4	128×128×128	≈ 247M
Patch × 1.5	52	$1\times1\times1$5	192×192×192	≈ 62M

Pretraining is conducted on 18,000 CT volumes (44 targets) using the nnU-Net framework and a composite loss (Dice + CE). Fine-tuning retains the crop size or upscales to 192³ for context-heavy variants.

Mean five-fold cross-validation DSC over six CT/MR benchmarks (144 structures):

Model	Mean DSC
MedNeXt-v2 Base	82.62
MedNeXt-v2 Patch × 1.5	83.37
MedNeXt-v2 Width × 2.0	82.40

All MedNeXt-v2 configurations consistently outperform previous pretrained backbones, with Patch × 1.5 providing the largest mean gain by leveraging greater spatial context at finetuning (Roy et al., 19 Dec 2025). A plausible implication is that context scaling in 3D, within practical VRAM constraints, provides greater returns than width scaling beyond certain capacity.

7. Empirical Insights and Implications

• Depth scaling: Held constant at 52 blocks (reflecting a performance plateau for this depth on available data), further increases yielded diminishing returns.
• Width scaling: Doubling base width to 247M parameters gave only marginal or no improvement, indicating capacity bottlenecks for current dataset sizes.
• Context scaling: Increasing patch size to 192³ was more beneficial than width scaling, suggesting that spatial context is critical for volumetric tasks.
• Modalities: Pretraining with CT or MR yielded nearly identical downstream segmentations after full finetuning, indicating common volumetric representations across modalities.
• Pathological segmentation: Pretraining disproportionately benefits rare or pathological structure segmentation compared to anatomical organ segmentation, making pathological tasks sensitive benchmarks for future pretraining evaluations (Roy et al., 19 Dec 2025).

Compound-scaled 3D ConvNeXt architectures, as reflected in MedNeXt-v1 and v2, represent a state-of-the-art paradigm for volumetric medical segmentation under diverse data regimes, unifying convolutional efficiency with explicit structural scaling and micro-architectural stabilization.