Johnson-Lindenstrauss Lemma-Guided Convolution (JLC)

• JLC is a theory-guided grouped 3D convolution design that uses the JL lemma to set minimal channel widths for preserving local feature geometry.
• It balances computational efficiency and robust segmentation performance, addressing the conflict between full 3D convolution and depthwise methods.
• Empirical results show improved Dice scores and parameter efficiency across various datasets in lightweight 3D medical segmentation.

Johnson–Lindenstrauss lemma-guided Convolution (JLC) is a lightweight 3D grouped-convolution design introduced as a component of VeloxSeg, where the Johnson–Lindenstrauss (JL) lemma is used not to place an explicit random projection inside the convolution kernel, but to guide the minimum channel group size so that grouped 3D convolution remains efficient without excessively degrading local geometric relationships in feature space (Lu et al., 26 Sep 2025). Within that formulation, JLC addresses what the authors call an efficiency/robustness conflict in lightweight 3D medical segmentation: full 3D convolution is robust but expensive, whereas depthwise and aggressively grouped variants are efficient but may fragment representations, especially for small lesions, complex anatomies, and heterogeneous modalities (Lu et al., 26 Sep 2025).

1. Definition and problem setting

JLC belongs to the grouped-convolution family, but its distinguishing claim is that the number of channels per group should not be chosen purely heuristically. Instead, the JL lemma is interpreted as a lower-bound principle on the dimensionality required to preserve local neighborhood structure after channel partitioning. In this sense, JLC is a theory-guided sizing rule for grouped 3D convolution rather than a new random-projection operator in the compressed sensing sense (Lu et al., 26 Sep 2025).

The motivation is explicitly comparative. Standard full 3D convolution is described as robust but computationally expensive. Depthwise separable convolution is described as efficient but as reducing cost by aggressively decoupling channels, which can “fragment” the representation and “destroy the adjacency relationship between data in the feature space.” Ordinary group convolution sits between these extremes, but its group size is usually hand-chosen. JLC is proposed as a middle ground: it preserves the efficiency advantages of grouped 3D convolution while imposing a JL-inspired minimum group width intended to avoid fragile compressed representations (Lu et al., 26 Sep 2025).

This positioning is central to the method’s scope. JLC is not a generic JL embedding for arbitrary finite sets, and it is not presented as a replacement for the full theory of random projections. It is an architectural mechanism inside a lightweight 3D segmentation network, specifically the CNN stream of a dual-stream CNN–Transformer system (Lu et al., 26 Sep 2025).

2. JL-based sizing rule

The paper motivates JLC from the classical JL statement that a finite set $\mathcal{X}\subset\mathbb{R}^d$ with $|\mathcal{X}|=N$ can be embedded into dimension $d'$ on the order of $\varepsilon^{-2}\log N$ while approximately preserving pairwise Euclidean distances. JLC reinterprets that embedding dimension as the minimum number of channels per convolution group (Lu et al., 26 Sep 2025).

The core design rule is written as

$$C_{\mathrm{group}} = d' \ge c_{\text{JL}}\varepsilon^{-2}\log N(M,v),$$

where $C_{\mathrm{group}}$ is the number of channels per group, $M$ is the number of modalities, and $v$ is the volume ratio of the input image to the intermediate feature at a given network stage. The paper further assumes that each voxel in the stage-$k$ feature must retain information from at least $v$ input voxels, and that the manifold $|\mathcal{X}|=N$0 of segmentation-relevant input patches can be covered by finitely many samples with covering count $|\mathcal{X}|=N$1 (Lu et al., 26 Sep 2025).

Because $|\mathcal{X}|=N$2 is unavailable in practice, the paper introduces the empirical approximation

$|\mathcal{X}|=N$3

where $|\mathcal{X}|=N$4 is a task-difficulty scaling factor. For a typical 4-stage lightweight 3D medical segmentation network with

$|\mathcal{X}|=N$5

the appendix derives the approximate stagewise group sizes

$|\mathcal{X}|=N$6

For implementation, these values are simplified first to

$|\mathcal{X}|=N$7

then to

$|\mathcal{X}|=N$8

with the practical stagewise rule

$|\mathcal{X}|=N$9

The paper states that $d'$0 is determined from the most challenging AutoPET-II dataset “to ensure multi-organ generalization capability,” and the final empirical configuration is

$d'$1

corresponding to $d'$2 (Lu et al., 26 Sep 2025).

3. Architectural realization in VeloxSeg

JLC is implemented as grouped 3D convolution in the convolution encoder of VeloxSeg. VeloxSeg itself is described as a dual-stream CNN–Transformer architecture composed of Paired Window Attention (PWA) and JLC, following a “glance-and-focus” principle: PWA “glances” by rapidly retrieving multi-scale information, whereas JLC “focuses” by ensuring robust local feature extraction with minimal parameters (Lu et al., 26 Sep 2025).

The network has two 4-stage encoders: a modal-fusion convolution encoder, where JLC is the key local operator, and a modal-cooperative Transformer encoder, where PWA is the key global and multi-scale operator. JLC is described as consisting of “3 parallel JLCs at different scales” that fuse modal information and model local features. The overview and figure annotations specify “GC: group convolution” and “$d'$3 convolution as modal mixer” (Lu et al., 26 Sep 2025).

The ablation text further specifies that replacing a single kernel setting $d'$4 with parallel small kernels $d'$5 improves both Dice and efficiency. This suggests a multi-branch grouped 3D convolution block with parallel kernel sizes $d'$6, $d'$7, and $d'$8, stagewise group sizes $d'$9, and $\varepsilon^{-2}\log N$0 convolution used as a modal mixer. The paper does not provide an explicit concatenation or summation formula for these branches, and the exact internal JLC equations are less fully specified than those for PWA (Lu et al., 26 Sep 2025).

The most reproducible implementation settings reported for JLC are a 4-stage encoder, final channel widths

$\varepsilon^{-2}\log N$1

parallel kernel sizes

$\varepsilon^{-2}\log N$2

and final JL-guided group sizes

$\varepsilon^{-2}\log N$3

The training framework is PyTorch 2.4.1 with batch size $\varepsilon^{-2}\log N$4, AdamW, learning rate $\varepsilon^{-2}\log N$5, weight decay $\varepsilon^{-2}\log N$6, $\varepsilon^{-2}\log N$7 epochs, and linear warmup plus cosine annealing. Patch sizes are $\varepsilon^{-2}\log N$8 for AutoPET-II and BraTS2021, and $\varepsilon^{-2}\log N$9 for Hecktor2022 (Lu et al., 26 Sep 2025).

4. Empirical characterization

The JLC ablations show a consistent trade-off between group width, computational cost, and segmentation accuracy. On AutoPET-II, a convolution-only model with width $$C_{\mathrm{group}} = d' \ge c_{\text{JL}}\varepsilon^{-2}\log N(M,v),$$0 and JLC reports $$C_{\mathrm{group}} = d' \ge c_{\text{JL}}\varepsilon^{-2}\log N(M,v),$$1 MParams, $$C_{\mathrm{group}} = d' \ge c_{\text{JL}}\varepsilon^{-2}\log N(M,v),$$2 GFLOPs, $$C_{\mathrm{group}} = d' \ge c_{\text{JL}}\varepsilon^{-2}\log N(M,v),$$3 patches/s, and $$C_{\mathrm{group}} = d' \ge c_{\text{JL}}\varepsilon^{-2}\log N(M,v),$$4 Dice. Replacing a large kernel $$C_{\mathrm{group}} = d' \ge c_{\text{JL}}\varepsilon^{-2}\log N(M,v),$$5 with parallel kernels $$C_{\mathrm{group}} = d' \ge c_{\text{JL}}\varepsilon^{-2}\log N(M,v),$$6 yields $$C_{\mathrm{group}} = d' \ge c_{\text{JL}}\varepsilon^{-2}\log N(M,v),$$7 MParams, $$C_{\mathrm{group}} = d' \ge c_{\text{JL}}\varepsilon^{-2}\log N(M,v),$$8 GFLOPs, $$C_{\mathrm{group}} = d' \ge c_{\text{JL}}\varepsilon^{-2}\log N(M,v),$$9 patches/s, and $C_{\mathrm{group}}$0 Dice (Lu et al., 26 Sep 2025).

A group-size sweep on the same benchmark reports the following progression: $C_{\mathrm{group}}$1 gives $C_{\mathrm{group}}$2 MParams, $C_{\mathrm{group}}$3 GFLOPs, $C_{\mathrm{group}}$4 throughput, and $C_{\mathrm{group}}$5 Dice; $C_{\mathrm{group}}$6 gives $C_{\mathrm{group}}$7 MParams, $C_{\mathrm{group}}$8 GFLOPs, $C_{\mathrm{group}}$9 throughput, and $M$0 Dice; $M$1 gives $M$2 MParams, $M$3 GFLOPs, $M$4 throughput, and $M$5 Dice; the JL-guided $M$6 gives $M$7 MParams, $M$8 GFLOPs, $M$9 throughput, and $v$0 Dice; $v$1 gives $v$2 MParams, $v$3 GFLOPs, $v$4 throughput, and $v$5 Dice; and $v$6 gives $v$7 MParams, $v$8 GFLOPs, $v$9 throughput, and $k$0 Dice. In a separate comparison, the paper states that the JL-guided configuration $k$1 “consistently surpasses” the larger setup $k$2, with the best performance of $k$3 Dice at $k$4 (Lu et al., 26 Sep 2025).

Cross-dataset results reinforce the robustness claim. On Hecktor2022, $k$5 yields $k$6 MParams, $k$7 GFLOPs, and $k$8 Dice; $k$9 yields $v$0 MParams, $v$1 GFLOPs, and $v$2 Dice; and $v$3 yields $v$4 MParams, $v$5 GFLOPs, and $v$6 Dice. On BraTS2021, the corresponding values are $v$7 MParams, $v$8 GFLOPs, $v$9 Dice; $|\mathcal{X}|=N$00 MParams, $|\mathcal{X}|=N$01 GFLOPs, $|\mathcal{X}|=N$02 Dice; and $|\mathcal{X}|=N$03 MParams, $|\mathcal{X}|=N$04 GFLOPs, $|\mathcal{X}|=N$05 Dice (Lu et al., 26 Sep 2025).

The comparison with pruning is particularly diagnostic. In domain generalization from BraTS2021 to MSD2019, full convolution reports $|\mathcal{X}|=N$06 MParams and $|\mathcal{X}|=N$07 Dice, pruned full convolution reports $|\mathcal{X}|=N$08 MParams and $|\mathcal{X}|=N$09 Dice, and JLC reports $|\mathcal{X}|=N$10 MParams and $|\mathcal{X}|=N$11 Dice. This is the paper’s strongest evidence that JLC is not merely a compact grouped-convolution variant, but a compact variant with better robustness and generalization than data-specific pruning (Lu et al., 26 Sep 2025).

5. Relation to JL theory and convolutional embeddings

Classical JL theory concerns random linear maps $|\mathcal{X}|=N$12 that preserve the norm of a fixed vector with probability at least $|\mathcal{X}|=N$13, with

$|\mathcal{X}|=N$14

and recent unified analysis covers spherical, binary-coin, sparse JL, Gaussian, and broader sub-Gaussian constructions under a common framework (Li, 2024). In that literature, the central object is an explicit random embedding matrix or a structured sketch.

JLC differs fundamentally from that setting. The implementation does not specify a Gaussian matrix, sparse JL transform, spherical construction, SRHT, or trainable JL embedding matrix inside the convolution. The learned grouped convolution is also not proved to be a JL embedding. The JL lemma guides only one architectural choice: the minimum channel-per-group width. For that reason, JLC is best understood as a theory-guided grouped-convolution sizing scheme rather than as a direct instantiation of a JL transform (Lu et al., 26 Sep 2025).

This distinction is sharpened by results on structured convolution-like JL maps. Random Toeplitz and partial circulant embeddings correspond to convolutional or correlation-like operators and can be applied in $|\mathcal{X}|=N$15 time via FFT, but in the worst case they require

$|\mathcal{X}|=N$16

rather than the optimal

$|\mathcal{X}|=N$17

for generic JL embeddings (Freksen et al., 2017). More broadly, worst-case pairwise-distance preservation cannot beat

$|\mathcal{X}|=N$18

even for nonlinear embeddings on suitable $|\mathcal{X}|=N$19-point sets (Larsen et al., 2016). In that context, JLC’s contribution is not a new optimal embedding theorem; it is a translation of JL-style dimensionality reasoning into a practical hyperparameter rule for lightweight 3D grouped convolution.

6. Interpretation, limitations, and significance

The paper is explicit that several steps in JLC are heuristic rather than theorem-enforced. The covering count $|\mathcal{X}|=N$20 is unavailable in practice and is replaced by

$|\mathcal{X}|=N$21

The task-difficulty exponent $|\mathcal{X}|=N$22 is selected empirically on AutoPET-II. The final pattern $|\mathcal{X}|=N$23 is a discretized engineering choice constrained by divisibility of channel counts, and there is no explicit proof that grouped 3D convolution with those widths preserves the geometry of the learned patch manifold in the strong sense of classical JL theory (Lu et al., 26 Sep 2025).

A further limitation is descriptive specificity. The network-level role of JLC is clear, but its internal equations are less fully specified than those for PWA, and the paper does not provide a standalone FLOPs derivation or pseudocode dedicated to JLC. Consequently, the method is easier to reproduce as a stagewise grouped-convolution prescription than as a formally closed module definition (Lu et al., 26 Sep 2025).

Its significance lies in how it reframes lightweight convolution design. Instead of treating group size as a purely empirical knob, JLC ties it to modality count, stage compression ratio, and a JL-inspired lower-bound argument. This suggests a broader methodological stance: lightweight local operators can be designed by preserving enough channel width to avoid feature-space fragmentation, even when the final implementation remains an ordinary grouped 3D convolution. A plausible implication is that JLC is best viewed not as a new mathematical embedding class, but as a principled design doctrine for robust low-cost 3D local feature extraction (Lu et al., 26 Sep 2025).