Depth-wise Harmonic Convolutions for LiDAR

• Depth-wise harmonic convolutions are a method combining circular-harmonic depth-wise kernels with point-wise mixing to maintain dense connectivity while reducing computational cost.
• They are integrated into LiDAR segmentation models like Lite-HDSeg, where pruned harmonic dense blocks efficiently reuse features for real-time performance.
• This approach optimizes runtime and memory usage by leveraging sparse connectivity and controlled channel growth without replacing standard 2D convolutions.

Searching arXiv for the specified paper and closely related work on HarDNet/LiDAR segmentation. First, retrieving the target paper on arXiv. Searching for HarDNet to identify the original harmonic dense connectivity reference mentioned in the paper. Depth-wise harmonic convolutions, in the context of LiDAR semantic segmentation, most commonly denote a prospective extension of Lite-HDSeg rather than the operator implemented in the original model. Lite-HDSeg introduces “Lite Harmonic Dense Convolutions” as a lightweight use of HarDNet-style harmonic dense connectivity inside a 2D encoder–decoder operating on spherical range images, where harmonic refers to the connectivity topology of dense blocks rather than to harmonic filter bases in the Fourier or circular-harmonic sense (Razani et al., 2021). The model retains standard learned 2D convolutional kernels, while reducing computational cost through pruned harmonic skip connections, controlled channel growth, contextual aggregation, and boundary refinement. The same source also outlines how a depth-wise harmonic convolution could be constructed as an implementation suggestion by combining circular-harmonic depth-wise kernels with point-wise mixing, but explicitly distinguishes that proposal from the architecture actually used in Lite-HDSeg (Razani et al., 2021).

1. Terminological scope and definition

In Lite-HDSeg, “Lite Harmonic Dense Convolutions” refer to a lightweight instantiation of HarDNet-style Harmonic Dense blocks inside a 2D encoder–decoder built on range images (Razani et al., 2021). The term harmonic follows HarDNet’s original meaning: a harmonic connectivity pattern within dense blocks, not harmonic filters in the Fourier or circular-harmonic sense (Razani et al., 2021). This distinction is central, because the architecture does not impose rotation-equivariant constraints and does not parameterize kernels through analytic harmonic bases.

The paper’s usage therefore differs from other meanings sometimes attached to “harmonic convolution.” Classical formulations such as

$$w(r,\theta) = \sum_{m=-M}^{M} a_m(r) e^{i m \theta}$$

or

$w(x,y) = \sum_n \alpha_n \phi_n(x,y)$

are explicitly identified as not being used in Lite-HDSeg to construct filters (Razani et al., 2021). Instead, the operator is a standard learned convolution over a 2D range image, embedded in a sparse dense-connectivity pattern.

This terminological precision matters because “depth-wise harmonic convolution” can be misunderstood as describing the Lite-HDSeg core operator. In the source paper, depth-wise separable convolutions and harmonic-basis filters are discussed only as efficiency-oriented or analytic alternatives; the architecture’s actual lightweight property comes from connection pruning and channel scheduling, not from replacing standard convolutions with depth-wise or grouped convolutions (Razani et al., 2021).

2. Harmonic dense connectivity in Lite-HDSeg

The harmonic component of Lite-HDSeg is a connectivity rule. HarDNet introduces Harmonic Dense blocks where layer $k$ connects to layers $k - 2^n$ if $2^n$ divides $k$ $(n \geq 0)$, rather than concatenating all previous layers as in DenseNet (Razani et al., 2021). Lite-HDSeg adopts and further prunes this idea in its “Lite-HD” block.

If $f_i$ denotes the $i$-th convolutional layer and $x_i$ its output feature map, Lite-HDSeg specifies the incoming connections for layer $w(x,y) = \sum_n \alpha_n \phi_n(x,y)$0 as

$w(x,y) = \sum_n \alpha_n \phi_n(x,y)$1

where $w(x,y) = \sum_n \alpha_n \phi_n(x,y)$2 is a small subset of prior layers selected by a pruned harmonic rule (Razani et al., 2021). Concretely, each layer takes at most $w(x,y) = \sum_n \alpha_n \phi_n(x,y)$3 inputs drawn from layers spaced by powers of $w(x,y) = \sum_n \alpha_n \phi_n(x,y)$4, with an essential immediate skip $w(x,y) = \sum_n \alpha_n \phi_n(x,y)$5 retained. The paper writes this compactly as

$w(x,y) = \sum_n \alpha_n \phi_n(x,y)$6

with floor/nearest-integer operators indicated and “or” denoted by $w(x,y) = \sum_n \alpha_n \phi_n(x,y)$7 (Razani et al., 2021).

Relative to conventional dense connectivity, this scheme preserves multi-path supervision and feature reuse while reducing the number of concatenations. The source characterizes the result as a form of harmonic pruning that is the core of Lite-HDSeg’s lightweight design (Razani et al., 2021). The blocks remain convolutional in the ordinary sense; what changes is the graph of inter-layer feature reuse.

A plausible implication is that the term “harmonic dense convolution” in this setting is best interpreted as a compound of two ideas: standard 2D convolution and a harmonic dense concatenation schedule. The convolution itself is not harmonic in basis design.

3. Computational structure and efficiency

The complexity reduction of Lite-HD is expressed through the cumulative number of concatenations across $w(x,y) = \sum_n \alpha_n \phi_n(x,y)$8 layers:

$w(x,y) = \sum_n \alpha_n \phi_n(x,y)$9

compared to $k$0 connections in a classical dense block (Razani et al., 2021). The source identifies fewer concatenation operations and smaller concatenated tensors as a primary reason for the FLOPs reduction.

For standard convolutions, the paper gives the reference estimate

$k$1

It also states that Lite-HDSeg’s FLOPs reduction primarily comes from two factors: pruned harmonic connectivity and controlled channel growth across blocks (Razani et al., 2021). The explicit channel schedule is

$k$2

The architecture is designed for a range-image input with image size $k$3, $k$4 (Razani et al., 2021). Because range-image LiDAR segmentation is memory-bandwidth-sensitive, the reduction of concatenation overhead is operationally significant. The source states that the Lite-HD backbone is particularly well-suited to embedded GPUs and to real-time segmentation on spherical projections where memory bandwidth and latency are critical (Razani et al., 2021).

This suggests that Lite-HDSeg’s efficiency is fundamentally graph-structural rather than operator-structural. The dominant savings arise before any hypothetical substitution of the convolution kernel family.

4. Encoder–decoder placement and surrounding modules

Lite-HDSeg is a 2D encoder–decoder architecture for semantic segmentation of full $k$5D LiDAR point clouds represented as spherical projections (Razani et al., 2021). The input is a 2D range image with channels $k$6, produced by the projection

$k$7

(Razani et al., 2021).

The encoder consists of a stack of Lite-HD blocks, denoted Enc1–Enc5, each followed by a Context Aggregation Module (CAM) for attention and denoising (Razani et al., 2021). At the input, the network places the Improved Contextual Module (ICM), described as a multi-branch Inception-like module that gathers multi-scale context using 2D convolutions with varied kernel sizes and dilations and outputs $k$8 channels (Razani et al., 2021). The source notes larger receptive fields and residual concatenation for ICM, while not enumerating exact kernels and dilations in the text.

The decoder contains four residual upsampling stages, Dec4–Dec1, using residual blocks with “increased kernel size” and skip connections from corresponding encoder stages (Razani et al., 2021). A final $k$9 convolution maps to $k - 2^n$0 output channels followed by softmax. Before this final layer, Lite-HDSeg inserts MCSPN, a multi-class Spatial Propagation Network for semantic-boundary refinement (Razani et al., 2021). One directional update is given as

$k - 2^n$1

and four directions are fused at the last time step (Razani et al., 2021).

The source characterizes the interaction among these modules in functional terms: harmonic connectivity improves gradient flow and feature reuse while keeping the model compact and fast, which complements ICM’s global context and MCSPN’s boundary refinement (Razani et al., 2021). In that arrangement, the Lite-HD block acts as the compact feature-extraction core, CAM mitigates LiDAR noise, and MCSPN sharpens small-object boundaries.

5. Relation to depth-wise separable and harmonic-basis convolutions

Lite-HDSeg discusses depth-separable convolutions as generally beneficial in projection-based methods and cites Xception in that context, but it does not state that standard convolutions inside Lite-HD blocks are replaced by depth-wise separable ones (Razani et al., 2021). The distinction is explicit: the model’s “lite” property comes from connection pruning and judicious channel growth, not from changing the convolution operator to depth-wise or grouped convolutions.

For completeness, the source provides the standard decomposition. A standard convolution is written as

$k - 2^n$2

with parameters $k - 2^n$3 and

$k - 2^n$4

A depth-wise convolution is

$k - 2^n$5

with parameters $k - 2^n$6 and

$k - 2^n$7

followed by point-wise mixing

$k - 2^n$8

with parameters $k - 2^n$9 and

$2^n$0

(Razani et al., 2021). If one wished to make Lite-HD lighter, replacing the per-layer convolution with depth-wise plus point-wise layers would yield

$2^n$1

and parameters

$2^n$2

but the source emphasizes that this is an implementation suggestion rather than a claim about Lite-HDSeg (Razani et al., 2021).

The same separation applies to harmonic-basis filters. The paper notes that a harmonic-basis parameterization of filters, not used in Lite-HDSeg, could reduce parameters by learning a small set of coefficients for analytic bases, for example

$2^n$3

with parameters per kernel equal to $2^n$4 rather than $2^n$5 (Razani et al., 2021). Again, this is presented as an alternative that the model does not implement.

A common misconception is therefore to treat Lite-HDSeg as a model based on Fourier, circular-harmonic, or rotation-equivariant filters. The source explicitly rejects that reading. The “harmonic” element belongs to the skip-connection schedule.

6. Suggested depth-wise harmonic extension

The paper provides practical guidance for a “depth-wise harmonic convolution” module in PyTorch as an implementation suggestion aligned with the lite design philosophy, while clearly separating it from the architecture actually evaluated (Razani et al., 2021). In this proposal, each depth-wise kernel is generated from circular-harmonic bases and then followed by a $2^n$6 point-wise mixing layer.

A $2^n$7 grid is defined with coordinates $2^n$8 centered at $2^n$9 and polar variables

$k$0

Using a radial envelope $k$1, for example Gaussian, the learned per-channel coefficients $k$2 for $k$3 define the depth-wise kernel

$k$4

This kernel is applied through depth-wise convolution and followed by point-wise mixing (Razani et al., 2021).

The source further recommends hyperparameters for range-image LiDAR segmentation: $k$5 or $k$6, $k$7–$k$8, and $k$9 (Razani et al., 2021). It suggests placing the depth-wise harmonic convolution inside the Lite-HD block as the per-layer convolution to reduce parameters and FLOPs, followed by point-wise mixing. It also states that this is most beneficial on high-resolution $(n \geq 0)$0 with large $(n \geq 0)$1, where depth-wise plus point-wise decomposition saves compute (Razani et al., 2021).

These recommendations remain speculative with respect to Lite-HDSeg’s reported results. The paper does not present benchmark numbers for the depth-wise harmonic extension. A plausible implication is that the proposal is best understood as a design avenue consistent with the lightweight intent of Lite-HD rather than as part of the validated baseline.

7. Training protocol, empirical results, and limitations

Lite-HDSeg is trained with SGD for $(n \geq 0)$2 epochs, with initial learning rate $(n \geq 0)$3 decayed $(n \geq 0)$4 per epoch, batch size $(n \geq 0)$5, dropout $(n \geq 0)$6, and data augmentation consisting of random rotation and translation together with $(n \geq 0)$7-axis flip (Razani et al., 2021). The total loss is

$(n \geq 0)$8

with $(n \geq 0)$9, $f_i$0, $f_i$1, and $f_i$2 (Razani et al., 2021). The boundary loss is

$f_i$3

using boundary maps

$f_i$4

(Razani et al., 2021).

On the SemanticKITTI test set, Lite-HDSeg achieves $f_i$5 at approximately $f_i$6 FPS on a single GTX 2080 Ti (Razani et al., 2021). The source states that it outperforms prior projection-based methods in the same real-time regime and reports strong gains on small, dynamic classes, including Motorcyclist at $f_i$7 IoU (Razani et al., 2021).

The ablation study on Sequence 08 validation is particularly informative for understanding the contribution of the harmonic-lite backbone and adjacent modules.

Configuration step	mIoU
SalsaNext baseline	56.9%
Lite-HDSeg encoder–decoder	59.1%
+ ICM	59.4%
+ CAM	60.3%
+ boundary loss	62.1%
+ MCSPN	64.4%

The source states that the largest single gain comes from MCSPN at $f_i$8, while the harmonic-lite backbone delivers consistent improvements without increasing runtime beyond the real-time envelope (Razani et al., 2021).

The same source also states several design trade-offs and limitations. Harmonic connectivity improves runtime by reducing concatenations and memory traffic but does not confer rotation equivariance (Razani et al., 2021). If circular-harmonic filters are introduced as an implementation suggestion, they can encourage orientation richness but may be sensitive to projection distortions and may not match LiDAR range-image anisotropy; tuning $f_i$9 and $i$0 is important (Razani et al., 2021). Depth-wise separable convolutions further reduce FLOPs but may reduce accuracy if channel mixing is insufficient, motivating sufficient $i$1 in the $i$2 layer and the use of residual connections to stabilize training (Razani et al., 2021). The paper also notes that Lite-HDSeg already mitigates LiDAR noise via CAM, whereas depth-wise harmonic kernels may be more sensitive to missing points unless combined with CAM and ICM (Razani et al., 2021).

Taken together, these points define the current status of depth-wise harmonic convolutions in this line of work. In the validated Lite-HDSeg model, the operative mechanism is pruned harmonic dense connectivity with standard 2D convolutions. Depth-wise harmonic kernels appear as a technically consistent but explicitly noncanonical extension layered atop that backbone.