3D Steerable CNNs: Equivariant Architectures

• 3D Steerable CNNs are convolutional architectures that maintain equivariance to 3D rotations using group representation theory and spherical harmonics.
• They construct equivariant kernels by parameterizing with spherical harmonics and radial profiles, ensuring consistent transformation under SO(3) and SE(3).
• Applications include medical imaging, molecular analysis, and physical simulations, with implementations that offer computational efficiency and robust performance.

A 3D steerable convolutional neural network (CNN) is a class of architectures designed to preserve equivariance to the group of 3D rotations, particularly the special orthogonal group SO(3) and the full special Euclidean group SE(3), in convolutional operations. These networks leverage group representation theory and spherical harmonics to build convolutional kernels whose responses transform according to prescribed rules under group actions. This enables precise modeling of volumetric and geometric data with inherent rotational symmetries—critical in applications across scientific computing, vision, and medical imaging.

1. Mathematical Foundations and Equivariance

The central property of 3D steerable CNNs is equivariance to the action of SO(3) or SE(3). For a spatial point $x \in \mathbb{R}^3$ and group element $g \in SO(3)$, the action is $x \mapsto g \cdot x$. Feature fields $f: \mathbb{R}^3 \to W$ transform via a representation $\rho(g)$ as $[\rho(g)f](x) = \rho(g) f(g^{-1} \cdot x)$. Input (and output) features may represent scalars, vectors, or higher-order tensors, corresponding to irreducible SO(3) representations (“irreps”) $W = \bigoplus_\ell \mathbb{R}^{2\ell+1}$ (Weiler et al., 2018).

A convolution $$f_{\text{out}}(x) = \int K(x-y) f_{\text{in}}(y) \,dy$$ is SO(3)-equivariant iff the kernel satisfies $$K(g \cdot z) = \rho_{\text{out}}(g) K(z) \rho_{\text{in}}(g)^{-1}$$. This constraint requires that kernel weights transform appropriately under rotation, maintaining the field structure.

2. Steerable Kernel Construction

The space of equivariant convolutional kernels between fields of irreducible types admits a parameterization in terms of spherical harmonics $Y_{Jm}$ for the angular part and radial profiles $g \in SO(3)$0 for the radial part (Weiler et al., 2018, Diaz et al., 2023). The general steerable kernel block for mapping input of order $g \in SO(3)$1 to output of order $g \in SO(3)$2 is

$g \in SO(3)$3

where $g \in SO(3)$4 and $g \in SO(3)$5 are Wigner-D matrices representing SO(3) on the respective output and input spaces. The Clebsch–Gordan decomposition $g \in SO(3)$6 underpins this form, ensuring exact equivariance.

Practical implementations precompute kernels on a small 3D grid, with each input→output block built as a sum of basis functions (Andrearczyk et al., 2020). Parameter sharing and reduced complexity result, with the number of free weights scaling as the sum over radial and angular basis elements rather than kernel volume.

3. Nonlinearities and Network Architecture

Maintaining equivariance through nonlinear layers presents technical challenges. For scalar fields ($g \in SO(3)$7), pointwise nonlinearities (e.g., ReLU) preserve equivariance. For higher-order fields ($g \in SO(3)$8), typical approaches include gated nonlinearities: each non-scalar field is rescaled by a learned scalar “gate” to preserve transformation laws under SO(3)/SE(3) (Weiler et al., 2018, Diaz et al., 2023). Harmonic analysis facilitates band-limited, equivariance-preserving nonlinearities via FFT-based schemes for SO(2), and extensions exist for E(3)-equivariant 3D surface networks (Franzen et al., 2021).

Architectural patterns include:

• Stacks of steerable convolutional blocks with interleaved equivariant normalization and gating
• Equivariant pooling (e.g., maximal norm for $g \in SO(3)$9 features)
• Downsampling via equivariant averaging or anti-aliased strided convs to prevent angular aliasing
• Global pooling for invariant outputs

Equivariant U-Net structures have been adopted for segmentation tasks, substituting standard convolutions, normalization, and pooling with their equivariant counterparts (Diaz et al., 2023).

4. Extensions and Generalizations

Several generalizations extend the basic analytic/harmonic steerable kernel construction:

Clifford-Steerable CNNs (CSCNNs): By encoding features as multivector fields valued in the Clifford algebra $x \mapsto g \cdot x$0, these networks accommodate pseudo-orthogonal symmetries (O(3), Pin(3)), yielding equivariance to broader transformation groups. Kernels are parameterized via O(3)-equivariant MLPs mapping inputs in $x \mapsto g \cdot x$1 (Szarvas et al., 15 Oct 2025). However, single-layer CSCNNs exhibit an expressivity gap—certain irreducible blocks are missing due to the algebraic structure of geometric products.

Conditional Clifford-Steerable Kernels: To address the incompleteness, kernels are conditioned on global O(3)-equivariant summaries of the input field (e.g., mean-pooled multivectors). An O(3)-equivariant neural network computes the kernel as a function of both relative position and context summary, ensuring the completed span of SO(3)-equivariant maps. This guarantees, both theoretically—by iterating geometric products of independent vectors—and empirically, a universal kernel basis spanning all possible SO(3)-equivariant linear maps (Szarvas et al., 15 Oct 2025).

PDO-s3DCNNs: Partial Differential Operator (PDO) based parameterization offers an alternative approach, representing steerable kernels as linear combinations of low-order derivatives. The equivariance conditions collapse into linear systems solvable via SVD, yielding finite filter bases valid for all SO(3) subgroups and representations. Discrete implementations utilize finite-difference or Gaussian-derivative approximations, with Gaussian smoothing crucial for near-exact equivariance under continuous SO(3) (Shen et al., 2022).

5. Sparse and Efficient Implementations

Steerable convolutions are computationally expensive when applied densely across 3D grids. Sparse Steerable Convolutions (SS-Conv) address this by operating on sparse voxel tensors, maintaining hash tables of active voxels and related features (Lin et al., 2021). Kernels are precomputed within small support neighborhoods and applied via rule-books that match offsets between active sites. The resulting speedup is significant: on a $x \mapsto g \cdot x$2 grid with 5% occupancy, SS-Conv achieves a 2.7× speedup and one-third memory usage compared to dense implementations, while strict SE(3)-equivariance is preserved. This is particularly beneficial in instance-level and category-level 6D pose estimation and tracking.

6. Empirical Performance and Applications

3D steerable CNNs demonstrate substantial advantages in parameter efficiency, robustness to pose variability, and sample efficiency across numerous domains:

• Protein structure and molecular tasks: SE(3)-equivariant steerable networks achieve near-universal performance with order-of-magnitude parameter reduction in tasks with inherent geometric symmetry (Weiler et al., 2018).
• Medical imaging: SO(3)-steerable convolutions enhance segmentation accuracy and robustness to arbitrary orientations, eliminating the need for rotation augmentation. Parameter counts decrease by 30–50% compared to standard CNNs, with performance maintained under limited data (Diaz et al., 2023, Andrearczyk et al., 2020).
• Physical simulation: Conditional Clifford-Steerable CNNs outperform baseline methods on PDE forecasting tasks (fluid dynamics, relativistic electrodynamics), reducing mean-squared error by 34–54% with negligible overhead and exact equivariance (Szarvas et al., 15 Oct 2025).
• 3D object analysis: Sparse steerable architectures excel at 6D pose estimation and tracking, outperforming existing pipelines by substantial margins in both accuracy and efficiency (Lin et al., 2021).
• Shape retrieval and segmentation: PDO-s3DCNNs match or surpass prior methods for 3D shape retrieval and EM segmentation, achieving competitive scores with far lower network complexity (Shen et al., 2022).

7. Limitations and Prospects

Known challenges include:

• Strided convolutions with stride $x \mapsto g \cdot x$3 may break equivariance, so careful design with equivariant pooling and interpolation is necessary (Lin et al., 2021).
• Batch normalization for irreducible feature types is limited to scale-only forms, potentially hindering universality and optimization (Shen et al., 2022).
• Discretization error can break full SO(3) equivariance except for large, smoothed kernels or specific subgroups.
• Standard algebraic constructions (e.g., via Clifford products) miss certain irreducible blocks without conditioning or basis augmentation (Szarvas et al., 15 Oct 2025).

Ongoing extensions focus on:

• Incorporating learned or hierarchical pooling operators or context-sensitive summaries for kernel conditioning (Szarvas et al., 15 Oct 2025).
• Multiscale context integration and attention mechanisms (Lin et al., 2021).
• Applications to unstructured point sets, mesh-based data, and non-Euclidean spaces.
• Higher-order PDOs and generalizations to symmetry groups beyond SO(3), such as Lorentzian or conformal groups.

Theoretical and empirical advances suggest 3D steerable CNNs, particularly those with universal or conditional kernel bases, constitute a comprehensive and adaptable framework for learning with volumetric data under rigid, and potentially more general, symmetry constraints.