---
title: 'CS-CNNs: Clifford-Steerable Convolutional Nets'
url: https://www.emergentmind.com/topics/clifford-steerable-convolutional-neural-networks-cs-cnns
type: topic
---

# CS-CNNs: Clifford-Steerable Convolutional Nets

Clifford-Steerable Convolutional Neural Networks (CS-CNNs) are a family of neural architectures for learning on multivector fields defined over pseudo-Euclidean spaces $\mathbb{R}^{p,q}$, engineered to be equivariant with respect to the isometries of these spaces, specifically the pseudo-Euclidean group $E(p,q) = \mathbb{R}^{p+q} \rtimes O(p,q)$. These networks feature an implicit parametrization of steerable kernels via Clifford algebra-valued equivariant neural networks, enabling $E(p, q)$-equivariance for arbitrary $p, q$, including physically relevant cases such as $E(3)$ on $\mathbb{R}^3$ (Euclidean space) and the Poincaré group $E(1,3)$ on Minkowski spacetime. CS-CNNs impose exact physical symmetry constraints, yielding significant improvements in data efficiency and invariance for PDE surrogate modeling in physics, such as fluid dynamics and electrodynamics [2402.14730], [2510.14007].

## 1. Mathematical Foundation: Clifford Algebras and Pseudo-Euclidean Symmetry

Clifford algebras $\mathrm{Cl}(p,q)$ underpin the signal representations in CS-CNNs. For a real vector space $V = \mathbb{R}^{p+q}$ with bilinear form $\eta$ of signature $(p, q)$, $\mathrm{Cl}(p,q)$ is constructed as the associative algebra generated by an orthonormal basis $\{e_i\}$, with relations
$$
e_i e_j + e_j e_i = 2 \eta_{ij}
$$
where $\eta_{ij}$ encodes the signature. A general multivector $M \in \mathrm{Cl}(p,q)$ decomposes as
$$
M = \sum_{k=0}^{p+q} \sum_{|I|=k} M^{(I)} e_I
$$
where each grade (scalar, vector, bivector, etc.) transforms under the spin representation: $g \in O(p,q)$ acts via $v \mapsto R_g v R_g^{-1}$ for $v \in V \subset \mathrm{Cl}(p,q)$, extended gradewise.

Feature fields in CS-CNNs are maps $f:\mathbb{R}^{p,q} \to \mathrm{Cl}(p,q)^c$ equipped with the natural representation $\rho(g)$ of $O(p,q)$. The $E(p, q)$ group acts by pulling back space and transforming Clifford algebra coefficients:
$$
[(t, g) \cdot f](x) = \rho_{\mathrm{cl}}(g)\left[ f(g^{-1}(x - t)) \right]
$$
where $\rho_{\mathrm{cl}}(g)$ is the standard Clifford action. This formalism naturally models all geometric field components that arise in physics, such as electromagnetic vector and bivector fields [2402.14730], [2510.14007].

## 2. Steerable Convolution: Equivariance Constraints and Kernel Parameterization

A convolutional layer in this framework has the form
$$
[L_K f_{\mathrm{in}}](x) = \int_{\mathbb{R}^{p,q}} K(x-y) \, f_{\mathrm{in}}(y) \, dy
$$
where the kernel $K:\mathbb{R}^{p,q} \to \operatorname{Hom}(W_{\mathrm{in}}, W_{\mathrm{out}})$. Full $E(p, q)$-equivariance requires (cf. Weiler & Cohen)
$$
K(gz) = \rho_{\mathrm{out}}(g) K(z) \rho_{\mathrm{in}}(g)^{-1}, \quad \forall g \in O(p, q), z \in \mathbb{R}^{p+q}
$$
This steerability constraint ensures outputs commute with $O(p, q)$ symmetries. For compact groups like $O(n)$ explicit kernel bases exist, but for non-compact $O(p, q)$ (e.g., Lorentz group), constructing an explicit steerable basis is intractable.

CS-CNNs circumvent this by parameterizing $K$ implicitly:
$$
K(z) = H[\mathcal{K}(z)]
$$
where $\mathcal{K}: \mathbb{R}^{p,q} \to \mathrm{Cl}(p,q)^{c_{\mathrm{out}}\times c_{\mathrm{in}}}$ is an $O(p, q)$-equivariant MLP (Clifford group equivariant neural network, CGENN), and $H$ is a fixed $O(p, q)$-equivariant "kernel head" that lifts multivector blocks into $\operatorname{Hom}(\mathrm{Cl}^c, \mathrm{Cl}^c)$. Equivariance follows by construction, as both $\mathcal{K}$ and $H$ are designed to commute with $O(p, q)$ actions [2402.14730].

## 3. Kernel Basis Completeness and Conditional Extensions

An important limitation, established in [2510.14007], is that the original implicit parameterization does not generate a complete basis for $O(p, q)$-equivariant kernels in general. For example, in the $(p, q) = (2, 0)$ case, the original kernel space lacks the Fourier angular modes corresponding to $\cos(2\phi)$ and $\sin(2\phi)$; only angular frequency 0 appears in the scalar channel. This restricts expressivity, as certain irreducible representations are omitted.

To resolve this, Conditional Clifford-Steerable Convolutions ("C-CSCNNs", *Editor's term*) are introduced, allowing the kernel to depend not only on the relative coordinate $z$, but also on global or local statistics $\zeta$ (typically mean-pooled inputs) of the feature field:
$$
[\hat{K}(z, \zeta)] = H[\hat{\mathcal{K}}(z, \zeta)]
$$
with $\hat{\mathcal{K}}$ being $O(p, q)$-equivariant in both arguments. The generalized steerability constraint is
$$
\hat{K}(g z, \rho(g) \zeta) = \rho_{\mathrm{out}}(g) \, \hat{K}(z, \zeta) \, \rho_{\mathrm{in}}(g)^{-1}
$$
This framework provably recovers the missing harmonics; for $O(2)$, $\zeta$-dependent combinations restore all angular frequencies, spanning the equivariant kernel space when $\zeta$ varies over $\mathrm{Cl}(p, q)^c$ [2510.14007]. *A plausible implication is that, in high-dimensional settings or complex geometric tasks, the conditional formulation is essential for full expressivity.*

## 4. Architectural Organization and Computational Considerations

A typical CS-CNN stack comprises several key components:

- **Embedding Layer:** Inputs (e.g., scalars, field components) are linearly mapped into $\mathrm{Cl}(p,q)^C$ channels using Clifford embedding plus a small MLP.
- **Residual Blocks:** Each block contains a CS-Convolution (computing $K(v)$ using the kernel network and head with a local support grid), channel-wise GroupNorm, and a nonlinear activation (e.g., grade-wise gating $\varphi(x^{(0)}) \cdot x$ with GELU CDF).
- **Final Readout:** Reduces Clifford channels to scalar or geometric output types.

Parameter and FLOP scaling are governed by the Clifford algebra dimension $2^d$, channel counts $C_{\mathrm{in,out}}$, and kernel network depth. For a kernel network of $M$ layers:
- Kernel network parameters: $\mathcal{O}(M C^2 2^d)$
- Kernel head parameters: $\mathcal{O}(d^2 C^2)$
- Total FLOPs per layer: $\mathcal{O}(S C^2 2^d)$, with $S$ support points

Conditioning incurs negligible additional cost: only one global pool per block. Implementations use JAX/Flax/XLA and ensure equivariance up to discretization artifacts through grid masking [2402.14730], [2510.14007].

## 5. Empirical Evaluation in Physical PDE Modeling

CS-CNNs and their conditional extensions have been validated on benchmarks central to physics-informed learning:

- **Navier–Stokes (2D incompressible):** Predicting velocity and pressure.
- **Shallow Water Equation (2D):** Forecasting combined height/velocity fields.
- **Maxwell Equations:** Both in $3$-D Euclidean and $2+1$-D relativistic settings (spacetime-multivector fields).

**Training:** Input consists of several past field states; prediction is for either one-step-ahead or $K$-step rollout. Loss is mean squared error; metrics include MSE and relative $L^2$ error. Baselines include standard ResNet, Clifford ResNet, $O(n)$-steerable CNNs, Fourier Neural Operator (FNO), group-equivariant FNO, and transformer-type models [2402.14730], [2510.14007].

**Empirical results:**
- CS-CNNs achieve 2–10$\times$ lower MSE than non-equivariant baselines, especially in low-data regimes. For Navier-Stokes, CS-CNNs match performance with 64 trajectories, versus 5120 for a ResNet.
- C-CSCNNs demonstrate further gains in accuracy, with relative $L^2$ error outperforming all baselines, including U-Net, FNO, Swin Transformer, and large-scale vision transformers. For the 5-step shallow water rollout, C-CSCNN with 10M parameters achieves 3.51% relative $L^2$ error, below FNO's 3.97%.
- Equivariance error is numerically negligible ($\sim 10^{-7}$), confirming symmetry enforcement.
- Only CS-CNNs and C-CSCNNs maintain stability and invariance under Lorentz boosts for Maxwell equations in Minkowski space, while other models break physical symmetry.

| Model              | #Params | Rel $L^2$ (SWE-5) |
|--------------------|---------|------------------|
| DilResNet          | 4.2M    | 13.20%           |
| U-Net$_{\mathrm{att}}$ | 148M    | 5.68%            |
| FNO                | 268M    | 3.97%            |
| CViT-L             | 92M     | 1.56%            |
| C-CSCNN (10M)      | 10M     | 3.51%            |
| C-CSCNN (55M)      | 55M     | 2.94%            |

## 6. Mechanisms Underlying Gains in Efficiency and Physical Consistency

The performance advantages of CS-CNNs derive from several interlocking properties:

- **Exact $O(p, q)$-equivariance:** The function space is restricted to symmetry-compatible solutions, decreasing sample complexity and increasing generalization.
- **Unified geometric representation:** Multivector features allow encoding scalars, vectors, and higher-grade objects without cumbersome manual design.
- **Implicit kernel parametrization:** Avoids explicit (and intractable) group theoretic kernel construction for non-compact groups, enabling practical network building for Lorentzian and other physically important scenarios.
- **Continuous and differentiable kernel networks:** Prevents aliasing and allows smooth steerable filters.
- **Residual stacking:** Successive application of CS-Convs recovers missing higher angular frequency modes, marginally mitigating expressivity loss of a single layer in the original, non-conditional scheme.

Although the original CS-CNN architecture omits parts of the full steerable kernel basis, as shown in [2510.14007], C-CSCNNs fully remedy this kernel incompleteness by including feature-dependent conditional terms in the kernel. This design achieves both algebraic completeness in the equivariant space and empirical data efficiency.

## 7. Limitations, Generalizations, and Outlook

The initial CS-CNN framework, while effective, exhibits kernel-basis incompleteness when only the position variable is used. The conditional formulation, by input-adaptive kernels, resolves this at minimal computational cost and ensures exact $E(p,q)$-equivariance to machine precision. Implementations are robust to discretization, but boundary effects (e.g. from grid masking) may break perfect equivariance.

CS-CNNs and C-CSCNNs present a unified, scalable methodology for symmetry-enforcing deep architectures in physics-informed learning, applicable to arbitrary pseudo-Euclidean geometries. The conditional extension is crucial for full expressivity in convolutions, as established by both formal analysis (harmonic decomposition, group representation) and empirical results on challenging PDE datasets [2402.14730], [2510.14007].

Source: https://www.emergentmind.com/topics/clifford-steerable-convolutional-neural-networks-cs-cnns