Tensor Equivariance in Geometric Models

• Tensor Equivariance is a property ensuring that tensor-valued features transform according to specified group actions, preserving geometric structure.
• It is applied in deep learning via methods such as SE(3) convolutions, local canonicalization, and pooling-based schemes to enforce symmetry.
• Recent work utilizes diagnostic tools like Lie derivatives to measure equivariance, correlating lower error with improved model performance on varying tasks.

Searching arXiv for recent and foundational papers on tensor equivariance across geometric deep learning, communications, and equivariance measurement. Tensor equivariance denotes the requirement that tensor-valued features or outputs transform in a prescribed way under transformations of the input, rather than remaining merely invariant. In the standard representation-theoretic form, a map $f:V_1\to V_2$ is equivariant to a group $G$ when

$f(p_1(g)x)=p_2(g)f(x),$

and, for tensor features under $\mathrm O(d)$, an order-$n$ tensor transforms as

$$T'_{i_1\ldots i_n}=R(g)_{i_1j_1}\cdots R(g)_{i_nj_n}T_{j_1\ldots j_n}.$$

Recent literature also uses “tensor equivariance” as a collective term for multidimensional equivariance, high-order equivariance, and multidimensional invariance on tensor axes. Across these usages, the common objective is to enforce or diagnose predictable transformation laws in learned representations, message-passing schemes, and tensor-valued predictions (Lippmann et al., 2024, Wang et al., 2024, Gruver et al., 2022).

1. Definitions and scope

The core distinction is between invariance and equivariance. An invariant function satisfies $f(g\!\cdot\!x)=f(x)$, while an equivariant function preserves the action of the symmetry group between input and output spaces. For geometric learning under $\mathrm O(d)$, tensor equivariance requires feature channels such as scalars, vectors, higher-order tensors, and pseudotensors to transform under the appropriate tensor representation. For communication tensors, the relevant symmetry is often permutation of tensor dimensions rather than Euclidean rotation; in that setting, tensor equivariance is defined through permutation actions on one or more axes (Lippmann et al., 2024, Wang et al., 2024).

Context	Symmetry action	Representative formulation
Geometric deep learning	Rotations and reflections in $\mathrm O(d)$	$$T'_{i_1\ldots i_n}=R(g)_{i_1j_1}\cdots R(g)_{i_nj_n}T_{j_1\ldots j_n}$$
General equivariance	Group action on input and output spaces	$G$0
Tensor-axis symmetries	Permutations on one or more dimensions	multidimensional equivariance, high-order equivariance, multidimensional invariance

A broad constructive viewpoint appears in the symmetric tensor network literature. For compact groups such as $G$1, Hilbert’s finiteness theorem is used to motivate finite generating sets for invariant polynomials, while equivariant functions are obtained from invariant constructions by differentiation. This framework explicitly handles Cartesian tensors with different rank and spherical tensors with different types, and represents valid invariant and equivariant operations as contractions of symmetric tensor networks (Zhang et al., 18 Aug 2025).

2. Euclidean-group tensor equivariance

In 3D geometric learning, tensor equivariance is commonly instantiated through irreducible representations of $G$2 or $G$3. Tensor Field Networks introduced layers that are locally equivariant to 3D rotations, translations, and permutations of points at every layer. Their filters are built from spherical harmonics, so each layer accepts as input and guarantees as output scalars, vectors, and higher-order tensors in the geometric sense. A typical filter has the form

$G$4

and tensor products are coupled through Clebsch–Gordan coefficients to ensure that the output transforms under the correct irreducible representation (Thomas et al., 2018).

A later unification showed that $G$5-convolution and steerable convolution are equivalent: steerable convolution is the Fourier transform of $G$6 convolution. This result provides explicit formulas relating kernels learned by group-convolution networks and steerable-kernel networks, gives theoretical justifications of separability of $G$7 group convolution, and places different roto-translationally equivariant constructions in a single formalism. The same analysis also derives new Tensor Field Network nonlinearities via the equivalence principle (Poulenard et al., 2022).

Not all Euclidean-group approaches target the full hierarchy of irreducible representations. Bilinear Tensor Networks restrict attention to scalars, vectors, and order-2 tensors, and show that judicious symmetry breaking from $G$8 to $G$9 can increase expressiveness while retaining physically relevant structure. On the b-tagging task, the baseline rejection at $f(p_1(g)x)=p_2(g)f(x),$0 signal efficiency is $f(p_1(g)x)=p_2(g)f(x),$1, while the best BTN rejection at $f(p_1(g)x)=p_2(g)f(x),$2 signal efficiency is $f(p_1(g)x)=p_2(g)f(x),$3, corresponding to a $f(p_1(g)x)=p_2(g)f(x),$4 improvement in rejection score (Shimmin et al., 2023).

3. Permutation, incidence, and symmetric-power formulations

Tensor equivariance is not limited to Euclidean geometry. For data whose symmetry is induced by permutations of indices, equivariant maps are characterized by parameter sharing patterns determined by group orbits. Incidence Networks formalize incidence tensors as higher-order generalizations of incidence matrices, show that any incidence tensor decomposes into invariant subsets, and prove that the corresponding equivariant linear maps admit an efficient pooling-and-broadcasting implementation: $f(p_1(g)x)=p_2(g)f(x),$5 This yields a unified equivariant linear algebra for graphs, simplicial complexes, and polytopes (Albooyeh et al., 2019).

For symmetric tensors, the recent characterization of all linear $f(p_1(g)x)=p_2(g)f(x),$6-equivariant maps between symmetric power spaces $f(p_1(g)x)=p_2(g)f(x),$7 and $f(p_1(g)x)=p_2(g)f(x),$8 gives two exact bases: an orbit basis and a diagram basis. The orbit basis corresponds bijectively to $f(p_1(g)x)=p_2(g)f(x),$9-bipartitions with at most $\mathrm O(d)$0 blocks, while the diagram basis provides an efficient computational rule through summation patterns on tensor entries. The paper reports that these functions are highly data efficient compared to standard MLPs and have potential to generalize well to symmetric tensors of different sizes (Pearce-Crump, 14 Mar 2025).

In permutation-heavy engineering settings, tensor equivariance is sometimes defined explicitly as a combination of multidimensional equivariance, high-order equivariance, and multidimensional invariance. For a tensor-valued map

$\mathrm O(d)$1

multidimensional equivariance requires

$\mathrm O(d)$2

and the resulting Tensor Equivariant Neural Network modules realize the required parameter sharing by concise tensor operations rather than dense fully connected layers (Wang et al., 2024).

4. Local canonicalization and tensorial message passing

A distinct line of work replaces specialized equivariant tensor products with local reference frames. In local canonicalization, each node predicts an orthonormal local frame $\mathrm O(d)$3, maps features into that frame, and communicates by changing basis between sender and receiver frames. The tensorial message-passing update is written as

$\mathrm O(d)$4

Because features and messages are transformed between local frames by the appropriate representation, the entire pipeline is rigorously equivariant under $\mathrm O(d)$5 (Lippmann et al., 2024).

This framework was motivated in part by a limitation of canonicalization-only schemes. If two nodes use different local frames, geometric information such as a direction is lost unless messages are transformed tensorially between frames. Tensorial messages are therefore strictly more expressive than scalar messages under local canonicalization. On normal vector regression on ModelNet40, the reported cosine similarities are $\mathrm O(d)$6 for PointNet++ with tensor messages, $\mathrm O(d)$7 for PointNet++ with scalar messages, $\mathrm O(d)$8 for PointNet++ with data augmentation, and $\mathrm O(d)$9 for Luo et al. (2022). The ablation further reports that random frames + tensor messages achieves $n$0, better than learned frames + scalar messages at $n$1 (Lippmann et al., 2024).

Canonicalization has also been used for crystal tensor property prediction. GoeCTP applies polar decomposition

$n$2

to the lattice matrix, uses the standardized lattice as an orientation-invariant representation, predicts the tensor in this canonicalized space, and then maps it back by the orthogonal factor. For a second-rank tensor,

$n$3

The framework is reported to achieve the best prediction accuracy and runs up to 13 times faster compared to existing state-of-the-art methods in benchmarking datasets (Hua et al., 2024).

The same general direction is extended by a framework that transfers existing tensor field networks into the local canonicalization paradigm while preserving equivariance and significantly improving the runtime. That work compares scalar, Cartesian, irreducible, and learned equivariant representations, and publishes the tensor_frames package, a PyTorchGeometric based implementation for local canonicalization that enables straightforward integration of equivariance into any standard message passing neural network (Gerhartz et al., 30 Sep 2025).

5. Measurement, diagnostics, and learned equivariance

Tensor equivariance is not only designed; it can also be measured. Gruver, Finzi, Goldblum, and Wilson introduce the Lie derivative as a method for measuring equivariance with strong mathematical foundations and minimal hyperparameters. For a vector field $n$4 associated with the Lie algebra of a symmetry group,

$n$5

and the Local Equivariance Error is defined by

$n$6

Because the Lie derivative satisfies a chain rule, equivariance violations can be decomposed layerwise (Gruver et al., 2022).

The resulting empirical picture is more nuanced than the classical opposition between equivariant and non-equivariant architectures. In an analysis of hundreds of pretrained CNNs, Vision Transformers, and Mixers, many violations of equivariance are linked to spatial aliasing in ubiquitous network layers, such as pointwise non-linearities, downsampling, and certain patch embedding layers. As models get larger and more accurate they tend to display more equivariance, regardless of architecture; transformers can be more equivariant than convolutional neural networks after training; lower local equivariance error correlates strongly with higher test accuracy; and all architectures become less equivariant on data further from the training distribution, so architectural biases do not close the equivariance gap on OOD data (Gruver et al., 2022).

At the same time, explicit architectural equivariance remains consequential in scientific machine learning. In a complex scalar field theory with periodic boundary conditions, translationally equivariant architectures using convolutions and spatial pooling with stride $n$7 and circular padding generalize significantly better than strided or flattening alternatives, including to unseen chemical potentials and different lattice sizes. In the multiple worm counting task, the equivariant architecture achieved $n$8 accuracy in all 20 tested instances (Bulusu et al., 2021).

6. Applications, misconceptions, and terminological issues

A recurring misconception is that tensor equivariance is a single method. The literature instead presents several non-equivalent constructions: steerable kernels and $n$9 group convolution are theoretically equivalent under a Fourier–Wigner view, whereas local canonicalization, symmetric tensor networks, pooling-and-broadcasting maps on incidence tensors, and tensor-axis permutation modules realize equivariance through different algebraic mechanisms (Poulenard et al., 2022, Zhang et al., 18 Aug 2025, Albooyeh et al., 2019, Wang et al., 2024).

A second misconception is that exact architectural equivariance is always necessary. The measurement literature shows that modern large models can learn strong equivariances from data and augmentation, and that more accurate models tend to be more equivariant regardless of architecture. A plausible implication is that explicit priors may be less critical for large-scale in-distribution visual recognition than once assumed. The same source also states that architectural equivariance remains critical where exact equivariance is indispensable, such as in physical modeling (Gruver et al., 2022).

A third misconception is that canonicalization by itself is sufficient. The local-frame message-passing results show that canonicalization-only approaches can fail to communicate geometric information consistently between different local coordinate frames, whereas tensorial messages provide exact by-design equivariance and are strictly more expressive in that setting (Lippmann et al., 2024).

The application space is correspondingly broad. In crystalline materials, tensor equivariance is required because tensor properties must be equivariant under the $$T'_{i_1\ldots i_n}=R(g)_{i_1j_1}\cdots R(g)_{i_nj_n}T_{j_1\ldots j_n}.$$0 group, and canonicalization yields fast tensor prediction without explicit equivariant layers (Hua et al., 2024). In MU-MIMO communications, TE modules support precoding and user scheduling with near-optimal performance, significantly lower computational complexity, and generalization to inputs with varying sizes across multiple dimensions (Wang et al., 2024). In symbol-level precoding, a TE-based framework leverages permutation-induced output permutations, achieves linear computational complexity, and reports an approximately 80-times speedup over conventional methods while maintaining strong generalization across user numbers and symbol block lengths (Zhang et al., 2 Oct 2025). In knowledge graph completion, the Equivariance Regularizer leverages semantic equivariance between head and tail entities, is generic for both distance based models and tensor factorization based models, and is connected to tensor nuclear norm regularization (Cao et al., 2022).

The acronym “TE” is itself overloaded. In robust tensor statistics, TE also denotes the tensor elliptical distribution, an extension of the multilinear normal distribution with affine-transformation properties and an integral representation as a mixture over scaled multilinear normal densities. This usage is unrelated to neural equivariance, but it illustrates that “tensor equivariance” should be interpreted contextually rather than assumed from abbreviation alone (Arashi, 2017).