Equivariant Approximation Theorem

• Equivariant approximation is defined as approximating functions while strictly preserving underlying symmetries, ensuring all approximants adhere to a specified group action.
• It leverages algebraic and local-analytic techniques, such as symmetric matrix properties and cancellation of lower order terms in Taylor expansions, to achieve enhanced accuracy.
• The theorem underpins neural network architectures by proving that symmetry-respecting models are dense in continuous equivariant maps, ensuring universal approximation within constrained symmetry classes.

An equivariant approximation theorem is a result asserting that approximation can be carried out within a prescribed symmetry class: if a target map, operator, or topological object is compatible with an action of a group, groupoid, or more general category, then the approximant can be chosen to satisfy the same compatibility. The term does not denote a single canonical theorem. In the literature it appears in several technically distinct forms, including symmetry-adapted finite-difference formulas, universal approximation theorems for equivariant neural architectures, constructive ridgelet-type reconstruction results, approximation of equivariant topological invariants, compact approximation of symmetric definable sets, and equivariant Oka principles (Dellnitz, 2017, Kumagai et al., 2020, Kutzschebauch et al., 2019, Basu et al., 2023).

1. Recurring meaning of equivariant approximation

Across its variants, the central datum is an action of a symmetry object on inputs and outputs. In the standard group-action setting, a map $F$ is equivariant if

$F[g\cdot x]=g\cdot F[x].$

For invariant maps, the output action is trivial. What changes from one theorem to another is the ambient category and the approximation notion: uniform approximation on compact sets, $L^2$-reconstruction, approximation of homotopy or homology, approximation of $L^2$-multiplicities, or deformation to holomorphic maps with approximation and jet interpolation.

Setting	Typical conclusion	Representative paper
Local Taylor approximation	symmetry cancels lower-order terms	(Dellnitz, 2017)
Group-equivariant deep learning	equivariant architectures are dense in continuous equivariant maps	(Kumagai et al., 2020)
Constructive joint-equivariant machines	ridgelet transform gives explicit parameter distribution	(Sonoda et al., 2024)
Invariant–equivariant decomposition	equivariant approximation reduces to stabilizer-invariant approximation	(Sannai et al., 2024)
Symmetric definable sets	compact $G$-symmetric model $T$ maps equivariantly to $S$	(Basu et al., 2023)
Equivariant Oka theory	continuous equivariant maps deform to holomorphic equivariant maps	(Kutzschebauch et al., 2019)

A persistent structural idea is reduction of degrees of freedom. In one form, an equivariant map is determined by a generator on a base space of orbit representatives; in another, by invariant functions attached to stabilizers of output points; in yet another, by invariant polynomials or by charge-preserving local features. This suggests that equivariant approximation is usually not approximation plus a posteriori symmetrization, but an intrinsic approximation problem on a smaller symmetry-reduced space (Kumagai et al., 2020, Sannai et al., 2024).

2. Algebraic and local-analytic origin

A particularly concrete formulation appears in the observation that a real matrix $A\in\mathbb R^{n\times n}$ is self-adjoint, i.e. symmetric, if and only if it is equivariant with respect to a subgroup $\Gamma\subset \mathbf O(n)$ isomorphic to $\mathbb Z_2^n$ (Dellnitz, 2017). The construction starts from

$F[g\cdot x]=g\cdot F[x].$0

for which commuting with every $F[g\cdot x]=g\cdot F[x].$1 is equivalent to being diagonal. If $F[g\cdot x]=g\cdot F[x].$2, choose $F[g\cdot x]=g\cdot F[x].$3 with $F[g\cdot x]=g\cdot F[x].$4 diagonal and set

$F[g\cdot x]=g\cdot F[x].$5

Then $F[g\cdot x]=g\cdot F[x].$6 for all $F[g\cdot x]=g\cdot F[x].$7. Conversely, if $F[g\cdot x]=g\cdot F[x].$8 commutes with such a conjugate of $F[g\cdot x]=g\cdot F[x].$9, then $L^2$0 is diagonal and $L^2$1 is symmetric.

The paper uses this characterization to organize Taylor approximation for a smooth scalar function $L^2$2. Since the Hessian $L^2$3 is symmetric, there exists $L^2$4 such that $L^2$5 is $L^2$6-equivariant. For $L^2$7,

$L^2$8

Subtracting two such identities for $L^2$9 cancels the Hessian term and yields

$L^2$0

Hence the difference is $L^2$1. The paper describes this as a symmetry-based improvement in approximation, with odd-order Taylor terms canceled by the $L^2$2 pairing and the quadratic term made invariant by the Hessian symmetry. It does not formulate a general theorem under the title “Equivariant Approximation Theorem,” but it explicitly proposes this principle in the Hessian and fourth-order setting (Dellnitz, 2017).

3. Universal approximation of equivariant maps by neural networks

In deep learning, the phrase usually refers to density of symmetry-respecting architectures in spaces of continuous equivariant maps. One general formulation considers a locally compact, $L^2$3-compact, Hausdorff group $L^2$4 acting continuously on index spaces $L^2$5 and $L^2$6, with induced pullback actions on function spaces. A map $L^2$7 is $L^2$8-equivariant when $L^2$9. A base space $G$0 meeting each orbit exactly once yields a generator $G$1, and every equivariant map is determined by that generator. Moreover, the sup norm of the difference of two equivariant maps equals the sup norm of the difference of their generators. This converts equivariant approximation into ordinary approximation on the base space (Kumagai et al., 2020).

The corresponding architectural result is the conversion theorem: if an ordinary fully connected network $G$2 approximates the generator $G$3 on a compact set $G$4, then one can construct a $G$5-CNN $G$6 with the same number of layers such that $G$7 approximates the full equivariant map $G$8 to essentially the same accuracy. Under assumptions $G$9 and $T$0, this yields a two-layer universal approximation theorem for continuous equivariant maps in the finite-dimensional discrete case and a two-layer universal approximation theorem for $T$1-equivariant Lipschitz maps $T$2 in the infinite-dimensional setting. The framework explicitly covers the translation group $T$3, rotation groups such as $T$4, Euclidean groups $T$5 and $T$6, scaling by $T$7, and Lorentz-type actions on hyperbolic space (Kumagai et al., 2020).

A complementary line treats compact groups and finite-dimensional representations through invariant theory. For a compact group $T$8 acting linearly on finite-dimensional spaces $T$9 and $S$0, any continuous $S$1-invariant or $S$2-equivariant map can be approximated uniformly on compact sets by shallow architectures built from finitely many polynomial invariants $S$3 and polynomial equivariants $S$4. In the equivariant case, the approximants have the form

$S$5

The same work proves that basic convnets are universal for continuous translation-equivariant operators on $S$6, and that “charge-conserving convnets” are universal for continuous $S$7-equivariant signal transformations on $S$8 (Yarotsky, 2018).

These results address a common misconception. Universality in the equivariant class does not follow automatically from universality without symmetry; it requires architectures whose linear layers, nonlinearities, and feature spaces intertwine the relevant representations. The contribution of equivariant approximation theorems is precisely to show density inside the symmetry-constrained subspace, not merely in the ambient unconstrained function space (Kumagai et al., 2020, Yarotsky, 2018).

4. Structural reductions and constructive refinements

A stronger form of equivariant approximation replaces existential density by an explicit reconstruction formula. In the theory of joint-group-equivariant machines, a feature map $S$9 is joint-$A\in\mathbb R^{n\times n}$0-equivariant if

$A\in\mathbb R^{n\times n}$1

The associated machine

$A\in\mathbb R^{n\times n}$2

is then a $A\in\mathbb R^{n\times n}$3-equivariant operator from parameter distributions to functions. If $A\in\mathbb R^{n\times n}$4 is another joint-equivariant feature map and the induced representation $A\in\mathbb R^{n\times n}$5 on $A\in\mathbb R^{n\times n}$6 is irreducible, Schur’s lemma forces

$A\in\mathbb R^{n\times n}$7

where

$A\in\mathbb R^{n\times n}$8

This yields exact reconstruction, hence constructive universality, for shallow and deep joint-equivariant machines. Fully connected networks, group-convolutional networks, depth-$A\in\mathbb R^{n\times n}$9 joint-equivariant machines, and a depth-2 quadratic-form network all appear as special cases of this ridgelet-based framework (Sonoda et al., 2024).

Another structural reduction describes equivariant maps in terms of invariant maps on stabilizers. If $\Gamma\subset \mathbf O(n)$0 is the orbit decomposition of a $\Gamma\subset \mathbf O(n)$1-set, then there is a bijection

$\Gamma\subset \mathbf O(n)$2

The inverse reconstructs $\Gamma\subset \mathbf O(n)$3 from a stabilizer-invariant “seed” evaluated on a transformed input. This makes equivariant approximation a family of invariant approximation problems, one for each orbit representative. The same paper derives complexity inequalities comparing equivariant approximation cost with the invariant costs of the stabilizer components (Sannai et al., 2024).

Earlier finite-group work already exploited this orbit–stabilizer mechanism. For $\Gamma\subset \mathbf O(n)$4, a continuous $\Gamma\subset \mathbf O(n)$5-equivariant map $\Gamma\subset \mathbf O(n)$6 on a compact $\Gamma\subset \mathbf O(n)$7-stable set can be written in terms of scalar functions invariant under stabilizers of orbit representatives, and then approximated by $\Gamma\subset \mathbf O(n)$8-equivariant ReLU networks assembled from those stabilizer-invariant subnetworks. The paper also proves parameter-count reductions for $\Gamma\subset \mathbf O(n)$9-equivariant and $\mathbb Z_2^n$0-invariant models when the layer actions are unions of permutations, concluding that the resulting architectures have exponentially fewer free parameters than usual models (Sannai et al., 2019).

Taken together, these refinements shift the emphasis from bare density to mechanism. The generator theorem of group CNNs, the ridgelet reconstruction theorem for joint-equivariant machines, and the stabilizer decomposition of equivariant maps all identify a smaller unconstrained object from which equivariant approximation can be lifted back to the full symmetry class (Kumagai et al., 2020, Sonoda et al., 2024, Sannai et al., 2024).

5. Quantitative approximation rates under symmetry

Qualitative universality leaves open the rate at which approximation error decays as model size grows. Recent work studies this question for $\mathbb Z_2^n$1-Hölder invariant and equivariant targets. The guiding principle is that the effective complexity is governed by the quotient $\mathbb Z_2^n$2, not merely by the ambient dimension of $\mathbb Z_2^n$3. If $\mathbb Z_2^n$4 acts by isometries and the quotient $\mathbb Z_2^n$5 has covering numbers bounded by

$\mathbb Z_2^n$6

then $\mathbb Z_2^n$7 plays the role of an effective dimension (Siegel et al., 23 Feb 2026).

For permutation invariance on $\mathbb Z_2^n$8, generalized Deep Sets achieve approximation error controlled by the Hölder modulus $\mathbb Z_2^n$9, and the derived corollary states that the relevant exponent is $F[g\cdot x]=g\cdot F[x].$00. For permutation-equivariant maps $F[g\cdot x]=g\cdot F[x].$01, the paper shows that any continuous equivariant $F[g\cdot x]=g\cdot F[x].$02 admits a decomposition

$F[g\cdot x]=g\cdot F[x].$03

where $F[g\cdot x]=g\cdot F[x].$04 is permutation-invariant in its set argument, and then transfers the invariant approximation theorem to Sumformer-style and Transformer-style equivariant architectures. The same quotient-dimension exponent appears for frame-averaging models with joint invariance under permutations and rigid motions, and for architectures of the form $F[g\cdot x]=g\cdot F[x].$05 built from bi-Lipschitz invariant embeddings $F[g\cdot x]=g\cdot F[x].$06 (Siegel et al., 23 Feb 2026).

The paper’s explicit conclusion is that equally-sized ReLU MLPs and equivariant architectures are equally expressive over equivariant functions, and that hard-coding equivariance does not result in a loss of expressivity or approximation power in these models. This does not mean all equivariant architectures automatically attain the same rates; rather, the theorem is proved for specific classes, including the permutation-invariant Deep Sets architecture, the permutation-equivariant Sumformer and Transformer architectures, invariant networks based on frame averaging, and general bi-Lipschitz invariant models (Siegel et al., 23 Feb 2026).

A related finite-group result gives approximation rates for $F[g\cdot x]=g\cdot F[x].$07-equivariant ReLU networks by reducing equivariant approximation to invariant approximation for stabilizers. There again the rate analysis is mediated by the invariant–equivariant correspondence, although the most explicit quotient-dimension statements are developed in the later quantitative work (Sannai et al., 2024, Siegel et al., 23 Feb 2026).

6. Other mathematical meanings and adjacent theorems

Outside neural approximation theory, the phrase appears in several distinct branches of mathematics.

In $F[g\cdot x]=g\cdot F[x].$08-cohomology, an equivariant approximation theorem concerns $F[g\cdot x]=g\cdot F[x].$09-multiplicities of irreducible representations of a finite subgroup $F[g\cdot x]=g\cdot F[x].$10. If $F[g\cdot x]=g\cdot F[x].$11, the $F[g\cdot x]=g\cdot F[x].$12-th $F[g\cdot x]=g\cdot F[x].$13-multiplicity $F[g\cdot x]=g\cdot F[x].$14 is defined from the induced character $F[g\cdot x]=g\cdot F[x].$15. For a proper cocompact $F[g\cdot x]=g\cdot F[x].$16-CW-complex $F[g\cdot x]=g\cdot F[x].$17 and finite-index subgroups $F[g\cdot x]=g\cdot F[x].$18, the theorem states

$F[g\cdot x]=g\cdot F[x].$19

with normal, Farber-type, and sofic-style variants. This refines classical approximation theorems for $F[g\cdot x]=g\cdot F[x].$20-Betti numbers by replacing dimensions with multiplicities of $F[g\cdot x]=g\cdot F[x].$21-types (Kionke, 2017).

In equivariant Oka theory, the main theorem says that if a finite group $F[g\cdot x]=g\cdot F[x].$22 acts on a Stein manifold $F[g\cdot x]=g\cdot F[x].$23 and a $F[g\cdot x]=g\cdot F[x].$24-Oka manifold $F[g\cdot x]=g\cdot F[x].$25, then every $F[g\cdot x]=g\cdot F[x].$26-equivariant continuous map $F[g\cdot x]=g\cdot F[x].$27 is homotopic, through such maps, to a $F[g\cdot x]=g\cdot F[x].$28-equivariant holomorphic map. The finite-group theorem includes approximation on a $F[g\cdot x]=g\cdot F[x].$29-invariant $F[g\cdot x]=g\cdot F[x].$30-convex compact subset $F[g\cdot x]=g\cdot F[x].$31 and jet interpolation along a $F[g\cdot x]=g\cdot F[x].$32-invariant subvariety $F[g\cdot x]=g\cdot F[x].$33: intermediate maps can be chosen uniformly close on $F[g\cdot x]=g\cdot F[x].$34 and agreeing with the original map to order $F[g\cdot x]=g\cdot F[x].$35 along $F[g\cdot x]=g\cdot F[x].$36 (Kutzschebauch et al., 2019). Here “approximation” refers not to density of a network class, but to holomorphic approximation constrained by group symmetry.

In o-minimal and semialgebraic topology, symmetric definable sets can be approximated by compact symmetric models. If a finite reflection group $F[g\cdot x]=g\cdot F[x].$37 acts orthogonally on $F[g\cdot x]=g\cdot F[x].$38 and $F[g\cdot x]=g\cdot F[x].$39 is definable and $F[g\cdot x]=g\cdot F[x].$40-invariant, then the Gabrielov–Vorobjov compact approximation can be upgraded to produce a compact $F[g\cdot x]=g\cdot F[x].$41-symmetric set $F[g\cdot x]=g\cdot F[x].$42 and a $F[g\cdot x]=g\cdot F[x].$43-equivariant map $F[g\cdot x]=g\cdot F[x].$44. In the constructible case, $F[g\cdot x]=g\cdot F[x].$45 induces equivariant isomorphisms on homotopy and homology groups up to degree $F[g\cdot x]=g\cdot F[x].$46 and equivariant epimorphisms in degree $F[g\cdot x]=g\cdot F[x].$47; if $F[g\cdot x]=g\cdot F[x].$48, then $F[g\cdot x]=g\cdot F[x].$49 (Basu et al., 2023). This makes compact approximation compatible with $F[g\cdot x]=g\cdot F[x].$50-module structures such as Specht-module multiplicities for $F[g\cdot x]=g\cdot F[x].$51-actions.

Conceptually adjacent, though not an approximation theorem in the strict density sense, is the equivariant Laudenbach–Poénaru theorem. It states that any finite group action on $F[g\cdot x]=g\cdot F[x].$52 extends to a linearly parted action on $F[g\cdot x]=g\cdot F[x].$53, and any two such extensions are equivariantly diffeomorphic rel boundary (Meier et al., 17 Jan 2025). A plausible implication is that the broader family of “equivariant approximation theorems” includes not only density and compactification statements but also controlled equivariant extension theorems, provided the class of admissible extensions is rigid enough.

Across these disparate meanings, the unifying content is not a single formal statement but a shared strategy: encode symmetry first, then approximate, deform, compactify, or extend in a way that preserves the encoded action.