Recurrent Equivariant Constraint Modulation

• RECM is a mechanism that autonomously modulates symmetry constraints in neural networks by balancing strictly equivariant and unconstrained behaviors using a recurrent state.
• It integrates parallel branch architectures where modulation weights are learned from data-driven symmetry violation estimates, enabling adaptive recovery of exact equivariance or controlled symmetry breaking.
• Empirical results across classification, physical simulations, and molecular tasks demonstrate that RECM flexibly improves performance by aligning constraint enforcement with the inherent symmetry of the training data.

Recurrent Equivariant Constraint Modulation (RECM) is a mechanism for learning per-layer relaxation of symmetry constraints in equivariant neural networks. Unlike prior approaches that require explicit or hand-tuned target relaxation levels, RECM autonomously modulates the degree of equivariance at each layer based entirely on the observed data and symmetry properties of the distribution passing through the network. This enables adaptive recovery of strict equivariance when warranted by perfect symmetry in the data distribution and automatic symmetry breaking when the symmetry is only approximate or absent (Pertigkiozoglou et al., 2 Feb 2026).

1. Formal Motivations and Core Principle

Equivariant neural networks encode task symmetries by enforcing constraints that guarantee commutation between the group action on inputs and outputs. Formally, for a group $G$ with representations $\rho_{in}(g)$ and $\rho_{out}(g)$, a layer $f:\mathbb{R}^n \to \mathbb{R}^k$ is strictly equivariant if

$$f(\rho_{in}(g)x) = \rho_{out}(g)\,f(x),\quad \forall g \in G,\, x \in \mathbb{R}^n.$$

For linear $f(x)=Wx$, this implies strict intertwiner constraints $W \rho_{in}(g) = \rho_{out}(g) W$ for all $g$.

Strict equivariance can fragment the optimization landscape, inhibiting effective learning, and, in some cases, unconstrained models empirically outperform strictly equivariant counterparts even for tasks with exact symmetry. Prior attempts to address these issues require manual specification of the symmetry relaxation per layer, which is both costly and task-dependent.

RECM proposes to learn the modulation weights $\{\alpha^{(l)}_i, \beta^{(l)}\}$ for each layer $l$ directly from the training objective, writing the layer’s output as an affine combination of strictly equivariant and fully unconstrained submodules:

$$f^{(l)}(z) = \beta^{(l)} f^{(l)}_{eq}(z) + \sum_{i=1}^K \alpha^{(l)}_i f^{(l)}_{un,i}(z).$$

The mechanism ensures that unconstrained components are suppressed when the data are fully symmetric, and retained when beneficial flexibility is warranted by approximate or broken symmetry.

2. Mathematical Foundation

RECM maintains a trainable, recurrent “state” vector $h_t \in \mathbb{R}^m$ per layer, summarized at iteration $t$ as

$$h_t = (1-\frac{a}{b+at}) h_{t-1} + \frac{a}{b+at} \ell_{\theta_t}(z_{t-1}, y_{t-1}),$$

where $\ell_{\theta_t}(z, y)$ is a data-driven estimator of symmetry violation, and $a$, $b$ are decay parameters. The modulation coefficients are then computed as

$$\alpha_{i,t} = s(w_{\alpha_i}^T h_t), \quad \beta_t = k(w_\beta^T h_t),$$

with $s(\cdot)$ and $k(\cdot)$ nonlinearities (e.g., GeLU) satisfying $s(0)=0$, $k(0)=1$, and bounded parameter norms.

A key theoretical concept is the per-layer symmetry gap $\Delta^{(l)}$, measured as the 1-Wasserstein distance between the actual per-layer (input, target) joint distribution $p$ and its group-symmetrized counterpart $p_G$, i.e.,

$\Delta^{(l)} := W_1(p, p_G),$

where $p_G(q) = \int_G p(T_g^{-1}q) \, d\mu_G(g)$ with $T_g(z, y) := (\rho_{in}(g)z, \rho_{out}(g)y)$.

The RECM update and state tracking provably guarantee that, under mild regularity assumptions, the steady-state unconstrained weights are upper bounded as

$|\alpha_i^*| \leq 2B \Delta^{(l)}$

where $B$ is the Lipschitz constant of the underlying estimator. Therefore, in the case of exact symmetry ($\Delta^{(l)}=0$), the unconstrained branch weights vanish and strict equivariance is recovered.

3. Architecture and Algorithm

Each learnable layer in a RECM-augmented network consists of parallel strictly equivariant and unconstrained branches. The architecture is augmented per-layer with:

• A vector-valued recurrent state $h^{(l)}$.
• Learnable parameters for the modulator ($w_{\alpha_i}^{(l)}, w_\beta^{(l)}$) and the symmetry-violation estimator (parameterized by an MLP with weights $\theta^{(l)}$).

The per-iteration update consists of:

1. Calculating the symmetry-violation score $\ell^{(l)}_{\theta_t}(z, y)$.
2. Updating the hidden state $h^{(l)}$ as an exponential moving average.
3. Computing modulation weights $\alpha^{(l)}_{i,t}$ and $\beta^{(l)}_t$ from the updated state.
4. Producing the forward layer output:

$$f^{(l)}(z) = \beta^{(l)}_t W^{(l)}_{eq} z + \sum_{i=1}^K \alpha^{(l)}_{i,t} W^{(l)}_{un,i} z.$$

1. Loss is backpropagated to update all parameters, including the modulator, estimators, and branch weights.

The following pseudocode outlines the RECM training loop:

initialize all layer weights W_eq, W_un, θ, w_α, w_β for t in 1…T: sample mini-batch {(x_i, y_i)} for l in 1…L: z^{(l)} = output of previous layer if t > 1: h^{(l)} ← (1 - a/(b + a (t−1))) h^{(l)} + (a/(b + a (t−1))) ℓ_{θ^{(l)}(z^{(l)},y)} α_i^{(l)} ← s(w_{α_i}^{(l)T} h^{(l)}) β^{(l)} ← k(w_β^{(l)T} h^{(l)}) z^{(l+1)} ← β^{(l)} W_eq^{(l)} z^{(l)} + ∑_i α^{(l)}_i W_{un,i}^{(l)} z^{(l)} compute loss L({z^{(L+1)}, {y}) backpropagate and update W_eq, W_un, θ, w_α, w_β

Architecturally, this requires only the addition of unconstrained branches and a small MLP per layer; existing nonlinearities and global configuration remain unchanged (Pertigkiozoglou et al., 2 Feb 2026).

4. Empirical Performance and Benchmarks

RECM was evaluated across four domains encompassing both strict and approximate symmetry scenarios:

Task	Base / Comparison Methods	RECM Metric (↑/↓)	Best Baseline
ModelNet40 Classification	VN-PointNet, +ES, +RPP	0.80/0.74 (Rot/Align Inst.)	+RPP: 0.77/0.71
N-body SO(3) Prediction	SEGNN, +ES, +ACE-exact, +ACE-appr, EGNN, EGNO, SE(3)-Tr.	3.7 (MSE ↓)	+ACE: 3.8
Motion Capture Trajectory	EGNO, +ES, +ACE-exact, +ACE-appr, SE(3)-Tr., TFN, EF, EGNN	22.6/6.6 (MSE ↓)	+ACE: 23.8/7.4
Molecular Conformer (GEOM)	ETFlow-Eq/Unc, DiTMC-Eq/Unc, MCF, GeoDiff, GeoMol, Torsional Diff	80.6/85.5 (Recall/Precision)	DiTMC-Eq: 80.8/85.6

Empirical studies show that RECM generally outperforms or matches baselines on both exact and approximate equivariant tasks. Ablations indicate that in fully symmetric cases, all $\alpha$ weights approach zero, automatically restoring strict equivariance; when symmetry is only partial, some $\alpha$ remain significantly positive, reflecting data-driven equivariance breaking (Pertigkiozoglou et al., 2 Feb 2026).

5. Mechanistic Insights and Theory

RECM’s adaptive modulation is theoretically underpinned by the per-layer symmetry gap $\Delta^{(l)}$. In the limit, $\alpha^{(l)}_i$ is strictly upper-bounded by $2B\Delta^{(l)}$, where $B$ is a known Lipschitz constant. Therefore, strictly equivariant solutions are recovered when the training distribution is invariant, while non-equivariant flexibility emerges only as prescribed by symmetry violation in the data.

RECM’s modulation dynamics hinge on the expressivity of the symmetry-violations estimator and regularity of the update (assumptions: uniform Lipschitzness and learning rate decay). The required group-generating set $C_G$ is finite and typically small for compact groups such as SO(3); in practice, 2–6 group elements suffice.

A plausible implication is that RECM can support architectures with varying degrees of symmetry structure without relying on external heuristics or domain knowledge to specify relaxation levels.

6. Limitations and Open Research Directions

RECM’s convergence theory presupposes sufficiently expressive estimators and proper scheduling of learning rates. In practice, it incurs moderate additional training overhead ($\approx 30$–$50\%$ increase) due to parallel branches and the per-layer MLP, though inference can be pruned when $\alpha_i<\epsilon$.

Current limitations include:

• The requirement to select a finite group generator subset $C_G$.
• Applicability focused on compact groups; extension to non-compact groups (e.g., translations $\mathbb{R}^d$) or local symmetry relaxations remains unresolved.
• Theoretical and practical behaviors in deep architectures or with large/continuous symmetry groups are not fully characterized.

Open questions concern the interaction of RECM with very deep networks, dynamics under large or continuous groups, and the potential for higher-order or adaptive updates to the hidden state to enhance convergence.

7. Context and Comparative Perspective

RECM’s data-driven, layer-wise modulation of equivariance addresses longstanding challenges associated with rigid symmetry enforcement in neural architectures. Prior approaches, including scheduled equivariance scheduling (ES), Residual-Pathway (RPP), and ACE-based (exact/appr) constraint relaxation, require tuning of per-layer targets and demonstrate sensitivity to hyperparameters and symmetry gap mis-specification.

By directly linking per-layer flexibility to the measured input-target distribution symmetry, RECM provides a principled solution for both recovering strict equivariance and deploying controlled symmetry breaking, as supported by empirical results on molecular, physical, and pose-prediction tasks (Pertigkiozoglou et al., 2 Feb 2026).