--- title: Local Canonicalization in Machine Learning url: https://www.emergentmind.com/topics/local-canonicalization type: topic --- # Local Canonicalization in Machine Learning Local canonicalization denotes a family of procedures that maps data to canonical coordinates defined relative to a sequence, node, agent, or individual test input, rather than relative to a single dataset-wide reference frame. In recent machine learning literature, this idea appears as per-sequence Generalized Procrustes alignment for deep non-rigid structure-from-motion, equivariantly predicted local frames for Lorentz- and Euclidean-equivariant networks, affine-invariant canonicalization for point clouds, agent-centered coordinate systems for swarm control, and test-time optimization over transformation parameters for robustness to affine distortions [2412.07230] [2505.20280] [2311.11106] [2509.14431] [2606.24178] [2509.26499]. Across these settings, the recurring objective is to remove nuisance transformations or rigid ambiguities before or during downstream inference, while preserving the task-relevant structure. ## 1. Conceptual scope and relation to global canonicalization Local canonicalization is not a single algorithm but a design pattern. In deep NRSfM, the canonical object is an entire sequence: each test or training sequence is aligned separately to its own canonical frame via Generalized Procrustes Analysis, in contrast to per-dataset canonicalization methods that force a single global canonical frame for all sequences [2412.07230]. In Lorentz-equivariant learning and O(3)-equivariant graph networks, the canonical object is usually a node-wise local frame, predicted equivariantly from local or relational geometry and used to convert tensorial features into invariant coordinates before applying a generic backbone [2505.20280] [2509.26499]. In swarm control, the canonical object is an agent-centered observation in which the focal agent is translated to the origin and rotated so that its velocity lies on the $+x$-axis [2509.14431]. In ShapeMatcher, by contrast, the method “does not attach an orthonormal triad to each point”; the locality lies in point-wise features that are invariant to global translation, rotation, and scale, while the explicit canonical frame is shared across the whole point cloud [2311.11106]. In zero-shot test-time canonicalization, the canonical object is the transformed test sample that minimizes an OOD score over a prescribed transformation family [2606.24178]. This literature also distinguishes local canonicalization from several adjacent ideas. “Global” canonicalization or alignment aligns an entire object once to a reference such as PCA axes, whereas local canonicalization uses sequence-wise or node-wise reference systems that adapt to the current sample [2509.26499]. Traditional equivariant GNNs and transformers bake the group action into every layer through specialized tensor-product operations; local canonicalization instead converts data into frame-invariant coordinates and then reuses ordinary MLPs, attention blocks, or message-passing layers [2505.20280] [2509.26499]. In LLoCa, data augmentation is explicitly identified as a special choice of reference frame rather than a distinct principle: fixing $L_i=\Lambda_0^{-1}$ for a random global $\Lambda_0$ recovers Lorentz data augmentation, while exact equivariance requires that the frame itself be learned equivariantly from the data [2505.20280]. | Setting | Canonical object | Mechanism | |---|---|---| | Deep NRSfM | Sequence-wise canonical shapes | GPA alignment | | Lorentz-equivariant networks | Node-wise local Lorentz frame | Predicted tetrads | | ShapeMatcher | Shared canonical point-cloud frame | Invariant features plus $S^c=s\cdot R\cdot S+T$ | | Swarm control | Agent-centered observation | Inverse $SE(2)$ pose action | | Test-time robustness | Canonicalized input sample | OOD-score minimization | | O(3) GNNs | Node-wise local frame | Predicted $R_i\in O(3)$ | ## 2. Mathematical structure A canonicalization can be formalized as a map $h:\mathcal{X}\to\mathcal{X}$ such that, for a transformation group $G$, one has $$ h(g\cdot x)=h(x)\quad \text{and}\quad h(x)\in G\cdot x. $$ When a classifier $f$ is composed with $h$, the resulting predictor $\phi(x)=f(h(x))$ is invariant under $G$ [2606.24178]. This formulation is used directly for test-time canonicalization, where one fixes a parameterized family $\{T_\theta\mid \theta\in\Theta\subseteq\mathbb{R}^d\}$ and solves $$ \theta^*=\arg\min_{\theta\in\Theta} E(T_\theta(x)), $$ with the energy instantiated by an OOD score [2606.24178]. In equivariant architectures, the same principle is expressed through local frames and representation maps. In LLoCa, a local Lorentz frame $L\in\mathbb{R}^{4\times 4}$ satisfies $L\,g\,L^\top=g$, where $g=\operatorname{diag}(1,-1,-1,-1)$ is the Minkowski metric, and under a global Lorentz transformation $\Lambda\in SO^+(1,3)$ the frame transforms as $L\to L'=L\Lambda^{-1}$. A local four-vector $x_l:=Lx$ is then invariant, since $x_l'=L'(\Lambda x)=Lx$. More generally, tensor features are canonicalized as $f_l=\rho(L)f$, processed by an arbitrary backbone, and mapped back to global coordinates by $y=\rho(L^{-1})f_l$, which guarantees exact equivariance [2505.20280]. The O(3) graph formulation uses the same algebra in node-wise form. With node features $F_i$ and predicted local frames $R_i\in O(3)$, the canonicalized features are $$ f_i=\rho_{\text{in}}(R_i)F_i. $$ Messages from $j$ to $i$ are built from the pair $$ \bigl(\rho_f(R_iR_j^{-1})f_j,\; R_i(x_i-x_j)\bigr), $$ which is invariant under a global rotation. Message passing can therefore be implemented by an ordinary MLP or attention block while preserving exact equivariance [2509.26499]. Other domains instantiate the same pattern with different transformation groups. ShapeMatcher explicitly reposes a point cloud by $$ S^c=s\cdot R\cdot S + T, $$ where $R\in SO(3)$, $T\in\mathbb{R}^3$, and $s\in\mathbb{R}^3$, while the learned point-wise features are invariant to global translation, rotation, and scaling [2311.11106]. LEGO defines, for each agent $i$, a canonicalization map $$ \phi_i:X\to C_i(X)=\rho_{SE(2)}((R_i,p_i)^{-1})X, $$ which produces the agent-centered observation $$ v_i'=[\|v_i\|,0],\qquad p_{j|i}=R_i^\top(p_j-p_i),\qquad v_{j|i}=R_i^\top v_j. $$ The local policy outputs $a_i^{loc}$, and the final action is de-canonicalized by $a_i=R_i a_i^{loc}$, yielding an $E(2)$-equivariant policy [2509.14431]. ## 3. Construction of local frames and canonical representations The literature differs most sharply in how the canonical frame or canonical representation is obtained. In deep NRSfM, the canonicalization is analytic rather than learned: after a sequence-level context layer produces refined shapes $\tilde{S}_i$, a GPA layer computes $$ \{\hat{R}_i\}_{i=1}^F=\arg\min_{R_i\in SO(3)}\sum_{i=1}^F \|R_i\tilde{S}_i-\bar{S}\|_F^2,\qquad \bar{S}=\frac{1}{F}\sum_{i=1}^F R_i\tilde{S}_i, $$ and outputs $\hat{S}_i=\hat{R}_i\tilde{S}_i$. Translation is implicitly removed by centering each $\tilde{S}_i$ before alignment [2412.07230]. LLoCa learns local Lorentz transformations through a small equivariant “Frames-Net” MLP $\phi$. For each node $i$, it predicts three four-vectors $$ v_{i,k}=\sum_{j=1}^n \operatorname{softmax}_j[\phi_k(s_i,s_j,\langle p_i,p_j\rangle)]\cdot (p_i+p_j),\qquad k=0,1,2, $$ where $p_i\in\mathbb{R}^4$ are four-momenta and $\langle p_i,p_j\rangle=p_i^\top g p_j$ are Minkowski inner products. These vectors are turned into an orthonormal tetrad by first defining the boost $B_i$ that sends $v_{i,0}$ to $(m,0,0,0)$ with $m=\sqrt{\langle v_{i,0},v_{i,0}\rangle}$, then applying $B_i$ to $v_{i,1},v_{i,2}$ and Gram–Schmidt orthogonalizing their spatial parts to build a spatial rotation $R_i$, and finally setting $L_i=R_iB_i$ [2505.20280]. In the O(3) setting, local frames are predicted by a small equivariant network $LFrames$ satisfying $$ R_i=LFrames(\{x_j:j\in N(i)\})\in O(3), $$ with the equivariance property $R_i(R\cdot x_j)=R_i(x_j)\cdot R$. The paper emphasizes that this can be implemented by a DG-equivariant MLP that outputs a $3\times 3$ orthogonal matrix [2509.26499]. ShapeMatcher uses a different strategy. A Vector-Neuron MLP outputs $$ R,T,F^* = VN\text{-}MLP(S)\in SO(3)\times\mathbb{R}^3\times\mathbb{R}^{n\times C}, $$ where $F^*$ is guaranteed $SE(3)$-invariant. A scale normalization head then predicts $$ s,F=\operatorname{normalize}(F^*), $$ so that $F$ is invariant under anisotropic or isotropic rescalings. The resulting features are point-wise and permutation-invariant, and no explicit $k$-NN or radius search is used [2311.11106]. LEGO constructs the local frame from physically interpretable geometry. The agent-centric $x$-axis is the normalized velocity $$ \hat{x}_i= \begin{cases} v_i/\|v_i\|, & \|v_i\|\neq 0\\ e_x, & \|v_i\|=0, \end{cases} $$ the reference vector is $d_i=c-p_i$ with center of mass $c=\frac{1}{N}\sum_{k=1}^N p_k$, and the $y$-axis is $$ \hat{y}_i=\operatorname{sgn}(\hat{x}_i^\top J d_i)\cdot J\hat{x}_i, $$ where $J=\begin{bmatrix}0&-1\\1&0\end{bmatrix}$. The rotation matrix is then $R_i=[\,\hat{x}_i\;\|\;\hat{y}_i\,]\in SO(2)$ [2509.14431]. ## 4. Integration with learning pipelines Local canonicalization is usually not an isolated preprocessing step; it is integrated into end-to-end pipelines with task-specific losses. In the NRSfM pipeline, a shared encoder $f(\cdot)$ and two heads $f_S,f_R$ first predict per-frame shapes and rotations, $$ S_i'=f_S(f(W_i);\Theta_S),\qquad R_i=f_R(f(W_i);\Theta_R). $$ The stacked shapes are mapped into a feature space $X=g(S')$, processed by a gated-Toeplitz module $H$ that uses both frame similarity and temporal order $L=[1\ldots F]^\top$ to produce a learnable coefficient matrix $C$, reconstructed as $\hat{X}=CX$, mapped back to 3D as $\tilde{S}=g_s(\hat{X})$, and then canonicalized by GPA. Training combines the reprojection loss $\|W-\Pi R\tilde{S}\|_2^2$ and the nuclear norm $\|\hat{S}\|_*$ on canonicalized shapes. Because the GPA layer has no parameters and uses a one-step gradient approximation, it disappears at test time; only the backbone and context layers remain [2412.07230]. LLoCa makes local canonicalization a wrapper around arbitrary backbones. The pipeline is: predict a local frame $L_i$, canonicalize features by $f_{L_i}=\rho(L_i)f$, process the invariant features with an arbitrary backbone network such as a transformer or GNN, and back-transform the outputs by $y=\rho(L_i^{-1})y_{can}$. For message passing, messages are built in the sender’s local frame and transferred to the receiver’s local frame through the invariant frame-shift $M_{ij}=L_iL_j^{-1}$, so that $$ m_{j,L_i}=\rho(M_{ij})m_{j,L_j}. $$ A special case is Lorentz-equivariant scaled dot-product attention, in which the keys and values are moved between local frames by $\rho(M_{ij})$ while the scalar products can be computed in the global frame [2505.20280]. The O(3) formulation follows the same architecture at graph level. Features are canonicalized once per node, neighbor features are re-rotated by $\rho_f(R_iR_j^{-1})$, local relative vectors $R_i(x_i-x_j)$ are encoded, and standard dense layers perform the actual message update. The framework compares scalar messages, Cartesian tensor representations, irreducible Wigner-$D$ representations, and a learned MLP representation, all within the same local-canonicalization backbone [2509.26499]. ShapeMatcher integrates canonicalization with segmentation, retrieval, and deformation. The canonicalization branch is supervised by a reconstruction branch that decodes $F^*$ back to $\hat{S}^c$ in canonical space using Chamfer distance, and by an orthogonality penalty $\operatorname{orth}(R)$. The canonicalization loss is $$ L_{can}=CD(S^c,\hat{S}^c)+\lambda\cdot\|R^\top R-I\|_2. $$ Training also includes cross-task consistency losses for partial and full inputs: $$ L_{ccan}=CD(S_{partial}^c,(S_{full}^c)\cdot U),\quad L_{cseg}=CD(F_{seg}^{(full)}\cdot U,F_{seg}^{(partial)}),\quad L_{ccen}=CD(K_{partial},K_{full}), $$ which couple canonicalization to segmentation and learned part centers [2311.11106]. Zero-shot test-time canonicalization is distinctive because the classifier is frozen. The canonicalizer is attached at inference time, evaluates an energy $S(T_\theta(x))$ for sampled or refined $\theta$, and transforms only the input. The framework benchmarks approximately twenty training-free OOD scores and nine search algorithms, and introduces two gates: a selection gate $$ \alpha_n(x)= \begin{cases} h(x), & S(x)>\tau\\ x, & \text{otherwise} \end{cases} $$ and an acceptance gate $$ \alpha_a(x)= \begin{cases} h(x), & S(h(x))