Canonical Embeddings and Registration

• Canonical embeddings are learned mappings that transform diverse data into a shared, invariant coordinate system for direct comparison.
• Registration methods, whether rigid via SVD or non-rigid through local linear approximations, optimize transformations for accurate alignment.
• These techniques drive applications in face verification, medical imaging, and object tracking, achieving high accuracy and computational efficiency.

Canonical embeddings and registration refer to machine learning and optimization methodologies that place points, shapes, or images into a shared, invariant coordinate system or space— a canonical embedding—where registration reduces to finding transformations (global or local, rigid or non-rigid) between instances, often via simple nearest-neighbor or linear operations. Across subfields, these shared representations (or maps into them) enable efficient and robust correspondence, shape or image alignment, motion compensation, re-identification, object tracking, and geometric reasoning. Canonical structures arise in a wide variety of domains, from deep face recognition and medical image registration to semantic 3D vision and non-rigid shape analysis, with current research focusing both on explicit parameterizations and learned representations.

1. Mathematical Definitions of Canonical Embeddings

A canonical embedding is a learned or determined function $f: X \to \mathbb{R}^d$ such that the representation $f(x)$ is geometrically or semantically aligned with respect to a reference system, enabling direct comparison (e.g., by distance or angle) across heterogeneous, transformed, or otherwise non-aligned data sources.

In face verification, for example, independent CNNs $f_1, f_2 : X \to \mathbb{R}^d$ (from different architectures or trainings) produce embeddings such that

$$f_1(x) \approx W_1 C(x),\quad f_2(x) \approx W_2 C(x)$$

for a shared "canonical" underlying embedding $C(x) \in \mathbb{R}^d$. This leads to an alignment (registration) property:

$$f_2(x) \approx W f_1(x), \quad W \in \mathbb{R}^{d \times d}$$

where $W$ can be unconstrained (linear), or restricted to orthogonal (rotation) transformations (McNeely-White et al., 2021).

In 3D medical image registration, canonical anatomical embeddings assign each voxel $x$ a dense feature vector $F_\mathrm{sam}(x)$ that is invariant to acquisition differences and anatomy-preserving, serving as coordinates in an emergent, learned anatomical space (Tian et al., 2023, Liu et al., 2021). For object-centric or non-rigid registration, canonical coordinate systems or embedding spaces ensure that semantically equivalent points map to common coordinates, regardless of shape deformation or sensor pose (Pozdeev et al., 4 Nov 2025, Gümeli et al., 2022, Sharma et al., 2021).

2. Estimating Registration Maps and Alignment

The core task in canonical embeddings and registration is to find the transformation mapping between representations of two instances (shapes, images, point clouds) such that their embeddings are as close as possible under a suitable error metric.

Rigid (Linear/Orthogonal) Alignment

Given embeddings $$A = [a_1^\top; ...; a_N^\top],\ B = [b_1^\top; ...; b_N^\top] \in \mathbb{R}^{N \times d}$$ from two sources, find:

• Unconstrained linear map $W^* = \arg\min_W \sum_i \|W a_i - b_i\|^2$
• Orthogonal (Procrustes) rotation $$R^* = \arg\min_{R^\top R = I} \sum_i \|R a_i - b_i\|^2$$

$R^*$ has the closed-form SVD solution:

$$M = A^\top B,\quad M = U \Sigma V^\top,\quad R^* = U \text{diag}(1,...,1,\det(UV^\top)) V^\top$$

This Procrustes/Kabsch method is numerically stable for moderate $d$ (e.g., $d=512$ in face verification) and computationally efficient (McNeely-White et al., 2021, Gümeli et al., 2022).

Non-Rigid and Locally Linear Registration

For non-rigid deformation or complex objects, registration is performed via locally linear transformations or learned pointwise correspondences:

• In the LTENet framework (He et al., 2022), for each source embedding $f(x_i)$, find $K$ nearest neighbors in the target, solve for weights $w_{i,l}$ in

$$f(x_i) \approx \sum_{l \in N^Y(i)} w_{i,l} f(y_l),\quad \sum_{l} w_{i,l} = 1$$

The cross-shape reconstruction $\hat{y}_i = \sum_l w_{i,l} y_l$ induces correspondences, regularized to align distributions (Cauchy-Schwarz divergence) as a probabilistic registration signal.

• In deep non-rigid shape matching (Sharma et al., 2021), the functional map paradigm is used: embeddings are arranged such that known self-symmetries and correspondences become linear in embedding space, optimizing:

$$\mathcal{L}_\text{tot} = \mathcal{L}_\text{euc} + \lambda \mathcal{L}_\text{lin} + \gamma \mathcal{L}_\text{comm}$$

with nearest-neighbor correspondence extraction at test time.

Dense Canonical Correspondence

Pixel- or voxel-wise mappings are learned using architectures such as transformers and UNet-style encoders (Tian et al., 2023, Liu et al., 2021, Pozdeev et al., 4 Nov 2025). For instance, DenseMarks predicts for each image pixel a 3D location in the canonical unit cube, supervised by tracked point matches, landmark, and segmentation constraints, enabling one-to-one dense registration via direct nearest-neighbor search in canonical space.

3. Training Methodologies and Loss Functions

Canonical embedding frameworks rely on a range of contrastive, correspondence, and reconstruction-based objectives, typically combining semantic, geometric, and smoothness constraints.

Contrastive, Correspondence, and Smoothness Losses

• Contrastive / InfoNCE Loss: Positive (matched) pairs are drawn together in embedding space, negatives are pushed apart, often at the pixel or point level (Pozdeev et al., 4 Nov 2025, Liu et al., 2021, Tian et al., 2023).
• Reconstruction/Alignment Losses: E.g., cross-reconstruction (LTENet), anchor-point regression for medical images (Hou et al., 2017), or landmark/segmentation losses (DenseMarks) promote semantic consistency across embedding fields.
• Regularization: Smoothness (e.g., bending energy, total variation), spatial continuity (as in 3D Gaussian smoothing of descriptors), and commutativity (via losses ensuring map compositions commute for symmetrical shapes (Sharma et al., 2021)) further stabilize alignment.

Architectural Choices

Encoders are domain-specific: 3D UNets or P-HNN (Liu et al., 2021, Tian et al., 2023) for volumes; PointNet or DGCNN for point clouds (Sharma et al., 2021, He et al., 2022); vision transformers for 2D images (Pozdeev et al., 4 Nov 2025). Decoder heads may upsample to full spatial resolution or predict auxiliary outputs (canonical coordinates, symmetries, instance properties).

4. Applications Across Domains

Face Verification

Canonical face embeddings yield strong cross-model compatibility: linear or orthogonal transformation between ten diverse CNNs allows matching embeddings with minor (<5%) loss in verification rate, e.g., True Accept Rate drops from 0.96 to 0.91 at FAR=1e-2 under rotation alignment (McNeely-White et al., 2021). Such compatibility enables cross-model matching but also surfaces security risks (de-anonymization, template attacks).

Medical Image Registration

Self-supervised anatomical embeddings (SAM, SAME) and their extensions (SAME++) serve as canonical spaces for inter-subject, inter-modal, and even cross-contrast registration. These achieve state-of-the-art Dice improvements (4.2–8.2% over baselines, running ≪1 min/scan), with exceptional organ-wise accuracy and substantial speedups over numerical or classical methods (Tian et al., 2023, Liu et al., 2021).

Non-Rigid and Partial Shape Matching

Pointwise canonical embeddings for 3D shapes (LTENet, deep functional map, etc.) enable dense correspondences with minimal reliance on spectral decompositions or eigenfunctions. Methods achieve mean geodesic errors ($\sim$0.05 for FAUST, $0.10/0.13$ for SHREC Holes/Cuts) that outperform prior spectral or learned-alignment baselines (Sharma et al., 2021, He et al., 2022).

Object-Centric and Scene Registration

Canonical mappings of objects into normalised coordinate spaces (NOC) drive semantic SLAM and object-centric camera registration in the presence of low inter-frame overlap. This yields recall improvements (e.g., from 24% to 45% at ≤10% overlap) and up to 50% SLAM trajectory error reduction over point-feature methods (Gümeli et al., 2022).

Dense Tracking and Geometric Reasoning

DenseMarks provides a direct map from RGB pixels to a 3D canonical space for all head regions, delivering geometry-aware matching MAE 3.68 px (vs. 7.6–14.88 for prior work) and enabling smooth, robust monocular head tracking even under occlusion, pose change, and appearance variation (Pozdeev et al., 4 Nov 2025).

5. Computational and Statistical Properties

Sample and Compute Efficiency

• Rigid alignment via SVD requires $\sim$256–1000 pairs for high-quality rotation estimation in 512d (McNeely-White et al., 2021).
• Medical registration pipelines (SAME, SAME++) achieve inference in 1–20 ms per slice (SVRNet (Hou et al., 2017)), or seconds per full volume (Tian et al., 2023).
• Non-rigid shape pipelines (LTENet) scale efficiently to large datasets/training sizes, with test-time nearest-neighbor search in low-dimensional canonical space (He et al., 2022, Sharma et al., 2021, Pozdeev et al., 4 Nov 2025).

Robustness and Limitations

Canonical embeddings demonstrate resilience to pose, anatomical, and partiality variations. Nevertheless:

• The learned canonical space may inherit biases from training corpora.
• In cross-domain or multi-modal scenarios, invariance is limited by embedding capacity and augmentation schemes.
• Security risks: cross-model alignment exposes biometric templates to de-anonymization and inversion attacks (McNeely-White et al., 2021).
• For object-centric scene registration, absence of mapped objects degrades to keypoint-based fallback (Gümeli et al., 2022).

6. Implications, Security, and Future Directions

Canonical embedding alignment reveals that deep models for tasks such as face verification or shape matching learn to sample a common low-dimensional manifold, often up to rotation or local linear transformation. This phenomenon suggests hard limits to architecture-based progress without new forms of supervision.

Security considerations are prominent: the possibility to linearly map between embedding spaces underlines the sensitivity of biometric templates (McNeely-White et al., 2021). Recommended mitigation includes secret or encrypted nonlinear embedding transforms, limiting exposure to global linear mapping.

Emergent research themes:

• Extending canonical alignment to broader deformations, multi-modal images, cross-population variance, and fine-grained anatomical or semantic regions.
• Bridging canonical embedding methodologies with part-level or jointly-learned alignment for complex objects and heterogeneous datasets (Gümeli et al., 2022, Pozdeev et al., 4 Nov 2025).
• Exploiting the efficiency and robustness of canonical registration in real-time and low-data regimes (e.g., fetal MRI pose estimation, in-the-wild head tracking (Hou et al., 2017, Pozdeev et al., 4 Nov 2025)).
• Dynamic or self-refining canonical embeddings leveraging downstream registration feedback (Liu et al., 2021).

Canonical embeddings and registration are foundational tools for robust, scalable, and semantically meaningful correspondence in computer vision, medical imaging, and geometric learning, characterized by shared mathematical frameworks, strong empirical performance, and distinct implications for model interoperability and information security.