Deformable Face Registration Module (DAM)

• DAM is a module that performs dense, nonlinear facial alignment by estimating deformation fields to map source facial representations onto targets.
• It underpins state-of-the-art approaches in face restoration, landmark localization, and 3D face morphometrics through various architectures such as U-Net, LDDMM, and TPS.
• DAM is trained using self-supervised objectives like local normalized cross-correlation and energy minimization, ensuring robust performance across diverse applications.

A Deformable Face Registration Module (DAM) is an architectural or algorithmic component designed to perform dense, nonlinear registration between two facial representations, explicitly modeling and compensating for geometric variability. DAMs underpin a range of state-of-the-art approaches in face restoration, dense landmark localization, and 2D/3D face alignment. They operate by estimating, learning, or optimizing a deformation field or parameterized mapping, enabling fine-grained correspondence between source and target facial structures or images.

1. DAM in Generative Face Restoration Pipelines

In advanced blind face restoration (BFR) frameworks, such as CodeFormer++ (Reddem et al., 6 Oct 2025), DAM is implemented as a learning-based image alignment module. It is responsible for "semantically stitching" two distinct face images:

• $I_F$: an identity-preserving but over-smoothed reconstruction (from the restoration branch),
• $I_G$: a high-quality, detail-rich face (from the generative branch), which may suffer from identity drift.

DAM predicts a dense, nonlinear displacement field $\phi \in \mathbb{R}^{512 \times 512 \times 2}$ that warps $I_G$ into the coordinate scaffold of $I_F$ via

$I_{\mathrm{warp}}(x) = I_G(x + \phi(x)).$

This operation transfers the high-frequency texture of $I_G$ while maintaining geometric agreement with $I_F$. The resulting pair $(I_F, I_{\mathrm{warp}})$ is fused in downstream networks, such as Texture-Prior Guided Restoration Networks, which leverage both identity and fidelity cues.

DAM is trained with a self-supervised objective: local normalized cross-correlation (NCC) for appearance matching and a smoothness regularizer on $\phi$. No external priors or keypoint supervision are required, rendering DAM both versatile and directly pluggable into two-stream image restoration pipelines (Reddem et al., 6 Oct 2025).

2. DAM via Large-Deformation Diffeomorphic Metric Mapping (LDDMM)

LDDMM-Face (Yang et al., 2021) implements DAM as a differentiable deformation layer grounded in LDDMM theory. Unlike regression-based or keypoint-matching approaches, this formulation leverages the formalism of geodesics on the diffeomorphism group to align facial boundaries and landmarks in a theoretically consistent and topology-preserving manner. DAM receives initial momenta $\alpha(0)$ parameterizing the registration flow, predicts a time-dependent velocity field $v_t$, and integrates Hamiltonian ODEs to generate a diffeomorphic flow $\varphi_t$. Curves (such as facial outlines) and sets of landmarks are propagated by the flow, facilitating registration at both global and local levels.

The energy functional minimized by DAM is

$$E(v) = \int_0^1 \|v_t\|_V^2\,dt + \frac{1}{\sigma^2} D(\varphi_1 \cdot C_0, C_{gt}),$$

where $D$ encodes discrepancies of both curve shapes (via moment embeddings in a dual RKHS $W^*$) and landmark positions. The initial momenta are learned via a regression head connected to a CNN backbone, allowing the entire geodesic registration operation to remain fully differentiable and compatible with standard deep learning toolchains (Yang et al., 2021).

3. Classical DAM by Landmark-Based Non-Rigid Registration

Extending DAM to 3D facial surfaces, Guo et al. (Guo et al., 2012) described a pipeline for fully automatic landmark annotation and dense correspondence registration. Their DAM consists of:

1. Automatic Landmark Annotation:
   • Salient landmarks (inner/outer eye corners, mouth corners) located by 2.5D PCA-based detection after frontal pose normalization.
   • Secondary landmarks localized by geometric or color-based heuristics.
2. Thin-Plate Spline (TPS) Registration:
   • Given 17 landmarks per face, DAM computes a TPS mapping $f:\mathbb{R}^3 \to \mathbb{R}^3$ by minimizing bending energy while interpolating all landmarks.
   • The reference face is remeshed, warped via TPS, and correspondences are extracted by nearest-neighbor search.

This pipeline achieves mean Euclidean landmark errors of 0.8–1.5 mm (up to 2.8 mm for earlobe points), robust performance across ethnicities, and enables high-throughput 3D face morphometrics (Guo et al., 2012).

4. Network Architectures and Algorithmic Structures

The architectural instantiation of DAM varies with modality:

• Fully Convolutional DAM (CodeFormer++):
  • A U-Net comprising four encoder/decoder levels, skip-connections, and channel dimensions scaling from 32 to 256.
  • Inputs: concatenation of two 512×512 RGB images (shape 512×512×6).
  • Outputs: dense flow field $\phi$ and the warped image.
  • Bilinear warping ("spatial transformer"), weight normalization, and LeakyReLU activations (Reddem et al., 6 Oct 2025).
• LDDMM-Based DAM:
  • Backbone CNN (e.g., HRNet, Hourglass), momentum regression head (FC layers), and differentiable ODE integration (RK4/Euler).
  • Flow field defined implicitly by kernels on landmark/curve control points; suitable for both sparse and dense annotation schemes (Yang et al., 2021).
• Classical Landmark/TPS DAM:
  • PCA-based patch detectors, heuristic modules for less-salient landmarks, and exact TPS solving without explicit regularization (17 points stabilize the fit).
  • Mesh remeshing and fast spatial index queries for dense correspondences (Guo et al., 2012).

5. Training, Losses, and Self-Supervision

DAMs are typically trained in a fully self-supervised or weakly-supervised regime:

• CodeFormer++: Local NCC loss $\mathcal{L}_{\mathrm{sim}}$ between $I_F$ and the warped $I_G$, plus smoothness regularization on $\phi$; no flow ground truth used (Reddem et al., 6 Oct 2025).
• LDDMM-Face: Inexact LDDMM energy penalizing both deformation norm (via kernel RKHS) and data-term in joint curve + landmark space, normalized by interocular distance; benefits from direct backpropagation through ODE integration (Yang et al., 2021).
• PCA+TPS (3D Faces): PCA detectors trained on manually annotated samples; TPS fitting is analytic, with no learned parameters beyond PCA basis; inherently unsupervised for dense registration (Guo et al., 2012).

6. Quantitative Impact and Evaluation

Empirical evaluation metrics vary by application:

System	Evaluation Metric	Score/Observation	Source
CodeFormer++ + DAM	Landmark Distance (LMD)	5.72 px (vs 6.28 px without DAM)	(Reddem et al., 6 Oct 2025)
	NIQE (perceptual quality)	4.136 (no degradation post DAM)	(Reddem et al., 6 Oct 2025)
3D DAM (Guo et al.)	Euclidean landmark error	0.8–1.5 mm (most); 2.8 mm (earlobe)	(Guo et al., 2012)
3D DAM (Guo et al.)	Registration accuracy	<0.9 mm error over >90% of surface (average faces)	(Guo et al., 2012)

Performance studies reveal that DAM consistently reduces geometric misalignment while maintaining—if not improving—appearance fidelity. CodeFormer++'s ablation demonstrates that DAM corrects major structural mismatches but leaves artifact suppression to subsequent fusion networks. In 3D face registration, DAM exhibits high speed and cross-ethnic robustness (Reddem et al., 6 Oct 2025, Guo et al., 2012).

7. Synthesis, Extensions, and Limitations

DAM is a highly modular concept, adaptable across domains from 2D generative restoration to 3D morphometric analysis:

• Learning-based DAMs (U-Net, LDDMM) yield effective, plug-and-play registration for deep facial pipelines.
• Classical TSP/PCA DAMs afford interpretable, analytic mappings, well-suited to mesh-based registration.

Extensions include replacing heuristic detection steps with learning-based landmark regressors, expanding landmark sets for expression invariance, and incorporating temporal or multi-view smoothness constraints. In LDDMM-based DAM, the same learned diffeomorphic flow can propagate arbitrary annotation sets without retraining, enabling unprecedented flexibility across datasets and protocols (Yang et al., 2021).

DAM modules do not require external priors, ground-truth flow, or keypoint annotation at test time. A persistent limitation is the incomplete suppression of fine-grained local artifacts in some architectures, which are typically resolved at subsequent texture fusion or refinement stages. For highly occluded or pathological faces, heuristic-based DAMs may require re-tuning or augmentation (Reddem et al., 6 Oct 2025, Guo et al., 2012).