DINOReg: DINO-based Registration Methods

• DINOReg is a family of registration methods that use pretrained DINO-family models to extract visual features for aligning images and 3D point clouds.
• It combines training-free and learned optimization strategies, integrating techniques like ConvexAdam and PCA for coarse-to-fine registration.
• Empirical results show that DINOReg approaches improve accuracy and efficiency, outperforming traditional handcrafted and deep-learning registration methods.

Searching arXiv for the relevant DINOReg papers and closely related context. DINOReg, also written as DINO-Reg in some publications, denotes a set of registration methods that use DINO-family vision foundation models as feature extractors for alignment problems. The name has been applied to at least three distinct systems: a fully training-free deformable medical image registration pipeline based on frozen DINOv2 features and ConvexAdam (Song et al., 2024), a multi-modal rigid point cloud registration network that fuses DINOv2 image features with geometric features (Chen et al., 29 Sep 2025), and a later training-free medical image registration pipeline that replaces DINOv2 with a frozen DINOv3 encoder and performs test-time optimization in a compact feature space (Wang et al., 20 Aug 2025). Across these usages, the common theme is the use of pretrained visual representations to supplement or replace modality-specific handcrafted descriptors and dataset-specific supervised encoders.

1. Terminology and scope

The term is not attached to a single canonical architecture. Instead, it has been reused for separate registration systems in different subfields. This creates a nomenclature issue: references to “DINOReg” must be interpreted in context rather than assumed to identify one standardized method.

Name	Domain	Defining formulation
DINO-Reg (Song et al., 2024)	Deformable medical image registration	Training-free DINOv2 feature extraction with ConvexAdam and optional MIND integration
DINOReg (Chen et al., 29 Sep 2025)	Rigid point cloud registration	DINOv2 visual embeddings fused with KPConv-FPN geometric features and a Visual-Geometric Transformer
DINOv3+T³ / DINOReg (Wang et al., 20 Aug 2025)	Deformable medical image registration	Frozen DINOv3 features, PCA reduction, and test-time deformation optimization

A practical consequence is that “DINOReg” can refer either to a test-time-optimized deformable registration pipeline or to a trained coarse-to-fine matching network, depending on the paper. This suggests that the label functions more as a family resemblance around DINO-based registration than as a fixed technical standard.

2. DINO-Reg for deformable medical image registration

In its original medical-imaging usage, DINO-Reg is defined as a fully training-free, deformable medical image registration pipeline that extracts global and local features from 3D medical volumes using the pretrained DINOv2 teacher network and feeds those features into a discrete optimizer to compute a dense deformation field (Song et al., 2024). The method is positioned between purely handcrafted-feature registration and end-to-end deep learning: unlike handcrafted metrics such as MIND or mutual information, it is intended to capture high-level semantics; unlike methods such as Voxelmorph, it requires no retraining or manual segmentation for each modality.

The feature extractor is the teacher network of DINOv2 (Vision Transformer ViT-L/14, embedding dimension 1024) pretrained on 1.3M ImageNet images, used without any finetuning. Each single-channel CT or MR slice is converted into a 3-channel RGB image, resized from 256×192 to 728×560, partitioned into non-overlapping 14×14 patches, and augmented with one class token and four “register tokens.” The resulting output contains 2080 tokens (2048 patch tokens + 1 class + 4 register), each a 1024-dimensional feature vector.

For a 3D volume of size $H \times W \times D$, one orthogonal view is selected, axial by default, and all 2D slices along that view are extracted and upsampled by a factor of 3. DINOv2 then produces a slice-wise feature map $F_{2D}(x) \in \mathbb{R}^{h \times w \times C}$ with $C=1024$, and stacking over slices yields a 4D feature volume $F \in \mathbb{R}^{h \times w \times d \times C}$. Dimensionality is reduced by applying PCA (or low-rank randomized PCA) jointly to the concatenated tokens from the reference and moving volumes, with $k=24$ retained components, producing reduced feature volumes in $\mathbb{R}^{h \times w \times d \times k}$.

The registration objective is written as

$E(\phi) = S(f, m \circ \phi) + \lambda R(\phi),$

where $\phi$ is the dense deformation field, $f = F(I_{\mathrm{ref}})$ and $m = F(I_{\mathrm{mov}})$ are the extracted feature maps, and $F_{2D}(x) \in \mathbb{R}^{h \times w \times C}$0 is a smoothness penalty such as bending energy or total variation. The similarity term is instantiated either as a feature-space SSD,

$F_{2D}(x) \in \mathbb{R}^{h \times w \times C}$1

or as feature-wise Local Cross-Correlation (LCC) for final local refinement. The paper states that this was, to its knowledge, the first application of general vision foundation models in medical image registration.

3. ConvexAdam optimization, handcrafted integration, and empirical behavior

The optimization backbone follows the two-stage scheme of Siebert et al. (Fast 3D Registration with Accurate Optimisation): a Global Convex Registration step produces an initial field $F_{2D}(x) \in \mathbb{R}^{h \times w \times C}$2, followed by Local Refinement with Adam on a non-convex LCC objective (Song et al., 2024). In the convex step, a discrete label set $F_{2D}(x) \in \mathbb{R}^{h \times w \times C}$3 represents candidate displacements, for example $F_{2D}(x) \in \mathbb{R}^{h \times w \times C}$4 voxels in steps of 1 voxel, and the cost volume is computed as

$F_{2D}(x) \in \mathbb{R}^{h \times w \times C}$5

Soft assignments $F_{2D}(x) \in \mathbb{R}^{h \times w \times C}$6 are then optimized under a spatial smoothness term on the voxel grid graph, after which the discrete solution is converted into a continuous initial field through

$F_{2D}(x) \in \mathbb{R}^{h \times w \times C}$7

The Adam refinement initializes $F_{2D}(x) \in \mathbb{R}^{h \times w \times C}$8, repeatedly warps the moving feature map, evaluates

$F_{2D}(x) \in \mathbb{R}^{h \times w \times C}$9

and updates $C=1024$0 with the Adam optimizer. The medical DINO-Reg formulation is therefore not merely a feature extractor but a feature-space registration system whose performance depends on the interaction between the frozen encoder, dimensionality reduction, and discrete-continuous optimization.

A central empirical observation is that DINOv2 features capture strong global semantics but can blur fine organ boundaries, whereas MIND descriptors excel at local contours. This motivates two ensemble strategies. In Field Averaging (“DINO-RegEn”), DINO-Reg and MIND+ConvexAdam are run separately and their displacement fields are averaged:

$C=1024$1

In Sequential Refinement, $C=1024$2 is used as global initialization and MIND+Adam performs local fine-tuning; the paper reports that this yielded the highest Dice in ThoraxCBCT experiments.

On the OncoReg Challenge hidden test set, the reported leaderboard values are: DINO-RegEn with TRE 3.51, TRE30 6.66, Dice 0.6225, sdLogJ 0.0394, Score 0.742, Runtime >300 s; DINO-Reg with TRE 3.88, TRE30 7.26, Dice 0.5983, sdLogJ 0.0310, Score 0.656, Runtime >300 s. The ensemble achieved first place (Score=0.742). Runtime was later reduced from >300 s to ~60 s per case by using low-rank PCA and slice interpolation.

The ThoraxCBCT ablations clarify several implementation choices. NCC (50 epochs) outperformed SSD (800 epochs) in TRE and Dice. Low-rank randomized PCA matched full PCA in accuracy while being ~50× faster. Encoding every 3rd slice with interpolation preserved registration quality while reducing encoding time. The axial view yielded the best Dice (0.733) and captured the most variance in the first 3 PCs. These findings reinforce the view that the registration quality derives from both the transferred semantics of DINOv2 and the engineering of the feature-space optimizer.

4. DINOReg for point cloud registration

In point cloud registration, DINOReg is a different method: a registration network for estimating a rigid transformation $C=1024$3 between partially overlapping point sets by solving

$C=1024$4

where $C=1024$5 and $C=1024$6 (Chen et al., 29 Sep 2025). The motivation is that geometry-only methods such as Predator and GeoTransformer can fail when overlap is very small or geometry is ambiguous, while earlier RGB-D methods such as ColorPCR and PointMBF append color or exchange features in an image-lossy manner and therefore do not fully exploit dense image structure and semantics.

The architecture takes as input two point clouds $C=1024$7 and their aligned RGB images $C=1024$8. Its components are a DINOv2 ViT visual backbone producing patch features $C=1024$9, a KPConv-FPN geometric backbone producing patch features $F \in \mathbb{R}^{h \times w \times d \times C}$0 and fine point features $F \in \mathbb{R}^{h \times w \times d \times C}$1, a spatial mapping + window aggregation stage that associates each geometric patch with a local window of visual patch features, a latent fusion module

$F \in \mathbb{R}^{h \times w \times d \times C}$2

and a Visual-Geometric Transformer with interlaced self- and cross-attention layers and mixed positional embeddings.

The spatial mapping projects geometric-patch centroids into the image with the camera intrinsic matrix $F \in \mathbb{R}^{h \times w \times d \times C}$3, discretizes them on the DINOv2 patch grid, and aggregates a $F \in \mathbb{R}^{h \times w \times d \times C}$4 local neighborhood of visual features through a convolutional window operator. The role of this design is to make the visual-geometric association more robust to mapping errors and sparsity. DINOReg further injects both 2D and 3D positional information by rotary embeddings and a shared geometric embedding in the mixed-attention score, rather than relying only on bias terms added to attention logits.

Training uses overlap-aware circle loss $F \in \mathbb{R}^{h \times w \times d \times C}$5 from GeoTransformer for patch-level supervision and negative log-likelihood $F \in \mathbb{R}^{h \times w \times d \times C}$6 for fine point matches, with total loss

$F \in \mathbb{R}^{h \times w \times d \times C}$7

and default weights $F \in \mathbb{R}^{h \times w \times d \times C}$8 and $F \in \mathbb{R}^{h \times w \times d \times C}$9. The DINOv2 parameters are frozen during training. The standard configuration uses DINOv2-small + 256-d fusion, 3 attention layers of 4 heads; DINOReg-Super uses DINOv2-base + 512-d fusion, 3 layers of 8 heads. Training runs for 20 epochs on indoor data and 60 epochs on KITTI, with batch size = 1, learning rate init = 1e−4, exp decay = 0.05/epoch, and weight decay = 1e−6.

On RGBD-3DMatch, DINOReg reports PIR 74.6, IR 54.9, FMR 99.6, and RR 96.2, compared with GeoTransformer at 64.0 / 48.4 / 95.7 / 87.5 and ColorPCR at 64.6 / 48.0 / 96.9 / 89.6. On RGBD-3DLoMatch, it improves PIR by 14.2 pp (29.6→43.8) and RR by 15.7 pp (57.2→72.9) over ColorPCR. On KITTI, the reported values are PIR 78.4, IR 60.2, RRE 0.37, RTE 9.8, and RR 99.3, outperforming the listed GeoTransformer and ColorPCR baselines.

The ablations identify three influential components. First, +FFN fusion outperforms geometric-only, visual-only, and direct concat-to-attention variants on RGBD-3DLoMatch. Second, window size $k=24$0 is better than $k=24$1 and slightly better than $k=24$2. Third, mixed positional embeddings outperform both no positional embedding and geometric-only embeddings. Under mapping noise with $k=24$3 px, DINOReg degrades by <1 pp, whereas ColorPCR drops >6 pp PIR and >10 pp RR.

5. DINOv3 with test-time training for medical image registration

A later medical-registration work presents a related training-free pipeline built around a frozen DINOv3 encoder and test-time optimization of the deformation field in feature space; in its detailed exposition, this framework is explicitly described as DINOReg (Wang et al., 20 Aug 2025). The inputs are fixed and moving volumes $k=24$4 and $k=24$5 of size $k=24$6. Each volume is decomposed into axial slices, every $k=24$7-th slice is encoded with frozen DINOv3, and skipped slices are recovered by linear interpolation of the 3D grid of patch tokens along the $k=24$8-axis. For DINOv3-L, the patch-token embedding dimension is $k=24$9.

All token embeddings from both volumes are concatenated into a joint feature bank and reduced by PCA to $\mathbb{R}^{h \times w \times d \times k}$0 dimensions, yielding compact feature volumes

$\mathbb{R}^{h \times w \times d \times k}$1

Registration then proceeds entirely in this feature space, without any encoder fine-tuning. The objective is

$\mathbb{R}^{h \times w \times d \times k}$2

with mean-squared error as the default similarity term and $\mathbb{R}^{h \times w \times d \times k}$3. No explicit Jacobian-determinant penalty or inverse-consistency term is included; regularity is monitored after optimization through the standard deviation of $\mathbb{R}^{h \times w \times d \times k}$4, where $\mathbb{R}^{h \times w \times d \times k}$5.

The deformation field is a free, dense displacement vector at each voxel, so the transform is non-parametric and potentially non-diffeomorphic. Optimization is performed in two phases at test time. Phase 1 is a coarse-to-fine discrete search on a multi-resolution pyramid to obtain an approximately global optimum $\mathbb{R}^{h \times w \times d \times k}$6 for large misalignment. Phase 2 initializes $\mathbb{R}^{h \times w \times d \times k}$7 and refines it with Adam, typically using learning rate $\mathbb{R}^{h \times w \times d \times k}$8, betas $\mathbb{R}^{h \times w \times d \times k}$9, and 200–500 iterations. The reported default configuration includes slice stride $E(\phi) = S(f, m \circ \phi) + \lambda R(\phi),$0, a discrete search grid of $E(\phi) = S(f, m \circ \phi) + \lambda R(\phi),$1 voxels at coarse levels (example), 2 levels of downsampling by factors of 2, and roughly 300 Adam iterations.

Quantitatively, on Abdomen MR–CT, the paper reports for DINOv3+T³ a mean DSC of 0.790, HD95 of 4.9±5.0 mm, and SDLogJ of 0.08±0.02. The same table lists DINO-Reg (v2) at DSC 0.780, HD95 7.6±10.3, and SDLogJ 0.17±0.02, and ConvexAdam(MIND) at DSC 0.722, HD95 8.9±10.9, and SDLogJ 0.14±0.01. On ACDC 4D Cardiac MRI, DINOv3+T³ reaches DSC 0.769±0.10, HD95 4.8±3.5 mm, and SDLogJ 0.11±0.06, while the table lists DINOv2+T³ at DSC 0.755±0.11, HD95 5.1±3.7 mm, and SDLogJ 0.16±0.09. Implementation details include incremental PCA from scikit-learn, PyTorch, a single GPU such as an NVIDIA V100 with 32 GB, and a runtime of ~2–3 minutes per 3D pair.

6. Interpretation, limitations, and recurrent misunderstandings

A recurrent misunderstanding is to treat DINOReg as a single algorithm. The literature instead shows three different systems sharing a reliance on DINO-family visual representations but differing in domain, supervision, and optimization strategy (Song et al., 2024). The 2024 medical DINO-Reg and the 2025 DINOv3-based medical variant are training-free at deployment and avoid encoder fine-tuning; the point-cloud DINOReg, by contrast, is a trained network whose DINOv2 backbone is frozen while the fusion and matching components are learned (Chen et al., 29 Sep 2025). A second misunderstanding is that foundation-model features remove the need for explicit registration machinery; in all three cases, performance depends critically on downstream optimization, fusion, or matching design rather than on DINO features alone (Wang et al., 20 Aug 2025).

The limitations are also domain-specific. In the 2024 medical system, computational cost remains substantial because volumetric slice encoding and global convex optimization are time-consuming; the paper identifies acceleration avenues including 3D ViTs, fewer patches, and multi-scale encoding. The same work also notes boundary ambiguity, since DINO features are less sensitive to sharp organ edges, and proposes future integration of edge-aware losses or boundary-focused encoders, as well as extension to MRI, PET, learned regularizers, and diffeomorphic constraints. In the point-cloud system, the reported future directions are extension to other multi-modal 3D tasks, including semantic SLAM and cross-modal retrieval, and exploration of end-to-end fine-tuning of VFMs. The DINOv3 medical variant emphasizes compact feature-space optimization and regular deformations, but its formulation still uses a non-parametric, potentially non-diffeomorphic transform with regularity controlled only through the smoothness term.

Taken together, these works establish a recognizable technical pattern: frozen DINO-family encoders provide transferable patch- or voxel-level descriptors; registration quality is then recovered through either discrete-plus-continuous deformation optimization or multi-modal correspondence learning. A plausible implication is that DINOReg is best understood not as one method but as a broader registration paradigm in which foundation-model visual priors are coupled to domain-specific geometric or variational machinery.