Encoder-Only Image Registration

• Encoder-Only Image Registration (EOIR) is a framework that separates feature extraction from flow estimation using a shared encoder and minimal deformation modules.
• It replaces heavy decoders with lightweight, multi-scale flow estimators and diffeomorphic composition, improving efficiency and smoothness in registration.
• EOIR underpins both strict and encoder-centric designs, demonstrating robust performance in mono-modal settings and offering pathways for cross-domain generalization.

Encoder-Only Image Registration (EOIR) denotes a family of image registration methods in which the principal learned machinery is concentrated in an encoder, while deformation estimation is performed either by lightweight flow estimators, classical optimization, or a minimal model-driven reconstruction stage rather than by a heavy decoder. In the narrow architectural sense, EOIR refers to the deformable registration framework introduced under that name, which separates feature learning from flow estimation with a 3-layer ConvNet encoder, a set of 3-layer flow estimators, and diffeomorphic composition over a Laplacian feature pyramid (Chen et al., 30 Aug 2025). In a broader usage that appears across recent medical-registration literature, EOIR also covers encoder-centric pipelines in which a frozen or dominant encoder supplies the correspondence representation and the remaining registration machinery is deliberately minimal, deterministic, or non-amortized (Song et al., 2024, Kögl et al., 2024, Razzaq et al., 2024).

1. Definition, scope, and historical placement

The broader deep-registration literature did not originally treat EOIR as a separate named paradigm. A survey of deep learning in medical image registration frames what is now called EOIR primarily as a family of encoder-based networks used for rigid or affine registration and parameter regression, in contrast to U-Net-like encoder–decoder architectures that became standard for dense deformable image registration (DIR) (Chen et al., 2023). In that survey’s architectural taxonomy, encoder-only models are naturally matched to low-dimensional global transforms, whereas dense deformable fields typically motivate encoder–decoder designs and explicit spatial transformers.

The later EOIR formulation extends this encoder-first logic into deformable registration by arguing that prior deep models often entangle feature extraction and flow estimation in over-architected backbones, even though the essential learned role of ConvNets is more limited: linearizing local intensities and harmonizing global contrast variations (Chen et al., 30 Aug 2025). This recasts EOIR from a rigid-registration convenience into a deliberate design principle for deformable registration.

Across adjacent work, the term is used with different strictness. DINO-Reg is described as EOIR because the only learned component is a frozen pretrained DINOv2 encoder and the deformation field is obtained by discrete and Adam-based optimization rather than by a trained registration network (Song et al., 2024). NeuReg is characterized as an encoder-centric, near-encoder-only 3D registration method because its Swin Transformer encoder performs the principal correspondence extraction and deformation prediction, while the “decoder” is a deterministic zero padding + inverse DFT reconstruction module (Razzaq et al., 2024). This variation in usage is central to the topic: EOIR can denote either a literal no-decoder architecture or a broader encoder-dominant registration philosophy.

2. Core architectural principles

In its strict formulation, EOIR is built from three components: a 3-layer ConvNet encoder, a set of 3-layer flow estimators, and a deformation composition module over a Laplacian feature pyramid (Chen et al., 30 Aug 2025). The encoder uses three Conv-Norm-Act blocks, each consisting of a $3\times 3\times 3$ convolution, instance normalization, and ReLU. Its channel pattern starts with $N_s$, expands to $2N_s$ using an inverted bottleneck design, and contracts back to $N_s$. The encoder is shared between moving and fixed images.

Feature extraction is explicitly separated from flow estimation. Rather than concatenating moving and fixed images at the input and asking a single backbone to infer everything jointly, EOIR first extracts features independently, constructs a pyramid from those features, and then estimates residual flows at each scale. Each flow estimator has the same lightweight structure but different weights across pyramid levels. Its input is organized by a Hadamard transform,

$$H_2[\mathbf{I}_m^{(l)}, \mathbf{I}_f^{(l)}]^\top = [\mathbf{I}_m^{(l)} + \mathbf{I}_f^{(l)},\ \mathbf{I}_m^{(l)} - \mathbf{I}_f^{(l)}]^\top,$$

which exposes similarity and difference signals directly rather than requiring the network to recover them from raw channel concatenation (Chen et al., 30 Aug 2025).

The pyramid is Laplacian-style. Features are extracted at full resolution and then repeatedly downsampled by $2\times$ trilinear interpolation $n-1$ times to produce $n$ scales. The number of levels is chosen to satisfy

$n > \log_2(d_{\max}) + 1,$

so that sufficiently large physical displacements become small in voxel coordinates at coarse scales. The default configuration uses 5 pyramid levels (Chen et al., 30 Aug 2025).

The principal theoretical background is the Horn–Schunck optical-flow approximation,

$$\nabla \mathbf{I}_f(p) \cdot \mathbf{u}(p) = \mathbf{I}_m(p) - \mathbf{I}_f(p),$$

with validity tied to sufficiently small motion, approximately

$N_s$0

EOIR pairs this with the Linearization-Harmonization assumption, according to which ConvNets serve two registration-specific roles: linearizing local intensities and harmonizing global contrast variations (Chen et al., 30 Aug 2025). The paper reports that 3 convolutional layers are enough to capture most of the gain, that more layers give diminishing returns, and that a full U-Net encoder can increase complexity while reducing accuracy in the reported mono-modal setting.

Large deformation is handled by composing small diffeomorphic increments rather than by predicting a single full-resolution field. For pyramid level $N_s$1,

$N_s$2

Scaling-and-squaring is used for $N_s$3, and the paper contrasts this composition-based scheme with additive composition used by LapIRN and MemWarp, arguing that addition is computationally simpler but yields less smooth deformation fields and more non-diffeomorphic voxels (Chen et al., 30 Aug 2025).

3. Methodological realizations

The literature contains several distinct EOIR realizations, ranging from direct deformable predictors to training-free optimization pipelines.

Method	EOIR relation	Distinctive mechanism
EOIR	Strict encoder-only	3-layer encoder + 3-layer flow estimators
DINO-Reg	Encoder-only in a training-free sense	Frozen DINOv2 + discrete optimization + Adam refinement
NeuReg	Near-encoder-only	Swin encoder + zero padding + inverse DFT
Feature-guided B-spline	Encoder-only in the similarity term	Frozen vision encoder in the dissimilarity objective
FMIR	Encoder-centric, not strictly EOIR	Frozen foundation encoder + lightweight pyramid head

DINO-Reg is a training-free deformable registration method that uses the teacher network of a pretrained DINOv2 model as a frozen slice-wise feature extractor without finetuning; 3D volumes are treated as stacks of 2D slices, and axial encoding was found to work best (Song et al., 2024). Resized slices are partitioned into non-overlapping $N_s$4 patches, patch tokens are used as features, and class and register tokens are added to capture global semantics and redundant background information. There is no trained decoder, segmentation head, or end-to-end registration network. Instead, the encoded features are passed to a discrete optimizer for initial deformation estimation and then to Adam-based continuous refinement. The refinement objective uses feature-wise local cross-correlation (LCC) rather than SSD, because the feature distributions may still differ between reference and moving images. A PCA reduction with $N_s$5 is used in the project formulation (Song et al., 2024).

NeuReg is an encoder-centric 3D brain-registration architecture built around a domain-generalization preprocessing layer and a shifting window-based Swin Transformer encoder (Razzaq et al., 2024). Input volumes are split into patches, patch statistics are used to form a domain-generalized representation, and the Swin encoder uses rectangular parallelepiped windows $N_s$6 to better fit anisotropic brain-volume shapes. After four repeated stages, encoded features are used to predict the deformation field $N_s$7 directly. The downstream module is not a learned CNN decoder; it reconstructs a low-dimensional deformation field through zero padding and inverse DFT and then warps the moving image,

$N_s$8

The loss is of the standard similarity-plus-regularization form,

$N_s$9

with MSE or NCC used as similarity criteria in different experiments (Razzaq et al., 2024).

A third realization uses the encoder not to predict deformation directly, but to define the objective landscape. In the B-spline FFD study on general vision encoder features, frozen DINOv2, SAM, and MedSAM encoders provide feature maps $2N_s$0, and registration is performed by minimizing either a pure feature-space objective or a combined intensity-plus-feature objective (Kögl et al., 2024). The combined form is

$2N_s$1

with cosine dissimilarity or L1/MAE for $2N_s$2. In this variant, the encoder is the only learned representation module, while deformation remains entirely classical.

FMIR occupies a nearby but distinct position. It uses a frozen 2D foundation model encoder such as DINO ViT-B during training and can also plug in SAM at inference, followed by channel regularization, a 3-layer 3D convolutional restoration block, and a multi-scale pyramid registration head (Zhang et al., 24 Jan 2026). The paper is explicit that FMIR is not strictly encoder-only, because deformation is not predicted directly by the encoder alone; nonetheless, it is described as encoder-centric and much closer to EOIR than to symmetric encoder–decoder registration systems.

4. Empirical performance and operating regimes

The strict EOIR framework was evaluated on five datasets spanning abdominal CT, cardiac MR, and brain MRI, with an emphasis on the accuracy–efficiency and accuracy–smoothness trade-offs (Chen et al., 30 Aug 2025).

Dataset	Reported EOIR result	Note
Abdomen CT	Dice 60.63%, HD95 17.61 mm, SDlogJ 0.17	Best Dice and HD95 among listed methods
ACDC	Dice 78.91%, HD95 9.07 mm, SDlogJ 0.084	114.21 G multi-adds, 0.91 MB
OASIS	Dice 88.83%, HD95 1.28 mm, SDlogJ 0.52	Slightly surpasses TransMorph-2 and LKU-Net in Dice
LUMIR	Dice 77.37%, HD95 3.33 mm, NDV 0.000%, TRE 2.35 mm	2nd place in challenge test phase
HippocampusMR	Dice 86.44%, HD95 6.20 mm, SDlogJ 0.10	0.35 s inference time

These numbers are paired with explicit compactness claims. On ACDC, a smaller EOIR variant with $2N_s$3 achieves Dice $2N_s$4, HD95 $2N_s$5 mm, SDlogJ $2N_s$6, $2N_s$7 G multi-adds, and $2N_s$8 MB parameters (Chen et al., 30 Aug 2025). On LUMIR, additive composition produces nearly identical accuracy—Dice $2N_s$9, HD95 $N_s$0 mm, TRE $N_s$1 mm—but NDV rises from $N_s$2 to $N_s$3, which isolates the smoothness advantage of diffeomorphic composition. On HippocampusMR, MemWarp obtains slightly higher Dice at $N_s$4 but much worse smoothness, with SDlogJ $N_s$5 versus EOIR’s $N_s$6 (Chen et al., 30 Aug 2025).

Related EOIR-style systems support the broader empirical claim that encoder-dominant registration can generalize well. DINO-RegEn achieved 1st place in the OncoReg Challenge with score $N_s$7, while native DINO-Reg placed 4th with score $N_s$8 in a training-free setup (Song et al., 2024). In the feature-guided B-spline framework on ACDC cardiac cine MRI, the best reported result was MedSAM + cosine + NCC with mean DSC $N_s$9 and mean HD95 $$H_2[\mathbf{I}_m^{(l)}, \mathbf{I}_f^{(l)}]^\top = [\mathbf{I}_m^{(l)} + \mathbf{I}_f^{(l)},\ \mathbf{I}_m^{(l)} - \mathbf{I}_f^{(l)}]^\top,$$0, compared with a baseline NCC mean DSC $$H_2[\mathbf{I}_m^{(l)}, \mathbf{I}_f^{(l)}]^\top = [\mathbf{I}_m^{(l)} + \mathbf{I}_f^{(l)},\ \mathbf{I}_m^{(l)} - \mathbf{I}_f^{(l)}]^\top,$$1 and mean HD95 $$H_2[\mathbf{I}_m^{(l)}, \mathbf{I}_f^{(l)}]^\top = [\mathbf{I}_m^{(l)} + \mathbf{I}_f^{(l)},\ \mathbf{I}_m^{(l)} - \mathbf{I}_f^{(l)}]^\top,$$2 (Kögl et al., 2024). NeuReg reported strong cross-domain results in 3D brain registration, including iSeg-2017 scores of DICE $$H_2[\mathbf{I}_m^{(l)}, \mathbf{I}_f^{(l)}]^\top = [\mathbf{I}_m^{(l)} + \mathbf{I}_f^{(l)},\ \mathbf{I}_m^{(l)} - \mathbf{I}_f^{(l)}]^\top,$$3, SSIM $$H_2[\mathbf{I}_m^{(l)}, \mathbf{I}_f^{(l)}]^\top = [\mathbf{I}_m^{(l)} + \mathbf{I}_f^{(l)},\ \mathbf{I}_m^{(l)} - \mathbf{I}_f^{(l)}]^\top,$$4 for $$H_2[\mathbf{I}_m^{(l)}, \mathbf{I}_f^{(l)}]^\top = [\mathbf{I}_m^{(l)} + \mathbf{I}_f^{(l)},\ \mathbf{I}_m^{(l)} - \mathbf{I}_f^{(l)}]^\top,$$5 and DICE $$H_2[\mathbf{I}_m^{(l)}, \mathbf{I}_f^{(l)}]^\top = [\mathbf{I}_m^{(l)} + \mathbf{I}_f^{(l)},\ \mathbf{I}_m^{(l)} - \mathbf{I}_f^{(l)}]^\top,$$6, SSIM $$H_2[\mathbf{I}_m^{(l)}, \mathbf{I}_f^{(l)}]^\top = [\mathbf{I}_m^{(l)} + \mathbf{I}_f^{(l)},\ \mathbf{I}_m^{(l)} - \mathbf{I}_f^{(l)}]^\top,$$7 for $$H_2[\mathbf{I}_m^{(l)}, \mathbf{I}_f^{(l)}]^\top = [\mathbf{I}_m^{(l)} + \mathbf{I}_f^{(l)},\ \mathbf{I}_m^{(l)} - \mathbf{I}_f^{(l)}]^\top,$$8, as well as OASIS-3 results of $$H_2[\mathbf{I}_m^{(l)}, \mathbf{I}_f^{(l)}]^\top = [\mathbf{I}_m^{(l)} + \mathbf{I}_f^{(l)},\ \mathbf{I}_m^{(l)} - \mathbf{I}_f^{(l)}]^\top,$$9 for $2\times$0 and $2\times$1 for $2\times$2, greatly exceeding the reported FourierNet and SynthMorph baselines in those unseen-domain settings (Razzaq et al., 2024).

A plausible implication is that EOIR is most competitive when registration quality depends on robust correspondence representations and smooth deformation composition rather than on a heavy image-reconstruction decoder. That implication is directly supported in mono-modal, large-deformation, and cross-domain brain-registration settings, but it is not uniformly supported for every modality pairing or every similarity formulation.

5. Boundaries, misconceptions, and adjacent paradigms

A recurring source of confusion is the boundary between encoder-only, encoder-centric, and decoder-dependent registration. The survey literature makes a sharp architectural distinction: encoder-only networks remain a natural fit for rigid or affine parameter regression, whereas dense deformable registration has historically been dominated by encoder–decoder pipelines because the output is a spatially varying field at input resolution (Chen et al., 2023). The strict EOIR framework challenges that division by using a lightweight pyramid of residual flow estimators instead of a full decoder, but it does not erase the distinction.

Several recent methods are therefore best described as EOIR-like rather than strictly EOIR. FMIR relies on a frozen foundation encoder, but it still requires a five-level pyramid registration head with residual deformation prediction at each scale (Zhang et al., 24 Jan 2026). LLaMA-Reg uses dual CNN encoders, adapters, a frozen pretrained LLaMA 2 block, and a multi-scale decoder that fuses LLaMA-derived features with intermediate encoder features to predict deformations; it is described as encoder-centric hybrid rather than pure EOIR (Ma et al., 2024). The shared-encoder method with auxiliary and pyramid decoders is explicitly not encoder-only because registration quality depends on both auxiliary-decoder features and a fusion pyramid decoder (Zhou et al., 2024). EncoderReg is similarly not pure EOIR, since it combines Encoder-G and Encoder-S with a ModeT decoder for multi-stage deformation estimation (Wang et al., 2024).

A second misconception is that encoder features can simply replace conventional similarity measures. The B-spline FFD study provides a clear counterexample: feature-only registration generally performed worse than the NCC baseline, whereas a combined objective using both NCC and encoder-feature distance consistently improved DSC and HD95 (Kögl et al., 2024). The evidence therefore supports encoder features as guidance, but not as a universal stand-alone replacement for classical similarity.

A third boundary concerns methods that minimize or remove decoders without fitting the EOIR definition. NIR is optimization-based and models the deformation itself as a continuous neural field $2\times$3, with either direct displacement output or velocity-field integration via RK4; it is closer to an implicit, decoder-free field parameterization than to an image-encoder EOIR model (Sun et al., 2022). Conditional Segmentation in Lieu of Image Registration is more radically orthogonal: it replaces dense correspondence prediction with ROI-conditioned segmentation in the fixed image and reported a median TRE of $2\times$4 mm versus $2\times$5 mm for a DDF-predicting registration network, a $2\times$6 TRE reduction in prostate MR–TRUS experiments (Hu et al., 2019). Such methods are adjacent to EOIR only in the broad sense that they reject the necessity of a heavy dense-warp decoder.

Outside medical imaging, Super Registration provides a still earlier conceptual precursor. It learns an optimal latent model for the underlying image and then shifts that model to match the data, rather than shifting one noisy image to another, and reports that this can yield $2\times$7 more precise particle positions while reaching the Cramér–Rao bound in the periodic case (Clement et al., 2018). This suggests a deeper common principle: the registration bottleneck often lies in representation quality more than in decoder depth.

6. Limitations and research directions

The clearest stated limitation of strict EOIR is modality robustness. The EOIR paper explicitly notes that the framework is optimized for mono-modal settings and that its shallow 3-layer encoder is not powerful enough to learn strong cross-modal invariances (Chen et al., 30 Aug 2025). In a ThoraxCBCT example, EOIR achieved only about $2\times$8 Dice, whereas replacing the encoder with a full U-Net increased Dice to $2\times$9. The same paper also notes that spatial downsampling in the pyramid can remove small structures such as lung nodules or retinal vessels, reducing suitability for very fine-grained anatomical targets.

Training-free and feature-guided EOIR variants have a different set of limitations. DINO-Reg reports that DINOv2 features are less enunciated on corner points and contours of organs, which can induce slight misalignment of fine structures; this is one reason the paper combines DINOv2 features with handcrafted MIND descriptors in ensembles and sequential pipelines (Song et al., 2024). The feature-guided B-spline study identifies additional practical issues: the encoders require 2D inputs, direct 3D use is awkward, runtime increases because features are recomputed during optimization, feature extraction and warping are not commutative, and feature maps lose spatial resolution (Kögl et al., 2024). These observations limit the claim that frozen foundation encoders are a complete substitute for registration-specific design.

Cross-domain generalization remains the principal research driver. NeuReg argues for domain-generalization preprocessing plus a minimal deterministic decoder in human and mouse brain registration (Razzaq et al., 2024). FMIR argues that channel regularization is central to out-of-domain robustness and reports that unsupervised training generalizes better out of domain than weakly supervised training, with the removal of channel regularization producing much worse out-of-domain performance despite similar in-domain results (Zhang et al., 24 Jan 2026). A plausible implication is that the next stage of EOIR research will continue to move toward foundation-model encoders, explicit domain-invariance mechanisms, and lightweight deformation heads or optimizers that preserve the efficiency gains of encoder-first design without sacrificing cross-domain performance.

EOIR therefore occupies a precise but expanding niche within registration research. In its strict form, it is a lightweight deformable-registration architecture that replaces the classical encoder–decoder template with a small shared encoder, shallow per-scale flow estimators, and diffeomorphic composition (Chen et al., 30 Aug 2025). In its broader form, it names a family of encoder-dominant methods that attempt to relocate most of the registration intelligence into the representation stage, leaving deformation estimation to optimization, deterministic reconstruction, or modest downstream heads (Song et al., 2024, Razzaq et al., 2024). The unifying premise is not the total absence of post-encoder processing, but the claim that registration performance is governed primarily by the quality, invariance, and geometry-awareness of the encoded representation.