Grid-Reg: Grid-Based Multimodal Registration

• The paper introduces a detector-free, patch-grid approach that replaces sparse keypoints with global affine matching in severely mismatched SAR–optical imagery.
• The method employs HSCMLNet to generate robust patch descriptors using pseudo-siamese and shared encoders for enhanced modality invariance.
• Grid-Solver refines transformation estimates through a coarse-to-fine candidate selection that boosts registration success under challenging conditions.

Grid-Based Multimodal Registration Framework, or “Grid-Reg,” denotes a class of multimodal registration methods that replace sparse keypoint correspondence with structured spatial sampling on a grid. In the specific formulation introduced for airborne SAR and spaceborne optical registration, the grid is a regular set of patch centers produced by a sliding window, and registration is solved as detector-free, global patch matching rather than as feature-point matching and RANSAC fitting. That formulation combines a descriptor network, Hybrid Siamese Correlation Metric Learning Network (HSCMLNet), with a coarse-to-fine grid-based solver, Grid-Solver, to estimate an affine transformation under severe cross-platform geometric and radiometric discrepancy (Wei et al., 6 Jul 2025). In the broader literature, however, “grid-based” also refers to sparse control-point lattices, dense voxel displacement grids, free-form deformation meshes, and spherical control grids, so the term encompasses multiple deformation parameterizations and solver architectures rather than a single model family (Yan et al., 15 Mar 2026).

1. Definition and scope

In the airborne-SAR/spaceborne-optical setting, Grid-Reg addresses registration between UAV MiniSAR images and Google Earth optical imagery when the two images differ strongly in geometry, appearance, coverage, and acquisition time. The motivating failure mode is that standard multimodal registration pipelines assume reliable keypoints, accurate nearest-neighbor correspondences, or modest geometric discrepancy. The framework instead assumes that accurate keypoint correspondences are inherently difficult in heterogeneous images with large geometric deformation, and therefore adopts detector-free, grid/patch-based global matching over whole images (Wei et al., 6 Jul 2025).

The “grid” in this formulation is not a deformable spline lattice and not a dense displacement field. It is a regular spatial sampling of patch centers induced by sliding-window partitioning. Each SAR patch center and optical patch center becomes a grid point, and each grid point receives a learned patch descriptor. Registration then proceeds by searching for a global affine transformation that minimizes a whole-image matching loss derived from descriptor distances between transformed SAR grid points and optical grid points. This design is explicitly motivated by the large cross-platform discrepancies of airborne SAR: speckle noise, defocusing, geometric distortions, inaccurate or missing POS, large rotations and flips, temporal scene changes, and large optical search regions (Wei et al., 6 Jul 2025).

A common misconception is that every grid-based multimodal registration framework is a local nonrigid deformation model. In the specific SAR–optical Grid-Reg, the estimated transformation is global affine. The grid is used to organize patch correspondences and define a global objective, not to parameterize a deformable field. This distinguishes it from sparse control-point-grid methods, dense voxel-grid methods, and free-form deformation frameworks discussed in later literature (Wei et al., 6 Jul 2025).

2. Gridization, patch descriptors, and HSCMLNet

Grid-Reg begins by cropping SAR and optical images into overlapping patches using a sliding window. With patch size $H^P \times W^P$, the patch sets are written as

$$P_S \in \mathbb{R}^{N_P^S \times H^P \times W^P}, \qquad P_O \in \mathbb{R}^{N_P^O \times H^P \times W^P}.$$

The paper uses $H^P=W^P=256$ at $0.51$ m resolution so that each patch covers a medium-sized building. The number of SAR patches is given as

$$N_P^S = \left( \frac{H^S - H^P/2}{\text{Step}} \right) \times \left( \frac{W^S - W^P/2}{\text{Step}} \right),$$

with the formula typeset imperfectly in the source but stated in that form (Wei et al., 6 Jul 2025).

Patch descriptors are produced by HSCMLNet, a Hybrid Siamese Correlation Metric Learning Network. HSCMLNet contains two components: HSModule and CMLModule. HSModule uses two pseudo-siamese encoders $En^S$ and $En^O$ for modality-specific low-level processing, followed by a shared encoder $En^{Sh}$ for common high-level representation. This architecture encodes the paper’s claim that low-level statistics differ greatly between SAR and optical, while high-level object semantics are shared for the same geographic area (Wei et al., 6 Jul 2025).

CMLModule operates on the high-level feature maps and constructs patch descriptors from local embeddings relative to equiangular unit basis vectors (EUBVs). The EUBVs are defined as

$V = L2Norm(I_{N_A}-1_{N_A\times N_A}/N_A),$

where the angle between any two unit row vectors of $V$ is equal and close to $$P_S \in \mathbb{R}^{N_P^S \times H^P \times W^P}, \qquad P_O \in \mathbb{R}^{N_P^O \times H^P \times W^P}.$$0. For local embeddings $$P_S \in \mathbb{R}^{N_P^S \times H^P \times W^P}, \qquad P_O \in \mathbb{R}^{N_P^O \times H^P \times W^P}.$$1, normalized correlation coefficients $$P_S \in \mathbb{R}^{N_P^S \times H^P \times W^P}, \qquad P_O \in \mathbb{R}^{N_P^O \times H^P \times W^P}.$$2 are computed with a softmax over basis vectors, and the patch-level coefficient vector is obtained by averaging over spatial locations: $$P_S \in \mathbb{R}^{N_P^S \times H^P \times W^P}, \qquad P_O \in \mathbb{R}^{N_P^O \times H^P \times W^P}.$$3 The final patch descriptor is then

$$P_S \in \mathbb{R}^{N_P^S \times H^P \times W^P}, \qquad P_O \in \mathbb{R}^{N_P^O \times H^P \times W^P}.$$4

The stated motivation is robustness: all local regions contribute to the final descriptor, which reduces overreliance on a few unstable local responses in the presence of noise, changed objects, or large modality gaps (Wei et al., 6 Jul 2025).

The descriptor learning objective combines a manifold loss and a contrastive loss: $$P_S \in \mathbb{R}^{N_P^S \times H^P \times W^P}, \qquad P_O \in \mathbb{R}^{N_P^O \times H^P \times W^P}.$$5 The manifold terms $$P_S \in \mathbb{R}^{N_P^S \times H^P \times W^P}, \qquad P_O \in \mathbb{R}^{N_P^O \times H^P \times W^P}.$$6 and $$P_S \in \mathbb{R}^{N_P^S \times H^P \times W^P}, \qquad P_O \in \mathbb{R}^{N_P^O \times H^P \times W^P}.$$7 constrain the normalized correlation between local embeddings and EUBVs across modalities and in joint multimodal space, while the contrastive term $$P_S \in \mathbb{R}^{N_P^S \times H^P \times W^P}, \qquad P_O \in \mathbb{R}^{N_P^O \times H^P \times W^P}.$$8 is an InfoNCE-like patch-level objective. The paper presents this as a way to obtain geometrically invariant, modality-invariant local embeddings without explicit local labels (Wei et al., 6 Jul 2025).

3. Grid-Solver and the global matching objective

Once descriptors are extracted, Grid-Reg forms the patch-level distance matrix

$$P_S \in \mathbb{R}^{N_P^S \times H^P \times W^P}, \qquad P_O \in \mathbb{R}^{N_P^O \times H^P \times W^P}.$$9

with $H^P=W^P=256$0 when descriptors are $H^P=W^P=256$1-normalized. For each SAR grid point, the nearest optical descriptors are sorted to construct a coarse candidate set $H^P=W^P=256$2 and a larger fine candidate set $H^P=W^P=256$3. The candidate sizes are defined by

$H^P=W^P=256$4

with the expression typeset imperfectly in the paper but stated in that form (Wei et al., 6 Jul 2025).

Grid-Solver estimates an affine transformation by random hypothesis generation, geometric filtering, and global objective evaluation. It first samples three point pairs $H^P=W^P=256$5 from $H^P=W^P=256$6 and applies an equal-area constraint. If the triangle areas in source and target are $H^P=W^P=256$7 and $H^P=W^P=256$8, the sample is retained only if

$H^P=W^P=256$9

This exploits the fact that SAR and optical images are at equal resolution in the benchmark and removes geometrically abnormal hypotheses before model fitting (Wei et al., 6 Jul 2025).

For a candidate affine transform $0.51$0, each SAR grid point $0.51$1 is transformed to $0.51$2, then quantized to the nearest optical grid index: $0.51$3 The current matching loss is then

$0.51$4

This is the central objective of Grid-Reg: instead of counting point inliers, it sums descriptor distances implied by the transformation over the whole SAR patch grid. Because the method does not commit to exact correspondences beforehand, robustness is achieved through global aggregation rather than local correspondence trust (Wei et al., 6 Jul 2025).

If $0.51$5 is below a threshold $0.51$6, Grid-Solver performs local refinement using candidate matches from $0.51$7 near the current transform prediction: $0.51$8 Updated hypotheses are accepted if they reduce the loss. The best transformation is retained with

$0.51$9

This yields a coarse-to-fine search that is coarse both in correspondence set and in search scope (Wei et al., 6 Jul 2025).

4. Benchmark design and empirical results

The Grid-Reg paper introduces a benchmark for airborne SAR to spaceborne optical registration using drone-based MiniSAR and Google Earth imagery, and presents it as the first benchmark for this specific cross-platform large-scale task. The dataset contains 29 image subsets, with 17 subsets for training and the remaining subsets for testing. For evaluation, 12 test subsets with better affine correction were selected, because some SAR–optical pairs contain nonlinear distortions that cannot be sufficiently modeled by affine transformation. Ground-truth affine transformations are obtained by manual annotation of control points for each image pair (Wei et al., 6 Jul 2025).

Difficulty is analyzed using four levels defined by overlap and search-region ratios:

• $$N_P^S = \left( \frac{H^S - H^P/2}{\text{Step}} \right) \times \left( \frac{W^S - W^P/2}{\text{Step}} \right),$$0: ROA $$N_P^S = \left( \frac{H^S - H^P/2}{\text{Step}} \right) \times \left( \frac{W^S - W^P/2}{\text{Step}} \right),$$1, RSO $$N_P^S = \left( \frac{H^S - H^P/2}{\text{Step}} \right) \times \left( \frac{W^S - W^P/2}{\text{Step}} \right),$$2
• $$N_P^S = \left( \frac{H^S - H^P/2}{\text{Step}} \right) \times \left( \frac{W^S - W^P/2}{\text{Step}} \right),$$3: ROA $$N_P^S = \left( \frac{H^S - H^P/2}{\text{Step}} \right) \times \left( \frac{W^S - W^P/2}{\text{Step}} \right),$$4, RSO $$N_P^S = \left( \frac{H^S - H^P/2}{\text{Step}} \right) \times \left( \frac{W^S - W^P/2}{\text{Step}} \right),$$5
• $$N_P^S = \left( \frac{H^S - H^P/2}{\text{Step}} \right) \times \left( \frac{W^S - W^P/2}{\text{Step}} \right),$$6: ROA $$N_P^S = \left( \frac{H^S - H^P/2}{\text{Step}} \right) \times \left( \frac{W^S - W^P/2}{\text{Step}} \right),$$7, RSO $$N_P^S = \left( \frac{H^S - H^P/2}{\text{Step}} \right) \times \left( \frac{W^S - W^P/2}{\text{Step}} \right),$$8
• $$N_P^S = \left( \frac{H^S - H^P/2}{\text{Step}} \right) \times \left( \frac{W^S - W^P/2}{\text{Step}} \right),$$9: ROA $En^S$0, RSO $En^S$1

Evaluation uses registration success rate

$En^S$2

and median error

$En^S$3

At threshold $En^S$4 pixels, the reported success rates are 75.32% at $En^S$5, 85.71% at $En^S$6, 83.12% at $En^S$7, and 79.87% at $En^S$8, substantially higher than HardNetF, SosNetF, HyNetF, and SDGMNetF under the same solver family (Wei et al., 6 Jul 2025).

The paper also isolates the effect of the solver. Using HSCMLNet descriptors with Feat-Solver versus Grid-Solver, $En^S$9 at $En^O$0 rises from 28.57% to 79.87%, and at $En^O$1 from 36.36% to 85.71%. Conversely, when Grid-Solver is fixed and descriptor networks are varied, HSCMLNetG remains markedly stronger than HardNetG, SosNetG, HyNetG, and SDGMNetG. The reported interpretation is that both the descriptor and the global solver are necessary: the descriptor improves cross-modal patch similarity estimation, and the solver removes dependence on exact correspondences (Wei et al., 6 Jul 2025).

A second empirical result concerns sensitivity to grid spacing and search effort. As Step increases from 16 to 128, MEE increases. The iteration count is set by

$En^O$2

with $En^O$3 for Grid-Solver and $En^O$4 for Feat-Solver. The paper reports that the method retains better performance than competitors up to step size 96, especially for $En^O$5, and that difficulty sensitivity is greater at $En^O$6 than at $En^O$7 (Wei et al., 6 Jul 2025).

5. Grid-based registration as a broader methodological family

The patch-grid formulation of Grid-Reg is only one interpretation of “grid-based” registration. In learning-based medical image registration, “GridReg” replaces dense voxel-wise decoding with displacement prediction on a sparse regular control-point grid, then interpolates to a dense deformation field. Its deformation complexity scales with $En^O$8 rather than $En^O$9, and its decoder uses cross-attention between grid queries and image tokens to estimate sparse gridded deformation fields (Yan et al., 15 Mar 2026). In that sense, GridReg is grid-based in the deformation-parameterization sense, whereas Grid-Reg for SAR–optical registration is grid-based in the correspondence-sampling and global-objective sense.

A second line of work uses free-form deformation lattices. EDFFDNet defines a regular control lattice $En^{Sh}$0, predicts control-point displacements $En^{Sh}$1, and synthesizes a dense field by an exponential-decay basis

$En^{Sh}$2

combining global homography initialization with local lattice refinement (Zhu et al., 9 Sep 2025). This is grid-based in a classical free-form deformation sense.

A third line operates on dense voxel grids. FFDP supports affine, deformable, and diffeomorphic registration with dense per-voxel displacement fields $En^{Sh}$3, where $En^{Sh}$4 is modeled as a grid of per-voxel vectors. Its contribution is primarily systems-level: distributed tensor sharding, ring-based interpolation, fused mutual information and LNCC kernels, and scaling to gigavoxel multimodal MRI volumes (Jena et al., 29 Sep 2025). Here the grid is the full voxel lattice and deformation tensor rather than a sparse control lattice or patch grid.

Surface registration provides yet another interpretation. GeoMorph uses a sparse set of control points on a spherical icosphere, predicts discrete label assignments for those control points, regularizes them with a CRF-RNN, and upsamples to the full surface by barycentric interpolation (Suliman et al., 2023). The control structure is grid-like but lives on a spherical mesh rather than a Cartesian lattice.

These examples show that “grid-based multimodal registration framework” is not a single architecture. It includes regular patch-center grids used for global affine search, sparse control lattices interpolated to dense fields, dense voxel displacement grids, and manifold-specific control meshes (Wei et al., 6 Jul 2025).

6. Limitations, misconceptions, and plausible extensions

The specific SAR–optical Grid-Reg is not a dense deformable registration method. Its final model is affine, and the benchmark explicitly selects test pairs with better affine correction because some pairs contain nonlinear distortions that affine transformation cannot model sufficiently. A common misconception is therefore to read Grid-Reg as a generic nonrigid grid-warp model. In the paper’s actual formulation, the grid organizes detector-free patch matching, while the estimated transform remains global affine (Wei et al., 6 Jul 2025).

A second misconception is that the method abandons geometry in favor of purely learned descriptors. The opposite is true: the solver is strongly geometric. It samples point triples, uses an equal-area constraint, evaluates a transformation-induced whole-image matching loss, and refines using spatial neighborhoods. The empirical ablation in which HSCMLNetF is compared with HSCMLNetG shows that replacing Feat-Solver by Grid-Solver alone produces a major performance increase, which indicates that the geometric solver is a central component rather than an implementation detail (Wei et al., 6 Jul 2025).

The broader literature suggests several plausible extensions. A plausible implication is that if stronger local deformation is required, the detector-free patch-grid philosophy of Grid-Reg could be combined with sparse control-point deformation models such as GridReg or with local-lattice FFD models such as EDFFDNet, while preserving descriptor learning and coarse-to-fine global matching (Yan et al., 15 Mar 2026). A second plausible implication is that when multimodal registration is limited by memory or volume scale rather than by patch ambiguity, dense-voxel execution substrates such as FFDP become relevant, especially for multimodal 3D MRI (Jena et al., 29 Sep 2025). These implications are not claims of the original Grid-Reg paper, but they follow directly from the distinct meanings of “grid-based” established across the cited work.

Within its stated scope, Grid-Reg is best understood as a detector-free, patch-grid, global-matching framework for airborne SAR to spaceborne optical registration across platforms. Its defining contribution is not a new nonrigid deformation family, but the combination of domain-robust multimodal patch descriptors and a global grid-based affine solver that remains stable when accurate cross-modal keypoints are inherently difficult to obtain (Wei et al., 6 Jul 2025).