ReMIND2Reg Benchmark
- The paper establishes a new benchmark that standardizes evaluation protocols for nonrigid MRI-to-iUS registration in challenging post-resection scenarios.
- It employs landmark-based evaluation, bootstrapped ranking, and quantitative metrics like TRE and TRE30 to assess registration performance.
- The dataset integrates paired 3D ceT1, T2 MRI, and iUS volumes with standardized preprocessing to support development of robust, clinically deployable methods.
Searching arXiv for the benchmark paper and closely related ReMIND2Reg methods. ReMIND2Reg Benchmark designates the standardized benchmark introduced in “The Brain Resection Multimodal Image Registration (ReMIND2Reg) 2025 Challenge” for nonrigid multimodal registration of preoperative MRI to post-resection intraoperative ultrasound in brain tumor surgery (Dorent et al., 13 Aug 2025). It targets compensation for brain shift during the most difficult intraoperative phase—after tissue removal—when neuronavigation based on preoperative MRI loses accuracy and registration must accommodate large deformations, topology change, and a substantial modality intensity gap (Dorent et al., 13 Aug 2025). The benchmark is built upon the ReMIND dataset and provides paired 3D ceT1 MRI, T2 MRI, and post-resection 3D iUS volumes with standardized preprocessing, landmark-based evaluation, and a submission protocol intended to support robust, generalizable, and clinically deployable multimodal registration algorithms for image-guided neurosurgery (Dorent et al., 13 Aug 2025).
1. Clinical problem and benchmark scope
ReMIND2Reg addresses the clinically critical problem of aligning preoperative MRI to post-resection iUS in order to compensate for brain shift during tumor resection (Dorent et al., 13 Aug 2025). Neuronavigation based on preoperative MRI loses accuracy intraoperatively due to nonlinear deformations driven by cerebrospinal fluid loss, gravity, edema, and tumor removal; the consequence is degraded spatial guidance at resection boundaries, affecting safety and extent of resection (Dorent et al., 13 Aug 2025).
The benchmark focuses specifically on the post-resection phase. This stage is described as the most challenging intraoperative phase because large anatomical and topological changes occur after tissue removal, including resection cavities that break correspondence and induce topology change (Dorent et al., 13 Aug 2025). A substantial modality intensity gap further complicates the task: MRI captures tissue morphology and contrast, whereas ultrasound measures echogenicity, has lower soft-tissue contrast, speckle noise, and operator-dependent artifacts such as shadowing and signal dropouts (Dorent et al., 13 Aug 2025). Ultrasound reconstruction and acquisition constraints, including sweeps through a craniotomy and limited views, also complicate spatial consistency (Dorent et al., 13 Aug 2025).
The task definition is nonrigid multimodal registration of preoperative MRI—ceT1 and/or T2—to post-resection iUS, with the moving image being MRI and the fixed image being iUS (Dorent et al., 13 Aug 2025). The challenge does not restrict transform types: rigid, affine, and deformable transformations are all permissible, provided the method aims to accurately warp MRI to iUS under large deformation and topology change (Dorent et al., 13 Aug 2025). This suggests that ReMIND2Reg is less a benchmark for a single registration paradigm than a common evaluation substrate for heterogeneous registration formulations.
2. Dataset composition, acquisition, and preprocessing
ReMIND2Reg is a pre-processed subset of the ReMIND dataset, assembled from consecutive image-guided tumor resections from 2018–2024 and compliant with IRB approval under Brigham and Women’s Hospital protocol 2002-P-001238 and informed consent for public sharing (Dorent et al., 13 Aug 2025). The benchmark provides 99 training cases, 5 validation cases, and 10 private test cases (Dorent et al., 13 Aug 2025).
The dataset composition is summarized below.
| Split | Patients | Paired MR–iUS image pairs |
|---|---|---|
| Training | 99 | 155 |
| Validation | 5 | 10 |
| Private test | 10 | 20 |
For the training split, the dataset contains 99 iUS, 93 ceT1, and 62 T2-SPACE volumes; the validation split contains 5 iUS, 5 ceT1, and 5 T2-SPACE volumes; and the private test split contains 10 iUS, 10 ceT1, and 10 T2-SPACE volumes (Dorent et al., 13 Aug 2025). Each case provides post-resection 3D iUS plus available preoperative 3D MRI sequences, specifically ceT1 MP‑RAGE and/or T2‑SPACE (Dorent et al., 13 Aug 2025). Not all patients have both MRI sequences, and the benchmark explicitly states that algorithms should accommodate missing modalities (Dorent et al., 13 Aug 2025). The challenge evaluates methods separately for each MR sequence in the testing and ranking stage (Dorent et al., 13 Aug 2025).
Acquisition details are fixed in the benchmark description. MRI consists of preoperative ceT1 acquired as MP‑RAGE across vendors with multi-institution USA provenance and T2‑SPACE acquired on Siemens 3T systems (Dorent et al., 13 Aug 2025). The iUS volumes were acquired in the AMIGO OR suite at Brigham and Women’s Hospital using a sterilizable 2D neuro‑cranial curvilinear transducer on the BK5000 system with probe N13C5, contact area 29 mm × 10 mm, and frequency 5–13 MHz (Dorent et al., 13 Aug 2025). Sweeps were performed unidirectionally through the craniotomy, with the plane selected to be as parallel as possible to axial, sagittal, or coronal axes, tracked via Brainlab adapters, and reconstructed to 3D with Brainlab Elements on Curve hardware (Dorent et al., 13 Aug 2025).
All images are resampled to a common voxel resolution of 0.5 × 0.5 × 0.5 mm and a spatial dimension of 256 × 256 × 256 (Dorent et al., 13 Aug 2025). Because the iUS volumes are reconstructed with calibrated tracking, they are placed in approximately the same image space as preoperative MRI, so residual misalignment primarily reflects brain shift rather than global pose (Dorent et al., 13 Aug 2025). This preprocessing choice is central to the benchmark’s scope: it reduces confounding from gross initialization and emphasizes deformational compensation.
3. Landmark protocol and evaluation metrics
Training data in ReMIND2Reg have no annotations, whereas validation and test performance are evaluated on manually annotated anatomical landmarks (Dorent et al., 13 Aug 2025). The landmark protocol begins with automatic detection of salient features via 3D‑SIFT in pre-dural and post-resection iUS, transfers features from pre-dural iUS to preoperative MRI using affine registration, and then has two experts with at least five years of experience identify approximately 10 corresponding anatomical landmarks for post-resection iUS–T2‑SPACE pairs, alternating selection and consensus refinement (Dorent et al., 13 Aug 2025). Landmarks lacking unanimous agreement were discarded (Dorent et al., 13 Aug 2025). ceT1 landmarks are obtained by computing an affine T2→ceT1 registration using NiftyReg and warping T2 landmarks (Dorent et al., 13 Aug 2025).
The benchmark notes prior inter-rater variability of 1.89 ± 0.37 mm in similar pre-dura versus post-resection settings and states that variability may increase because of the extra MR transfer step (Dorent et al., 13 Aug 2025). Landmark counts vary by case: validation cases contain 6 to 18 landmarks, and test cases contain 4 to 16 (Dorent et al., 13 Aug 2025).
Registration quality is assessed using three quantities: Target Registration Error (TRE), robustness to worst-case landmark misalignment (TRE30), and runtime (Dorent et al., 13 Aug 2025). TRE is defined as the Euclidean distance between corresponding landmarks in fixed iUS and warped moving MRI:
TRE30 is defined as the “30th percentile of the largest landmark distances,” intended to highlight worst-case performance (Dorent et al., 13 Aug 2025). If the landmark errors are sorted in descending order , then with ,
Runtime is reported as mean end-to-end execution time per case, including image I/O, measured on an NVIDIA A100 GPU with 40 GB memory (Dorent et al., 13 Aug 2025).
The ranking procedure is statistically structured to accommodate variable landmark counts. It uses bootstrap samples, selecting landmarks per case without replacement, where equals the minimum number of landmarks across cases: for validation and for test (Dorent et al., 13 Aug 2025). TRE ranking uses a paired Wilcoxon signed-rank test with one-sided and no multiple-comparison adjustment; TRE30 ranking uses an unpaired Wilcoxon rank-sum test because landmark sets differ between methods for the worst-case statistic (Dorent et al., 13 Aug 2025). The bootstrap task rank score is the geometric mean of TRE and TRE30 rank scores, and the final task rank score is the average over all bootstraps (Dorent et al., 13 Aug 2025). A plausible implication is that the benchmark treats mean accuracy and upper-tail failure modes as co-equal ranking criteria rather than subordinating robustness to average-case performance.
4. Submission protocol, assets, and benchmark rules
ReMIND2Reg standardizes data splits, paired modalities, common preprocessing, evaluation metrics, ranking, and submission mechanics (Dorent et al., 13 Aug 2025). Validation is hosted on Grand Challenge, while evaluation code is publicly available through the ReMIND2Reg repository (Dorent et al., 13 Aug 2025). Validation submissions consist of per-case predictions uploaded through the platform, and incomplete validation submissions are rejected (Dorent et al., 13 Aug 2025). For the test phase, participants submit containerized methods in Docker, one submission per team, and these containers are executed on the organizers’ cluster against the private test landmarks (Dorent et al., 13 Aug 2025).
The benchmark explicitly allows external data and pre-trained models (Dorent et al., 13 Aug 2025). Organizers’ team members may participate but are ineligible for awards (Dorent et al., 13 Aug 2025). The test set remains private, and the output for each case must be sufficient for landmark-based evaluation, such as a transform or warped landmarks, with exact file specifications defined on the platform and in the evaluation repository (Dorent et al., 13 Aug 2025).
The paper does not report baseline methods or official benchmark numbers for TRE, TRE30, or runtime, and specifically notes that NiftyReg affine registration is mentioned only as part of the annotation pipeline rather than as an official MRI→iUS benchmark baseline (Dorent et al., 13 Aug 2025). Participants are therefore instructed not to assume any official baseline (Dorent et al., 13 Aug 2025). This absence is methodologically significant: ReMIND2Reg is presented primarily as an evaluation infrastructure rather than as a leaderboard paper centered on comparative algorithmic analysis.
The benchmark is contrasted with the original ReMIND dataset in that ReMIND2Reg pivots to the post-resection alignment problem, adds rigorous landmark-based evaluation, runtime measurement, standardized bootstrapped ranking, and multimodal pairing of ceT1 and T2‑SPACE with 3D iUS, and is described as being at larger scale than prior post-dural-opening benchmarks (Dorent et al., 13 Aug 2025).
5. Methods evaluated on ReMIND2Reg
Although the challenge paper does not provide official baselines, subsequent work reports benchmark results on ReMIND2Reg. “Unsupervised MRI-US Multimodal Image Registration with Multilevel Correlation Pyramidal Optimization” introduces MCPO, an unsupervised multimodal registration framework using Mind-SSC features, dense correlation analysis, multilevel pyramidal fusion, coupled convex optimization, inverse consistency, and optional instance-wise Adam refinement (Wang et al., 6 Feb 2026). That paper states that MCPO achieved first place in both the validation phase and test phase of ReMIND2Reg, with MCPO-deform obtaining a final normalized score of 0.911 on the organizer-run test evaluation (Wang et al., 6 Feb 2026). On the five-patient validation phase, the reported mean TRE values were 1.790 ± 0.536 mm for MCPO-rigid and 2.766 ± 1.001 mm for MCPO-deform; comparator methods included ConvexAdam-Rigid at 2.609 ± 1.217 mm, NiftyReg at 2.807 ± 1.228 mm, and MCBO at 2.367 ± 0.638 mm (Wang et al., 6 Feb 2026).
A different methodological direction appears in “A 3D Cross-modal Keypoint Descriptor for MR-US Matching and Registration,” which proposes a patient-specific 3D keypoint–descriptor pipeline based on matching-by-synthesis, probabilistic keypoint detection, and supervised contrastive or triplet-based descriptor learning (Morozov et al., 24 Jul 2025). Evaluated on ReMIND2Reg validation data, the method reports a mean TRE of 2.385 ± 0.397 mm for rigid registration and is described as ranking third among public leaderboard entries reported in that paper (Morozov et al., 24 Jul 2025). The same comparison table in that paper lists VROC at 1.903 ± 0.582 mm, next-gen-nn at 1.969 ± 0.459 mm, Coarse-to-Fine with 3D CycleGAN style transfer plus NiftyReg at 2.419 ± 0.669 mm, and a Topological Higher-Order MRF method at 3.680 ± 0.620 mm (Morozov et al., 24 Jul 2025).
These reports show that the benchmark supports both dense deformable registration methods and sparse keypoint-based rigid pipelines. This suggests that ReMIND2Reg functions not only as a challenge for modality-robust similarity modeling but also as a testbed for the trade-off between interpretable sparse correspondence strategies and high-capacity deformation models.
6. Practical considerations, limitations, and future directions
The benchmark description identifies several practices that are not mandated but are typically beneficial for this problem: MRI intensity standardization and skull stripping, ultrasound preprocessing such as despeckling or gain normalization, resolution matching and ROI cropping around the craniotomy or resection region, modality-robust similarity measures such as mutual information or cross-correlation variants, robust loss functions or masks for resection cavities, and learning of cross-modal representations such as MIND or learned features (Dorent et al., 13 Aug 2025). These are framed as direct responses to the benchmark’s stated pitfalls: modality gap, ultrasound artifacts, and cavity-induced topology change (Dorent et al., 13 Aug 2025).
The challenge paper also identifies several limitations. Landmark-based evaluation inherits annotation uncertainty, and the reported inter-rater variability from similar settings is approximately 1.89 ± 0.37 mm, with the possibility of larger variability due to the MR transfer step (Dorent et al., 13 Aug 2025). Large deformations and resection cavities challenge standard diffeomorphic models, so explicit cavity handling remains an open problem (Dorent et al., 13 Aug 2025). Modality gap remains central, and methods must avoid overfitting to specific ultrasound artifacts (Dorent et al., 13 Aug 2025). Data diversity is limited by the fact that iUS originates from a single center in the AMIGO suite, although MRI acquisition is multi-institutional and multi-vendor (Dorent et al., 13 Aug 2025).
Future directions named in the benchmark include uncertainty estimation, integration of multi-sequence MRI, physics-informed models of brain shift, and real-time deployment (Dorent et al., 13 Aug 2025). The emphasis on TRE30 as a robustness metric indicates that upper-tail failure modes are treated as clinically salient because large local misregistrations near critical structures may be more consequential than modest degradation in average error (Dorent et al., 13 Aug 2025). A plausible implication is that future ReMIND2Reg-style benchmarking may increasingly prioritize calibrated failure detection and uncertainty-aware deployment criteria in addition to geometric accuracy.
7. Significance within MRI–iUS registration research
ReMIND2Reg situates post-resection MRI→iUS registration as a standardized benchmark problem defined by multimodal disparity, incomplete MRI sequence availability, tracked-yet-residual alignment errors, and topology change induced by tissue removal (Dorent et al., 13 Aug 2025). Its stated aim is to accelerate the development of robust, generalizable, and clinically deployable multimodal registration algorithms for image-guided neurosurgery (Dorent et al., 13 Aug 2025).
Its significance lies in how it formalizes the evaluation regime. By combining unlabeled training data, private landmark-based validation and test sets, sequence-specific evaluation, runtime measurement, and bootstrapped Wilcoxon-based ranking, the benchmark imposes a reproducible structure on a task that had previously been fragmented across datasets and protocols (Dorent et al., 13 Aug 2025). The fact that later methods as different as MCPO’s multilevel convex optimization pipeline and CrossKEY’s patient-specific keypoint descriptor both report ReMIND2Reg results underscores the benchmark’s role as a shared comparison target across otherwise divergent methodological traditions (Wang et al., 6 Feb 2026, Morozov et al., 24 Jul 2025).
In summary, ReMIND2Reg is a benchmark for post-resection MRI–iUS registration under clinically realistic brain-shift conditions, with 99/5/10 train/validation/test cases, 155/10/20 paired MR–iUS image pairs, standardized 0.5 mm isotropic 256³ preprocessing, landmark-based TRE and TRE30 evaluation, and Docker-based private test execution (Dorent et al., 13 Aug 2025). Its structure makes it a reference benchmark for clinically oriented multimodal registration in image-guided neurosurgery.