RadGS-Reg: 3D CT/X-ray Spine Registration

• RadGS-Reg is a vertebral-level CT/X-ray registration framework that reconstructs 3D vertebral volumes from biplanar X-rays using a radiative Gaussian representation.
• It employs Counterfactual Attention Learning to mitigate noise and overlapping anatomical structures, ensuring robust reconstruction and registration.
• The method achieves state-of-the-art performance with sub-millimetric accuracy and near real-time inference (0.82 s per vertebra) through rigid 6-DOF pose estimation.

Searching arXiv for the main paper and directly related background on 3D Gaussian/X-ray methods. RadGS-Reg is a vertebral-level CT/X-ray registration framework that formulates intraoperative alignment as a joint problem of 3D reconstruction and 3D/3D registration rather than a conventional iterative 2D/3D “render and compare” loop. In the formulation introduced in “RadGS-Reg: Registering Spine CT with Biplanar X-rays via Joint 3D Radiative Gaussians Reconstruction and 3D/3D Registration,” the method reconstructs a vertebra-specific 3D Radiative Gaussians (RadGS) representation from calibrated AP/LA biplanar X-rays, voxelizes that representation into a volumetric density, and then registers the reconstructed volume to the preoperative CT vertebra with a rigid 6-DOF transform (Shen et al., 28 Aug 2025). The framework is designed for image-guided spine navigation, where the stated requirements are vertebral-level sub-millimetric accuracy and near real-time inference, and where traditional DRR-based registration is limited by spatial information loss, domain gap, and sensitivity to noisy fluoroscopic appearance (Shen et al., 28 Aug 2025).

1. Problem setting and conceptual basis

RadGS-Reg addresses intraoperative alignment of biplanar X-rays to a preoperative CT in spine navigation. Its central premise is that direct 2D/3D registration via repeated projection to digitally reconstructed radiographs discards depth and shape information, while also exposing the optimization to the discrepancy between simulated DRRs and real X-rays. The framework therefore moves the problem into a 3D/3D regime by reconstructing a vertebral volume from two calibrated X-ray views and aligning that reconstructed volume to the CT vertebra (Shen et al., 28 Aug 2025).

The method is organized at vertebral level. AP/LA X-rays with calibrated geometry and a preoperative CT are preprocessed so that vertebral levels are localized on X-rays and the corresponding vertebra is segmented on CT. This produces a per-vertebra workflow rather than a global spine registration pipeline. Within that workflow, the reconstruction module, denoted RecM, predicts a RadGS representation from the two X-rays, and the registration module, denoted RegM, estimates a rigid transform between the voxelized RadGS volume and the vertebral CT volume (Shen et al., 28 Aug 2025).

The conceptual departure from classical render-and-compare pipelines is explicit. RadGS-Reg is described as using Radiative Gaussians to bridge the domain gap, avoid iterative DRR rendering loops, and enable efficient differentiable rendering and voxelization. A plausible implication is that the method treats X-ray formation and volume alignment as mutually informative learned components, rather than as separate handcrafted stages.

2. 3D Radiative Gaussians reconstruction from biplanar X-rays

RadGS extends 3D Gaussian Splatting to X-ray rendering by parameterizing a volumetric density as a sum of anisotropic Gaussians and projecting them differentiably under the Beer–Lambert law (Shen et al., 28 Aug 2025). Each Gaussian $i$ has learnable central density $\rho_i$, center position $p_i \in \mathbb{R}^3$, and covariance $\Sigma_i \in \mathbb{R}^{3\times 3}$. The RadGS set is written as

$$\mathcal{G}^3 = \{ G_i^3(\rho_i, p_i, \Sigma_i) \}_{i=1..N}.$$

The corresponding volumetric density is

$$\rho(x) = \sum_i \rho_i \mathcal{N}(x; p_i, \Sigma_i).$$

X-ray intensity at detector coordinate $u$ is modeled as

$I(u) = I_0 \exp\big(- \int_L \rho(x)\,dx\big),$

with the line integral decomposed into per-Gaussian terms,

$$I(u) = I_0 \exp\left( - \sum_i \tau_i(u) \right), \qquad \tau_i(u) = \rho_i \int \mathcal{N}\!\big(r(t); p_i, \Sigma_i\big)\, dt.$$

The paper states that the rasterizer from RadGS/R$^2$-Gaussian provides an efficient differentiable approximation for $\rho_i$0 and its gradients with respect to $\rho_i$1 (Shen et al., 28 Aug 2025).

RecM encodes the biplanar X-rays, applies attention-enhanced feature processing, and uses a GSHead to predict per-Gaussian parameters of $\rho_i$2. The predicted Gaussians are consumed by two operators: an X-ray rasterizer that generates simulated projections and a density voxelizer that converts the Gaussian set into a volumetric grid. Reconstruction supervision is then applied in both image and volume domains through photometric, structural, and total-variation terms (Shen et al., 28 Aug 2025).

The paper also presents a discrete compositing interpretation of the Beer–Lambert integral: $\rho_i$3 with

$\rho_i$4

This situates RadGS-Reg within differentiable volumetric rendering while preserving an explicit Gaussian parameterization (Shen et al., 28 Aug 2025).

3. Counterfactual Attention Learning and progressive pre-training

A defining component of RadGS-Reg is Counterfactual Attention Learning (CAL), introduced to address noisy X-rays and overlapping anatomy. The goal is to emphasize the target vertebra and suppress confounding structures such as ribs or adjacent vertebrae. The formulation uses encoder features $\rho_i$5, factual attention $\rho_i$6, and a counterfactual attention $\rho_i$7 created through randomized intervention $\rho_i$8. With GSHead $\rho_i$9, the causal RadGS output is defined as

$p_i \in \mathbb{R}^3$0

The corresponding CAL reconstruction loss is

$p_i \in \mathbb{R}^3$1

with $p_i \in \mathbb{R}^3$2 and $p_i \in \mathbb{R}^3$3 in the reported experiments (Shen et al., 28 Aug 2025).

The paper attributes two effects to CAL: it penalizes the output gap between factual and counterfactual attentions, and it improves robustness to confounders arising from overlap and noise. In the reported ablation, CAL improves reconstruction under simulated pretraining from $p_i \in \mathbb{R}^3$4 SSIM to $p_i \in \mathbb{R}^3$5 SSIM (Shen et al., 28 Aug 2025).

A second central component is the three-stage patient-specific pre-training strategy. Stage 1 uses VERSE’20 CTs with simulated biplanar DRRs perturbed by $p_i \in \mathbb{R}^3$6 AP/LA. The paper reports 32k DRR pairs for RecM and 1,280 CT-CT pairs for RegM with randomized rigid poses. Stage 2 uses in-house real AP/LA X-rays, specifically 10 cases and 30 vertebral-level samples with five-fold cross-validation, to reduce the real-vs-simulated domain gap. Stage 3 is patient-specific: for each fold, the validation preoperative CT is used to synthesize 1,800 biplanar DRR pairs and 600 CT-CT pairs so that vertebral shape priors and camera geometry are adapted to the target case (Shen et al., 28 Aug 2025).

The quantitative effect of this curriculum is large. With only Stage 1, the system reports $p_i \in \mathbb{R}^3$7 SSIM, $p_i \in \mathbb{R}^3$8 dB PSNR, $p_i \in \mathbb{R}^3$9 mm mTRE, $\Sigma_i \in \mathbb{R}^{3\times 3}$0 mm capture range, and $\Sigma_i \in \mathbb{R}^{3\times 3}$1 success rate. Adding Stage 2 raises performance to $\Sigma_i \in \mathbb{R}^{3\times 3}$2 SSIM, $\Sigma_i \in \mathbb{R}^{3\times 3}$3 dB PSNR, $\Sigma_i \in \mathbb{R}^{3\times 3}$4 mm mTRE, $\Sigma_i \in \mathbb{R}^{3\times 3}$5 mm capture range, and $\Sigma_i \in \mathbb{R}^{3\times 3}$6 success rate. Full three-stage pre-training yields $\Sigma_i \in \mathbb{R}^{3\times 3}$7 SSIM, $\Sigma_i \in \mathbb{R}^{3\times 3}$8 dB PSNR, $\Sigma_i \in \mathbb{R}^{3\times 3}$9 mm mTRE, $$\mathcal{G}^3 = \{ G_i^3(\rho_i, p_i, \Sigma_i) \}_{i=1..N}.$$0 mm capture range, and $$\mathcal{G}^3 = \{ G_i^3(\rho_i, p_i, \Sigma_i) \}_{i=1..N}.$$1 success rate (Shen et al., 28 Aug 2025).

4. 3D/3D registration module and joint learning objective

RegM performs rigid 3D/3D registration between the voxelized RadGS reconstruction and the preoperative CT vertebra. The predicted pose is an element of $$\mathcal{G}^3 = \{ G_i^3(\rho_i, p_i, \Sigma_i) \}_{i=1..N}.$$2,

$$\mathcal{G}^3 = \{ G_i^3(\rho_i, p_i, \Sigma_i) \}_{i=1..N}.$$3

The registration loss is

$$\mathcal{G}^3 = \{ G_i^3(\rho_i, p_i, \Sigma_i) \}_{i=1..N}.$$4

with $$\mathcal{G}^3 = \{ G_i^3(\rho_i, p_i, \Sigma_i) \}_{i=1..N}.$$5. Here $$\mathcal{G}^3 = \{ G_i^3(\rho_i, p_i, \Sigma_i) \}_{i=1..N}.$$6 is described as a geodesic distance between poses, combining a rotation geodesic on $$\mathcal{G}^3 = \{ G_i^3(\rho_i, p_i, \Sigma_i) \}_{i=1..N}.$$7 and translation distance (Shen et al., 28 Aug 2025).

The full system is trained synergistically: $$\mathcal{G}^3 = \{ G_i^3(\rho_i, p_i, \Sigma_i) \}_{i=1..N}.$$8 The paper states that RecM and RegM are first trained independently and then jointly optimized with $$\mathcal{G}^3 = \{ G_i^3(\rho_i, p_i, \Sigma_i) \}_{i=1..N}.$$9. This coupling is presented as beneficial in both directions: reconstruction is guided toward the CT target shape, while registration benefits from reconstruction consistency (Shen et al., 28 Aug 2025).

RegM itself is described as a volume encoder and pose head operating on concatenated voxelized RadGS and CT vertebra volumes. The paper does not specify a closed-form optimizer, because the transform is directly regressed by the network rather than estimated by iterative classical registration. That design choice is directly related to runtime: the reported end-to-end inference is $$\rho(x) = \sum_i \rho_i \mathcal{N}(x; p_i, \Sigma_i).$$0 s per vertebra case (Shen et al., 28 Aug 2025).

The distinction between RadGS-Reg and traditional 2D/3D registration is therefore not merely representational. The method replaces iterative pose search over projected images with learned volumetric alignment between a reconstructed intraoperative density and the preoperative CT, supervised by volumetric similarity and pose geodesics (Shen et al., 28 Aug 2025).

5. Implementation, datasets, and empirical performance

The implementation described in the paper uses Adam for 300 epochs, with initial learning rate $$\rho(x) = \sum_i \rho_i \mathcal{N}(x; p_i, \Sigma_i).$$1, halved every 50 epochs, and batch size $$\rho(x) = \sum_i \rho_i \mathcal{N}(x; p_i, \Sigma_i).$$2. Hardware consists of an AMD EPYC 7R32 with 48 cores, 128 GB RAM, and an NVIDIA RTX 4090 with 24 GB (Shen et al., 28 Aug 2025).

For RecM, evaluated backbones include ResNet-50, DenseNet-121, and ViT-12. CAL is implemented as Linear–ReLU–Linear, and GSHead predicts $$\rho(x) = \sum_i \rho_i \mathcal{N}(x; p_i, \Sigma_i).$$3. RegM consists of a volume encoder and pose head on the concatenated voxelized RadGS and CT vertebra volumes. Preprocessing is based on vertebral-level localization on X-rays and CT vertebral segmentation using the cited ABLSpineLevelCheck and SCN + U-Net pipeline (Shen et al., 28 Aug 2025).

The evaluation uses VERSE’20 and an in-house intraoperative dataset. VERSE’20 contains 253 CTs and 1,280 vertebrae. DRRs are synthesized by a C-arm simulator at $$\rho(x) = \sum_i \rho_i \mathcal{N}(x; p_i, \Sigma_i).$$4 resolution with focal length $$\rho(x) = \sum_i \rho_i \mathcal{N}(x; p_i, \Sigma_i).$$5 mm and $$\rho(x) = \sum_i \rho_i \mathcal{N}(x; p_i, \Sigma_i).$$6 AP/LA perturbations. The in-house dataset consists of 10 intraoperative cases with preoperative CTs of size $$\rho(x) = \sum_i \rho_i \mathcal{N}(x; p_i, \Sigma_i).$$7, spacing $$\rho(x) = \sum_i \rho_i \mathcal{N}(x; p_i, \Sigma_i).$$8 mm, and paired AP/LA X-rays of size $$\rho(x) = \sum_i \rho_i \mathcal{N}(x; p_i, \Sigma_i).$$9 with focal length $u$0 mm. The evaluation uses 30 vertebral-level samples and five-fold cross-validation (Shen et al., 28 Aug 2025).

The principal reported results are summarized below.

Task	Method	Reported result
Reconstruction	DiffVox	SSIM $u$1, PSNR $u$2 dB
Reconstruction	3DGR	SSIM $u$3, PSNR $u$4 dB
Reconstruction	SAX-NeRF	SSIM $u$5, PSNR $u$6 dB
Reconstruction	R$u$7-GS	SSIM $u$8, PSNR $u$9 dB
Reconstruction	Ours	SSIM $I(u) = I_0 \exp\big(- \int_L \rho(x)\,dx\big),$0, PSNR $I(u) = I_0 \exp\big(- \int_L \rho(x)\,dx\big),$1 dB
Registration	DiffPose	mTRE $I(u) = I_0 \exp\big(- \int_L \rho(x)\,dx\big),$2 mm, CR $I(u) = I_0 \exp\big(- \int_L \rho(x)\,dx\big),$3 mm, SR $I(u) = I_0 \exp\big(- \int_L \rho(x)\,dx\big),$4, RT $I(u) = I_0 \exp\big(- \int_L \rho(x)\,dx\big),$5 s
Registration	DDGS-CT	mTRE $I(u) = I_0 \exp\big(- \int_L \rho(x)\,dx\big),$6 mm, CR $I(u) = I_0 \exp\big(- \int_L \rho(x)\,dx\big),$7 mm, SR $I(u) = I_0 \exp\big(- \int_L \rho(x)\,dx\big),$8, RT $I(u) = I_0 \exp\big(- \int_L \rho(x)\,dx\big),$9 s
Registration	TS-SAR	mTRE $$I(u) = I_0 \exp\left( - \sum_i \tau_i(u) \right), \qquad \tau_i(u) = \rho_i \int \mathcal{N}\!\big(r(t); p_i, \Sigma_i\big)\, dt.$$0 mm, CR $$I(u) = I_0 \exp\left( - \sum_i \tau_i(u) \right), \qquad \tau_i(u) = \rho_i \int \mathcal{N}\!\big(r(t); p_i, \Sigma_i\big)\, dt.$$1 mm, SR $$I(u) = I_0 \exp\left( - \sum_i \tau_i(u) \right), \qquad \tau_i(u) = \rho_i \int \mathcal{N}\!\big(r(t); p_i, \Sigma_i\big)\, dt.$$2, RT $$I(u) = I_0 \exp\left( - \sum_i \tau_i(u) \right), \qquad \tau_i(u) = \rho_i \int \mathcal{N}\!\big(r(t); p_i, \Sigma_i\big)\, dt.$$3 s
Registration	Ours	mTRE $$I(u) = I_0 \exp\left( - \sum_i \tau_i(u) \right), \qquad \tau_i(u) = \rho_i \int \mathcal{N}\!\big(r(t); p_i, \Sigma_i\big)\, dt.$$4 mm, CR $$I(u) = I_0 \exp\left( - \sum_i \tau_i(u) \right), \qquad \tau_i(u) = \rho_i \int \mathcal{N}\!\big(r(t); p_i, \Sigma_i\big)\, dt.$$5 mm, SR $$I(u) = I_0 \exp\left( - \sum_i \tau_i(u) \right), \qquad \tau_i(u) = \rho_i \int \mathcal{N}\!\big(r(t); p_i, \Sigma_i\big)\, dt.$$6, RT $$I(u) = I_0 \exp\left( - \sum_i \tau_i(u) \right), \qquad \tau_i(u) = \rho_i \int \mathcal{N}\!\big(r(t); p_i, \Sigma_i\big)\, dt.$$7 s

These results support the paper’s claim that the method achieves state-of-the-art performance for both reconstruction and registration on the reported in-house data (Shen et al., 28 Aug 2025). The paper also states that performance is model-agnostic across ResNet-50, DenseNet-121, and ViT-12, with DenseNet and ViT variants retaining mTRE around $$I(u) = I_0 \exp\left( - \sum_i \tau_i(u) \right), \qquad \tau_i(u) = \rho_i \int \mathcal{N}\!\big(r(t); p_i, \Sigma_i\big)\, dt.$$8 mm (Shen et al., 28 Aug 2025).

6. Position within the literature, limitations, and scope

RadGS-Reg is situated in a line of work that uses Gaussian-based volumetric representations to support medical X-ray reconstruction and registration. The paper explicitly compares against DiffVox, 3DGR, SAX-NeRF, and R$$I(u) = I_0 \exp\left( - \sum_i \tau_i(u) \right), \qquad \tau_i(u) = \rho_i \int \mathcal{N}\!\big(r(t); p_i, \Sigma_i\big)\, dt.$$9-GS, and uses the rasterizer from RadGS/R$^2$0-Gaussian for efficient differentiable approximation of ray integrals (Shen et al., 28 Aug 2025). More broadly, its reconstruction-first strategy contrasts with classical CT/X-ray registration pipelines that optimize similarity between real X-rays and rendered DRRs.

The stated limitations are specific. The method requires biplanar views with accurate calibration. Extreme noise or severe occlusion can still degrade reconstruction. Registration is currently rigid per vertebra, so non-rigid effects such as soft tissue motion and intervertebral flexibility are not modeled. The full pipeline also depends on vertebral-level detection and segmentation preprocessing (Shen et al., 28 Aug 2025).

The paper identifies three future directions: incorporation of deformation models for non-rigid anatomy, post-reconstruction Gaussian segmentation to reduce dependence on preprocessing, and broader clinical validation across anatomies and C-arm systems (Shen et al., 28 Aug 2025). These directions are consistent with the current design: the framework is already vertebra-centric, calibrated, and rigid, so extending it to deformable and less preprocessed settings would require changes in both the representation and the loss formulation.

A further point of clarification is terminological. In the supplied literature, similar strings such as “RadGS-Reg” or “Reg” also appear in unrelated contexts, including radar-camera 3D detection and radar-vision SLAM, where “Reg” denotes a detection regression head or a registration component rather than the spine CT/X-ray method described here (Bai et al., 26 Jul 2025). In the present sense, however, RadGS-Reg refers specifically to the joint 3D Radiative Gaussians reconstruction and 3D/3D registration framework for vertebral CT/X-ray navigation (Shen et al., 28 Aug 2025).