The Impact of Loss Functions and Scene Representations for 3D/2D Registration on Single-view Fluoroscopic X-ray Pose Estimation (2308.00214v3)

Abstract: Many tasks performed in image-guided procedures can be cast as pose estimation problems, where specific projections are chosen to reach a target in 3D space. In this study, we first develop a differentiable projection (DiffProj) rendering framework for the efficient computation of Digitally Reconstructed Radiographs (DRRs) with automatic differentiability from either Cone-Beam Computerized Tomography (CBCT) or neural scene representations, including two newly proposed methods, Neural Tuned Tomography (NeTT) and masked Neural Radiance Fields (mNeRF). We then perform pose estimation by iterative gradient descent using various candidate loss functions, that quantify the image discrepancy of the synthesized DRR with respect to the ground-truth fluoroscopic X-ray image. Compared to alternative loss functions, the Mutual Information loss function can significantly improve pose estimation accuracy, as it can effectively prevent entrapment in local optima. Using the Mutual Information loss, a comprehensive evaluation of pose estimation performed on a tomographic X-ray dataset of 50 patients$'$ skulls shows that utilizing either discretized (CBCT) or neural (NeTT/mNeRF) scene representations in DiffProj leads to comparable performance in DRR appearance and pose estimation (3D angle errors: mean $\leq$ 3.2{\deg} and 90% quantile $\leq$ 3.4{\deg}), despite the latter often incurring considerable training expenses and time. These findings could be instrumental for selecting appropriate approaches to improve the efficiency and effectiveness of fluoroscopic X-ray pose estimation in widespread image-guided interventions.