---
title: CBCT-to-MDCT Translation
url: https://www.emergentmind.com/topics/cbct-to-mdct-translation
type: topic
---

# CBCT-to-MDCT Translation

CBCT-to-MDCT translation denotes the synthesis of a CT-like or synthetic CT (sCT) volume from cone-beam CT acquired during treatment or intraoperatively, with the objective of preserving the treatment-day anatomy represented by CBCT while recovering the Hounsfield-unit fidelity, contrast characteristics, and artifact profile of planning or diagnostic multi-detector CT. In radiotherapy, the target is usually a planning CT or a deformed planning CT aligned to CBCT geometry rather than a generic diagnostic CT, so the task is often framed as CBCT-to-CT or CBCT-to-sCT synthesis. The central technical difficulty is to correct scatter, beam hardening, shading or cupping, streaks, truncation, and noise without introducing anatomical hallucinations or erasing clinically relevant detail. Contemporary work spans paired supervised synthesis, unpaired distribution-level transport, physics-based generation of synthetic training pairs, multimodal fusion with preoperative CT, and conditional generative modeling based on GANs, diffusion, Schrödinger bridges, and flow matching [2308.13553][2409.18355][2303.02649][2603.00535].

## 1. Clinical role and task definition

CBCT-to-MDCT translation is principally motivated by radiotherapy planning and verification. CBCT is widely used for image-guided radiotherapy and adaptive radiotherapy because it is acquired on treatment machines and reflects the patient’s current anatomy, but it has poor quantitative accuracy due to scatter, shading, streaking, noise, and related artifacts. Planning CT, by contrast, provides calibrated Hounsfield units and remains the reference for electron-density-based dose calculation. The operative mapping is therefore from on-treatment CBCT to planning-CT-like sCT in the geometry of the treatment day, enabling dose recalculation, adaptive replanning, daily dose accumulation, and improved contouring or assessment [2308.13553][2603.00535].

Several works make this target explicit by using planning CT, deformed planning CT, or challenge-provided registered CT as ground truth. In the conditional DDPM and conditional flow-matching studies, the reference image is a deformed planning CT aligned to CBCT, so the learned target is effectively planning-CT quality in CBCT anatomy [2303.02649][2603.05796]. In the 2.5D SynthRAD2023 study, the target CT is the planning CT from a radiotherapy dataset, and the authors note that the focus is radiotherapy planning or verification CT rather than diagnostic CT use [2308.13553]. ARTInp extends the same logic to discontinuous CBCT in the setting of Total Marrow and Lymph Node Irradiation, where multiple CBCT acquisitions can leave anatomical gaps that must be inpainted before CT-like synthesis [2502.04898].

A recurrent misconception is that CBCT-to-MDCT translation is merely denoising. The literature instead treats it as a coupled problem of artifact suppression, HU restoration, and anatomical preservation. That distinction matters because a visually cleaner image can still be dosimetrically inadequate if HU calibration or boundary fidelity is wrong. This is why many studies evaluate not only image similarity metrics but also segmentation consistency or dose-related endpoints when such endpoints are available [2312.02017][2103.05690].

## 2. Data regimes, pairing assumptions, and preprocessing

The field uses several supervision regimes. In paired supervised settings, challenge-style datasets provide registered CBCT and CT volumes for direct slice-wise or voxel-wise learning. The SynthRAD2023 study reports four datasets—MRI-to-sCT brain, MRI-to-sCT pelvis, CBCT-to-sCT brain, and CBCT-to-sCT pelvis—with 162 training cases and 18 validation cases per dataset [2308.13553]. ARTInp uses the SynthRad 2023 brain cohort of 180 patients and a random patient-wise split of 144 training, 18 validation, and 18 test cases, with rigid CBCT-CT registration via Elastix already available in the distributed data [2502.04898]. The SynthRAD2025 nnResU-Net work uses rigid registration from the challenge preprocessing and then deformable registration with elastix, retaining all CBCT-to-CT volumes because registration quality was considered sufficient for Task 2 [2509.22394].

Unpaired regimes arise because high-quality voxel-wise CBCT-CT pairs are often unavailable or unreliable. RAFM formalizes a strict subject-level true-unpaired protocol on SynthRAD2023 pelvis: 126 training subjects are split into 63 CBCT-only and 63 CT-only subjects with no overlap, while paired volumes are reserved for validation and test [2603.00535]. This design encodes a clinically realistic situation in which legacy planning CT archives and CBCT cohorts are disjoint. A related implication, stated across several papers, is that temporal gaps, anatomy change, and imperfect registration can make nominally paired data a weak supervisory signal even when both modalities exist [2603.00535][2506.08654].

A third regime uses CT-only data to synthesize aligned CBCT-like inputs. SinoSynth starts from planning CT volumes, projects them through a cone-beam forward model with randomized geometry and artifact simulation in the sinogram domain, and reconstructs simulated CBCT by FDK, thereby producing arbitrarily many aligned sCBCT-CT pairs without real paired acquisitions [2409.18355]. The multitask 3D cGAN work for lung follows a related strategy: artifact components are extracted from registered week-1 CBCT, mapped onto planning CT, forward-projected, and reconstructed by OS-SART, yielding 17 perfectly registered psCBCT-pCT-label triplets per patient and 1360 such 3D volumes from 80 internal cases, plus 732 augmented AAPM triplets [2103.05690].

Preprocessing is correspondingly heterogeneous. Some methods deliberately keep it minimal; the 2.5D EfficientNet-U-Net paper explicitly mentions histogram normalization only for MRI and no explicit intensity normalization or clipping for CBCT [2308.13553]. Others impose aggressive harmonization, such as clipping CBCT and CT to \([-1024,3071]\) HU with dataset-level \(z\)-score normalization in the residual nnU-Net pipeline [2509.22394], clipping to \([-1024,2000]\) HU and normalizing to \([-1,1]\) in RAFM [2603.00535], or transforming HU to linear attenuation coefficients after clipping to \([-1000,200]\) in the pulmonary feature-oriented framework [2311.00412]. This variation is methodologically important because claims of generalization often depend as much on preprocessing and registration policy as on network architecture.

## 3. Architectural families and representational choices

Early CBCT-to-PlanCT work by Kida et al. used an unpaired 2D CycleGAN for pelvic CBCT, augmented with air-preservation, gradient, total-variation, and idempotence losses to maintain anatomy while translating to PlanCT-like appearance [1901.05773]. A later multi-channel CycleGAN encoded each slice with three windowed channels—full dynamic range, soft-tissue window, and high-density window—and added an auxiliary fusion network to combine the translated channels into a single HU-like image on SynthRAD2023 [2312.02017]. These architectures treat CBCT-to-MDCT as style transfer under structural constraints.

Supervised encoder-decoder models span 2D, 2.5D, and 3D formulations. The 2.5D SynthRAD2023 method uses a U-Net-style decoder with an EfficientNet-B7 encoder and \(N=3\) consecutive transverse slices as input channels, predicting one sCT slice per stack and reconstructing the volume slice-by-slice [2308.13553]. The lung multitask cGAN uses a full 3D conditional GAN with a \(128^3\) input volume and joint outputs for synthetic CT and organs-at-risk segmentation, thereby coupling image translation with segmentation supervision in a single volumetric model [2103.05690]. The adapted nnU-Net line extends self-configuring 3D U-Net design to synthesis, with both standard nnU-Net and residual nnU-Net variants, region-specific patch sizes, and a subsequent AFP fine-tuning stage [2509.22394].

A distinct line introduces multimodal fusion with preoperative CT. One 3D U-Net study concatenates CBCT and preoperative CT as two input channels and shows that multimodal sCT can outperform CBCT-only baselines, especially in well-aligned, low-quality CBCT settings [2506.08716]. A later extension adds an end-to-end affine Spatial Transformer Network that learns CT-to-CBCT alignment jointly with sCT generation, using the registered CT and CBCT as a two-channel input to the 3D U-Net [2507.06067]. These works treat preoperative CT as a high-quality anatomical prior rather than merely a target.

Generative transport models now occupy a substantial part of the literature. Conditional DDPMs learn \(p_\theta(x_0\mid y)\) by predicting diffusion noise on CT slices conditioned on CBCT [2303.02649]. Patient-specific score-based priors train an unconditional diffusion model on the same patient’s planning CT and solve CBCT-to-sCT as a Bayesian inverse problem with a CBCT data-fidelity term and \(z\)-axis TV regularization [2406.15219]. EqDiff-CT replaces the conventional U-Net denoiser with a \(C_4\)-equivariant conditional U-Net built with e2cnn steerable layers and equivariant attention [2509.21913]. Flow-based approaches include RAFM, which adapts rectified flow to subject-level unpaired training through retrieval-guided pseudo pairs [2603.00535], and a supervised conditional flow-matching model that gradually transforms a normal sample into CT conditioned on CBCT for brain, head-and-neck, and lung cohorts [2603.05796]. Human-guided Schrödinger-bridge translation adds classifier-free guidance and tournament-style binary preference selection to steer CBCT-to-MDCT refinement toward clinically preferred shade-artifact suppression while preserving boundaries [2507.11025].

## 4. Objectives, constraints, and mechanisms of anatomical preservation

Loss design is unusually diverse because the field tries to optimize HU accuracy, artifact suppression, and anatomy simultaneously. At one extreme, the 2.5D EfficientNet-U-Net study uses only per-slice \(L_1\) loss in HU between predicted sCT and ground-truth CT, optimized with AdamW and cosine annealing, with no additional SSIM, adversarial, gradient, or perceptual terms [2308.13553]. ARTInp uses the classical pix2pix objective \(L_{\mathrm{GAN}}+\lambda L_1\) for the CBCT-to-CT translator, while its completion network combines a masked MSE term with adversarial losses from global and local discriminators [2502.04898]. The lung multitask cGAN adopts an LSGAN objective plus \(\lambda=100\) \(L_1\) loss on concatenated CT and label channels, thereby letting segmentation accuracy emerge from joint adversarial and reconstruction supervision rather than from a separate Dice term [2103.05690].

Other methods inject stronger anatomical or physics priors. SinoSynth adds two explicit consistency terms on top of a base generator loss: an image-domain structure-consistency loss, \(\mathcal{L}_{\text{struc}}=\mathbb{E}_y[\|\mathcal{D}(G_\theta(y))-y\|_1]\), and a sinogram-consistency loss, \(\mathcal{L}_{\text{sino}}=\mathbb{E}_{x,y}[\|M\odot(\mathcal{R}(G_\theta(y))-\mathcal{R}(x))\|_1]\), so that generated CT remains consistent with the differentiable CBCT degradation model [2409.18355]. The Anatomical Feature-Prioritized loss computes \(\frac{1}{N}\sum_i \|\phi_i(x)-\phi_i(y)\|_1\) between multilayer features from a compact segmentation network trained on TotalSegmentator-derived labels, and is combined with \(5.0\cdot L_1\) during fine-tuning of nnU-Net and nnResU-Net [2509.22394]. The pulmonary feature-oriented framework similarly replaces generic VGG perceptual loss with a customized feature-to-feature perceptual loss derived from a multi-task encoder trained for CT reconstruction, CBCT-CT registration, and CBCT-versus-CT classification [2311.00412].

Generative transport models encode structure preservation differently. RAFM uses the rectified-flow objective \(\mathbb{E}_{(x_0,x_1)\sim\rho_{\text{retr}},\, t\sim\mathcal{U}(0,1)}[\|v_\theta((1-t)x_0+t x_1,t)-(x_1-x_0)\|_2^2]\), where the crucial design choice is retrieval-guided pseudo pairing via a frozen DINOv3 encoder and a global CT memory bank rather than random coupling [2603.00535]. Human-guided Schrödinger-bridge translation conditions the score network on CBCT, diffusion time, and binary human preference \(r\), then uses classifier-free guidance to bias sampling toward “good” outputs without an explicit reward model [2507.11025]. The multimodal STN-U-Net pipeline optimizes a composite sCT loss \(0.2\cdot\mathrm{MAE}+0.1\cdot(1-\mathrm{SSIM})+0.7\cdot\mathrm{perceptual}\) together with a registration loss \(10^{-3}\cdot\mathrm{MSE}(V_{\mathrm{CT}},Y)\), so alignment becomes part of the image-synthesis objective rather than a frozen preprocessing step [2507.06067].

These choices reflect a broader methodological divide. One camp assumes that paired supervision plus simple intensity loss is sufficient if registration is good enough; another treats structure preservation as an explicit modeling problem that requires physics consistency, segmentation-derived features, retrieval, or human preference. The literature does not settle this divide, but it repeatedly shows that structural priors matter most when artifact severity, registration error, or domain shift are large.

## 5. Evaluation practice and reported performance

The most common image metrics are MAE in HU, PSNR, and SSIM or MS-SSIM. Several papers add NCC, FID, VIF, IFC, or anatomy-related metrics such as Dice, HD95, or SegScore. A smaller subset reports radiotherapy endpoints such as dose MAE, DVH difference, or gamma analysis [2308.13553][2312.02017][2103.05690][2603.00535][2509.22394].

| Setting | Dataset / regime | Reported result |
|---|---|---|
| 2.5D EfficientNet-U-Net [2308.13553] | SynthRAD2023 validation, CBCT-to-sCT | Brain: PSNR 27.38 dB, MAE 81.44 HU; pelvis: PSNR 28.12 dB, MAE 68.07 HU |
| 3D multitask cGAN [2103.05690] | Real week-1 lung CBCT to pCT-like sCT | MSSIM 0.92 ± 0.01, MAE 29.31 ± 12.64 HU, PSNR 34.69 ± 2.41 dB |
| Multi-channel CycleGAN [2312.02017] | SynthRAD2023 test | MAE 71.58 ± 13.79 HU, PSNR 28.34 ± 1.50 dB, SSIM 0.86 ± 0.04 |
| RAFM [2603.00535] | SynthRAD2023 pelvis, subject-level true-unpaired | MAE 101.2 HU, SSIM 80.96%, PSNR 25.15 dB, FID 53.29, SegScore 75.77% |
| Conditional DDPM [2303.02649] | Brain and H&N | Brain: MAE 25.99 ± 11.84 HU, PSNR 30.49 ± 3.73 dB, NCC 0.99; H&N: MAE 32.56 ± 12.86 HU, PSNR 27.65 ± 2.41 dB, NCC 0.98 |
| Conditional flow matching [2603.05796] | Brain, H&N, lung | Brain: 26.02 HU, 32.35 dB, 0.99; H&N: 33.17 HU, 28.68 dB, 0.98; lung: 25.09 HU, 32.81 dB, 0.99 |
| Federated Pix2Pix [2506.08654] | External head-and-neck validation | MAE 75.22 ± 11.81 HU, SSIM 0.904 ± 0.034, PSNR 33.52 ± 2.06 dB |

Image metrics are not the whole story. On SynthRAD2023 test data, the multi-channel CycleGAN reported photon-plan dose MAE \(0.01 \pm 0.01\), DVH difference \(0.07 \pm 0.02\), and gamma index \(98.42\% \pm 4.95\%\) at \(2\%/2\) mm; the corresponding proton results were dose MAE \(0.07 \pm 0.04\), DVH difference \(0.27 \pm 0.27\), and gamma index \(92.32\% \pm 5.87\%\) [2312.02017]. The residual nnU-Net line illustrates another trade-off: nnResU-Net with \(L_1\) achieved the best intensity metrics on SynthRAD2025 Task 2, with MAE \(52.958 \pm 14.730\) HU, PSNR \(31.690 \pm 2.454\) dB, and MS-SSIM \(0.963 \pm 0.026\), whereas nnResU-Net with \(L_1+\)AFP achieved the best anatomy-related scores, Dice \(0.835 \pm 0.088\) and HD95 \(5.210 \pm 2.886\) [2509.22394].

The literature itself notes that developments in sCT generation have been difficult to compare because of the lack of large public datasets and sizeable variation in training procedures [2312.02017]. A plausible implication is that cross-paper numerical ranking should be interpreted cautiously: target definition, anatomy, registration quality, supervision regime, and the use of planning CT versus deformed planning CT vary substantially.

## 6. Limitations, controversies, and emerging directions

The most persistent limitation is registration. Paired supervised methods depend on CBCT-CT alignment quality, yet several papers explicitly note that temporal gaps, anatomical change, and deformable-registration error can blur supervision or make it unreliable [2308.13553][2603.00535][2506.08654]. Multimodal fusion papers show that better alignment monotonically improves sCT quality, while end-to-end STN registration helps primarily under moderate misalignment and low CBCT quality; large non-affine discrepancies remain difficult [2506.08716][2507.06067]. This registration problem explains the continued interest in unpaired learning, CT-only synthetic pair generation, and patient-specific priors.

A second controversy concerns dimensionality. The 2.5D literature argues that multi-slice input is a compromise between the memory burden of 3D CNNs and the inconsistency of pure 2D slice-wise processing [2308.13553]. Yet several newer methods still operate in 2D, including conditional diffusion, EqDiff-CT, and federated Pix2Pix, often because diffusion or multi-center training is already computationally heavy [2303.02649][2509.21913][2506.08654]. This suggests that through-plane consistency remains an open systems problem rather than a settled design choice. The RAFM paper explicitly identifies 3D RF or 2.5D extensions as a future direction for better volumetric consistency [2603.00535].

A third unresolved issue is the gap between image-level and clinical validation. Many papers report only MAE, PSNR, and SSIM and explicitly note the absence of dose recalculation, DVH analysis, or gamma evaluation [2308.13553][2502.04898][2509.22394][2506.08716]. Where dose endpoints are reported, performance can differ by modality; the multi-channel CycleGAN was notably stronger for photon than proton plans [2312.02017]. This suggests that HU fidelity sufficient for general image similarity is not automatically sufficient for all downstream radiotherapy tasks.

Current directions respond directly to these bottlenecks. Physics-based domain randomization and artifact simulation attempt to reduce scanner and protocol dependence without requiring real paired data [2409.18355]. Federated learning addresses cross-center heterogeneity and privacy constraints while preserving data locality [2506.08654]. Retrieval-augmented flow matching and conditional flow matching aim to keep the deterministic, non-adversarial behavior of transport models while reducing the sampling cost associated with diffusion [2603.00535][2603.05796]. Human-guided Schrödinger-bridge methods add controllability and require only 10 sampling steps, which directly targets real-time deployment constraints [2507.11025]. Patient-specific score-based priors point toward Bayesian formulations in which CBCT consistency and CT realism are both explicit, and the same paper argues that projection-domain data fidelity may further improve performance in severe metal-artifact cases [2406.15219]. Together, these developments indicate that CBCT-to-MDCT translation is evolving from a purely image-to-image problem into a broader problem of transport, physics, registration, and task-specific validation.

Source: https://www.emergentmind.com/topics/cbct-to-mdct-translation