MSRepaint: Unified Diffusion for MS Lesion Editing

Updated 14 July 2026

MSRepaint is a unified diffusion-based model for MS MRI that performs both lesion filling and synthesis via conditional inpainting.
The framework achieves high 3D consistency using a multi-view DDIM inversion and fusion pipeline while preserving surrounding anatomy.
Its bidirectional design, lesion dictionary augmentation, and contrast dropout enhance downstream analyses and segmentation training.

Searching arXiv for MSRepaint and related RePaint papers to ground the article. MSRepaint is a unified diffusion-based generative model for bidirectional multiple sclerosis lesion manipulation in magnetic resonance imaging. It is designed to perform both lesion filling, which produces lesion-free and anatomically plausible images for downstream analysis, and lesion synthesis, which generates realistic lesions in healthy scans for data augmentation. The framework treats both tasks as conditional image inpainting with explicit voxel-wise control via lesion masks, incorporates contrast dropout to handle missing inputs, uses a repainting mechanism to preserve surrounding anatomy, and introduces a multi-view DDIM inversion and fusion pipeline for 3D consistency with fast inference (Zhang et al., 2 Oct 2025).

1. Clinical motivation and problem formulation

MSRepaint addresses two linked problems in MS imaging: lesions break downstream anatomical analyses such as brain parcellation, volumetry, and deformable registration, and manually annotated lesion datasets are small and expensive, which limits lesion segmentation model development (Zhang et al., 2 Oct 2025). The method is therefore organized around two complementary tasks. In lesion filling, lesion effects are removed to restore normal anatomy for downstream processing. In lesion synthesis, realistic lesions are introduced into healthy scans to augment segmentation training.

The framework’s central formulation is to treat both directions as conditional image inpainting with explicit lesion-mask control. This unifies restoration and generation within one model rather than separating them into distinct pipelines. A plausible implication is that the same conditional machinery can be reused across preprocessing and augmentation settings, provided the lesion masks specify the desired edit.

2. Conditional diffusion model and training objective

MSRepaint builds on DDPM/DDIM-style diffusion modeling and RePaint-inspired inpainting. It trains a conditional denoiser on multicontrast 2D slices extracted from 3D MRI volumes in the axial, coronal, and sagittal planes. The input slice $x_0$ is a concatenation of T1w, T2w, and FLAIR channels, and the reverse model is a time-embedded U-Net denoiser $\phi_w$ conditioned on the lesion mask $M$ , with

$\hat{\epsilon}_t = \phi_w(M, x_t; t).$

The denoiser is used to estimate either $\hat{x}_0$ or $\hat{x}_{t-1}$ through the diffusion reparameterization (Zhang et al., 2 Oct 2025).

The training loss is the standard diffusion noise-prediction objective,

$\|\epsilon_t - \hat{\epsilon}_t\|^2,$

but it is modulated by lesion status. For lesion-free inputs, the loss is computed over the full image. For lesioned inputs, the loss is upweighted inside lesion regions by a factor of 10. This design is paired with contrast dropout: for each minibatch, a random subset of input contrasts is zeroed out to improve robustness when some of T1w, T2w, or FLAIR are missing. The implementation uses $T = 1000$ diffusion steps, a cosine noise schedule, Adam, an initial learning rate of $3 \times 10^{-4}$ , batch size 32, and training for 300 epochs (Zhang et al., 2 Oct 2025).

3. Bidirectional mask control and repainting dynamics

A distinctive feature of MSRepaint is its dual-mask inference scheme. It uses $M^{\text{target}}$ for the desired lesion configuration and $\phi_w$ 0 for the region to be actively updated. This is what makes the method bidirectional (Zhang et al., 2 Oct 2025).

Task	$\phi_w$ 1	Effect
Lesion filling	$\phi_w$ 2 inside the editing region	Replaces lesion voxels with healthy-appearing tissue
Lesion synthesis	$\phi_w$ 3	Generates lesion-like appearance in the specified healthy region

The repainting mechanism is inspired by RePaint. After the model predicts $\phi_w$ 4, this prediction is mixed with the true noisy sample so that only voxels inside the repaint mask are edited, while anatomy outside the region of interest is preserved (Zhang et al., 2 Oct 2025). The method then performs an additional forward-noise and reverse-denoise cycle: it takes the mixed sample, adds noise forward, and denoises again at the same timestep. This can be repeated multiple times; the reported experiments use two repaint iterations per timestep. The stated purpose is to reduce boundary artifacts, discontinuities, and unnatural transitions between lesion and surrounding tissue.

This masking scheme gives explicit voxel-level control over where lesions are removed or created. It also separates the semantic intent of the edit from the preservation constraint, which is important when filling and synthesis are treated within one conditional generative model.

4. DDIM acceleration and multi-view 3D consistency

To accelerate inference, MSRepaint uses DDIM reverse sampling with a reduced timestep subsequence $\phi_w$ 5 with $\phi_w$ 6 (Zhang et al., 2 Oct 2025). The paper notes that because DDIM is not Markovian in the same way as full diffusion sampling, lesion repainting regenerates $\phi_w$ 7 using an inpainted $\phi_w$ 8 rather than directly mixing at $\phi_w$ 9. For the $M$ 0 case, the inpainted clean estimate preserves voxels outside the repaint mask and edits voxels inside it.

A separate issue is inter-slice inconsistency in 3D MRI generation. MSRepaint addresses this through a multi-view DDIM inversion and fusion pipeline:

Stage	Operation	Output
Axial inversion	Full DDIM inversion on axial slices	$M$ 1
Coronal refinement	Reorient axial volume, add Gaussian noise, truncated DDIM inversion	$M$ 2
Sagittal refinement	Repeat the same process in the sagittal view	$M$ 3

The three view-specific outputs are then fused. For lesion synthesis, the pipeline uses a dedicated fusion network. For lesion filling, it uses the per-voxel median intensity. The reported inference settings are axial DDIM with every 10th timestep, yielding 100 reverse steps; coronal and sagittal truncated inversion starting at $M$ 4, yielding 4 reverse steps; and repainting repeated twice per timestep (Zhang et al., 2 Oct 2025).

The stated effect of this multi-view procedure is improved 3D spatial consistency, inter-slice anatomical coherence, and synthetic lesion plausibility. This suggests that MSRepaint’s 3D behavior is achieved not by direct 3D diffusion training, but by coordinated multi-planar inversion and fusion.

5. Large-scale lesion synthesis and lesion dictionaries

For large-scale synthesis, MSRepaint constructs lesion dictionaries from a large private MS dataset. The approach builds five lesion dictionaries, each aligned to a different healthy subject space. Lesion-filled T1w images are deformably registered to a healthy target subject, and lesion masks are warped into that subject’s space (Zhang et al., 2 Oct 2025). The reported source pool comprises 4,047 sessions from 1,132 people with multiple sclerosis.

Candidate masks are assembled compositionally. A mask is formed by randomly selecting 8 sessions, taking one-eighth of the lesion components from each, and aggregating them into a composite mask. These masks are then supplied, together with healthy multicontrast scans, to MSRepaint to produce synthetic multicontrast images with paired lesion labels (Zhang et al., 2 Oct 2025).

The paper states that this increases lesion diversity while preserving anatomical plausibility. In the context of segmentation training, this dictionary-based sampling is important because it decouples lesion placement from a single donor anatomy and provides a scalable route to generating labeled synthetic data.

6. Empirical evaluation, downstream analyses, and longitudinal simulation

The lesion-filling evaluation uses BrainWeb synthetic T1w lesions, private real lesions consisting of 15 PwMS T1w scans, and private synthetic lesions produced by inserting lesions into healthy scans. The main filling metric is RMSE inside lesion regions, normalized by surrounding normal-appearing white matter intensity. MSRepaint is compared with FSL-filling, NiftySeg-filling, and FastSurfer-LIT. The reported findings are that FSL has the highest RMSE and visible spike-like artifacts, NiftySeg is better than FSL but still produces artifacts, FastSurfer-LIT is smooth and accurate, and MSRepaint matches FastSurfer-LIT accuracy with artifact-free outputs. Approximate runtimes are reported as 2 min CPU for FSL, 5 min CPU for NiftySeg, 60 min GPU for FastSurfer-LIT, and 3 min GPU for MSRepaint (Zhang et al., 2 Oct 2025).

The paper also evaluates whether lesion filling improves downstream image analysis. For brain parcellation and regional volume measurement, and for deformable registration with VoxelMorph and EOIR, MSRepaint-filled images are reported to reduce lesion-induced registration artifacts and improve alignment of lesion-sensitive structures including ventricles, caudate, and putamen. Relative to FSL and NiftySeg, MSRepaint and FastSurfer-LIT yield more anatomically consistent deformed images and better regional Dice scores in key structures. For deformation fields, MSRepaint significantly reduces RMSE within lesion regions relative to FSL and NiftySeg and is reported to be comparable to or slightly better than FastSurfer-LIT (Zhang et al., 2 Oct 2025).

For lesion synthesis, the authors train UNISELF on four training sets: original ISBI training data, CarveMix on ISBI, MSRepaint on ISBI with CarveMix lesion masks, and MSRepaint with lesion dictionaries. Evaluation is performed on MICCAI 2016 and UMCL with Dice, Precision, Sensitivity, voxel-wise F1, and lesion-wise TPR, FPR, and F1. Models trained on MSRepaint-synthesized data consistently outperform models trained on original ISBI data and CarveMix-augmented ISBI data, with the strongest improvement on lesion-wise F1. The lesion-dictionary-based MSRepaint dataset performs best and most stably, while voxel-level scores remain comparable (Zhang et al., 2 Oct 2025).

A further use case is longitudinal lesion evolution simulation. Using mask control, the same model is used to simulate lesion appearance, disappearance, growth, and shrinkage over time on a healthy subject with synthetic longitudinal masks: a baseline scan without lesions, a follow-up with new synthesized lesions, and a second follow-up with lesion growth, shrinkage, appearance, and disappearance. The paper presents this as high-fidelity simulation of lesion evolution in MS progression (Zhang et al., 2 Oct 2025).

7. Scope, nomenclature, strengths, and limitations

MSRepaint should be distinguished from other similarly named methods. “RePaint-NeRF” is a two-stage NeRF editing method for repainting or replacing selected 3D content in neural scenes using semantic masks and diffusion guidance (Zhou et al., 2023). “RePainter” is a reinforcement-learning framework for e-commerce object removal via spatial-matting trajectory refinement and GRPO (Guo et al., 9 Oct 2025). By contrast, MSRepaint is a conditional DDIM-based lesion inpainting and synthesis model for MS MRI (Zhang et al., 2 Oct 2025).

Within its stated scope, the paper identifies several strengths: a unified framework for filling and synthesis, voxel-level mask control, multicontrast and missing-contrast robustness, anatomically faithful lesion editing, 3D consistency through multi-view inversion and fusion, fast inference relative to prior diffusion methods, and improved downstream tasks and segmentation generalization (Zhang et al., 2 Oct 2025). These properties follow directly from the combination of mask-guided editing, contrast dropout, repainting, DDIM acceleration, and multi-view fusion.

The limitations are also explicit. The model does not explicitly model lesion temporal dynamics; longitudinal simulation is controlled by masks rather than learned disease progression. Clinical validation remains limited, and applicability to other diseases or modalities is not yet explored (Zhang et al., 2 Oct 2025). Another common misconception is to treat MSRepaint as a generic “repainting” framework. The paper instead defines it narrowly as a method for bidirectional lesion manipulation in multicontrast MS MRI, with filling and synthesis both expressed as mask-conditioned diffusion-based inpainting.