Enable segment-level editing and continuation in DiffRhythm

Investigate and implement segment-level editing capabilities in DiffRhythm, including song inpainting and outpainting, by developing training and inference mechanisms such as random masking of variational autoencoder latent representations to enable editing of specific regions within generated compositions.

Background

DiffRhythm is introduced as an end-to-end latent diffusion model for generating full-length songs with vocals and accompaniment. While the system demonstrates strong performance and fast inference, the authors explicitly note that editing functionality—modifying specific parts of a generated song and continuing a composition—has not yet been explored within the framework.

They suggest that random masking of latent representations during training could be a promising approach to enable inpainting and outpainting, indicating a concrete direction but leaving the problem unresolved within the current work.