AudEdit: Source-Faithful Text-Guided Audio Editing

• AudEdit is a text-guided audio editing framework that modifies recordings while preserving source-specific timing, timbre, and structure.
• The latent-diffusion approach employs embedding optimization, UNet fine-tuning, and prompt interpolation to ensure accurate source preservation during edits.
• The inversion-free variant uses a rectified-flow generator and velocity-difference ODE for training-free, controlled edits, though it struggles with extreme replacements.

AudEdit denotes a line of text-guided audio editing work centered on modifying an existing recording while preserving edit-irrelevant structure. In the literature, the name has been used for two distinct methods: a latent-diffusion approach for non-rigid text edits that adapts the “Imagic” idea from image editing to audio, and a later inversion-free method for text-guided editing of real audio with a pretrained rectified-flow audio generator (Paissan et al., 2023, Fu, 13 Jun 2026). In both cases, the core problem differs from text-to-audio generation: the system must alter specified content without discarding timing, timbre, or other source-specific information. That requirement places AudEdit within the broader research area of instruction-based audio editing, where recent benchmarks argue that evaluation has lagged behind model development and that exact instruction-faithful editing remains unreliable (Ma et al., 5 Jun 2026).

1. Terminological scope and historical placement

The 2023 paper "Audio Editing with Non-Rigid Text Prompts" presents AudEdit as a latent-diffusion-based audio editing method whose goal is to perform non-rigid text-prompted edits while staying faithful to the source audio, especially preserving the timing structure of acoustic events such as onsets and offsets (Paissan et al., 2023). Its motivation is explicitly comparative: AUDIT supports only a closed set of commands like Add, Drop, Replace; SDEdit allows free-form prompting but may lose fidelity to the original input if too much noise is injected; and AudioLDM is described as less faithful to the source audio, particularly in preserving temporal structure (Paissan et al., 2023).

A separate 2026 paper, "AUDEDIT: Inversion-Free Text-Guided Editing with Pretrained Audio Flow Models," reuses the name for a method built on Stable Audio 3 and the SAME latent autoencoder (Fu, 13 Jun 2026). Here the central claim is different: instead of adapting per-sample prompt optimization and fine-tuning, the method edits directly from source to target in latent space through a velocity-difference ODE, avoiding the standard inversion-style route through noise (Fu, 13 Jun 2026).

This naming overlap is important because AudEdit is distinct from AUDIT, the 2023 supervised model "Audio Editing by Following Instructions with Latent Diffusion Models." AUDIT is trained explicitly on triplets of $$(\text{instruction}, \text{input audio}, \text{output audio})$$, uses about 0.6M triplets, and supports adding, dropping, replacement, inpainting, and super-resolution in one model (Wang et al., 2023). A plausible implication is that "AudEdit" should be read less as a single stable architecture than as a recurring label attached to source-faithful, text-guided audio editing methods.

2. The 2023 latent-diffusion AudEdit

The 2023 AudEdit pipeline has three stages. First, it performs embedding optimization: given input audio $X_{\text{inp}}$, the audio is encoded into a latent representation, and the text embedding is optimized so that the diffusion model can reconstruct the original latent from noise-conditioned denoising. Second, the UNet parameters are fine-tuned using the same reconstruction loss. Third, editing is produced by interpolating between the optimized source-specific embedding and the target prompt embedding,

$$\mathbf e_{\text{int}} := \eta \mathbf e_{\text{target}} + (1-\eta)\mathbf e_{\text{opt}},$$

with $\eta \in [0,1]$, where $\eta=0$ reconstructs the original audio and larger $\eta$ increases edit strength (Paissan et al., 2023).

The paper attributes source faithfulness to the fact that the model is first trained to reconstruct the source audio latent as closely as possible, then mixes in the target prompt gradually through interpolation. Because conditioning is anchored to the source through $\mathbf e_{\text{opt}}$, the edit tends to retain the original event timing, onset positions, offset positions, and general temporal alignment (Paissan et al., 2023). The system is built on a latent diffusion model with a VAE encoder/decoder for mel-spectrogram latents, a UNet denoiser, a text encoder from the underlying text-to-audio model, and a pretrained HiFi-GAN vocoder. The base generative model is TANGO, which leverages FLAN-T5 as the text encoder (Paissan et al., 2023).

The paper evaluates three edit types: addition, style transfer, and inpainting. In addition, the text prompt should describe the full context, not just the new event. In style transfer, the model should preserve content, onset/offset timing, and event occurrence positions while changing acoustic character. In inpainting, only the missing region is generated and the observed part of the input is not replaced (Paissan et al., 2023).

Quantitatively, the paper reports evaluation on 27 audio samples, each about 10 seconds long. Using the sum of Text CLAP and Audio CLAP as a tradeoff metric, AudEdit reports 1.366 for addition, 1.472 for inpainting, and 1.141 for style transfer; the LoRA variant reports 1.407, 1.499, and 1.161, respectively (Paissan et al., 2023). The paper also states that editing time for a 10-second clip drops from about 17 minutes to 9 minutes on an RTX 3090 with LoRA, with little or no loss in edit quality (Paissan et al., 2023). In a user study with 17 machine-learning researchers using webMUSHRA, AudEdit was preferred by a large margin over AudioLDM and SDEdit on addition and inpainting, while style transfer was roughly comparable to those baselines (Paissan et al., 2023).

The method’s limitations are explicit. It requires per-sample optimization, making it slower than a feed-forward editor; the fine-tuning stage is computationally expensive; tradeoff tuning via $\eta$ is delicate; CLAP is not ideal for evaluating timing; and, although the method is free-form in principle, the paper benchmarks only addition, style transfer, and inpainting (Paissan et al., 2023).

3. The 2026 inversion-free flow-based AudEdit

The 2026 AUDEDIT paper shifts the design point from per-sample optimization to training-free, inversion-free, text-guided audio editing on a pretrained rectified-flow audio generator, specifically Stable Audio 3 with the SAME codec (Fu, 13 Jun 2026). The method is motivated by the claim that inversion-based or SDEdit-style noising and denoising forces a difficult trade-off: too little noise preserves the source but weakens prompt adherence; too much noise improves prompt following but destroys rhythm, transients, timbre, phase coherence, and long-range musical structure (Fu, 13 Jun 2026).

Its core latent dynamics are written as

$d z_t = V_\theta(z_t,t,c)\,dt,$

with classifier-free guidance

$$V_\theta^{w}(z,t,c) = V_\theta(z,t,\varnothing) + w\left[ V_\theta(z,t,c)-V_\theta(z,t,\varnothing) \right].$$

Rather than sending the source through a source-to-noise-to-target detour, the method constructs a direct source-to-target ODE in Stable Audio 3 latent space. The key identity is

$X_{\text{inp}}$0

which yields a velocity-difference ODE with

$X_{\text{inp}}$1

In practice, the editor samples a shared stochastic source marginal at each timestep and updates the edited latent through the difference between source- and target-conditioned velocity fields under the same sampled noise (Fu, 13 Jun 2026).

The update rule reported in the paper is

$X_{\text{inp}}$2

where $X_{\text{inp}}$3 is averaged over $X_{\text{inp}}$4 stochastic samples (Fu, 13 Jun 2026). The method takes as input the source latent $X_{\text{inp}}$5, a source prompt $X_{\text{inp}}$6, a target prompt $X_{\text{inp}}$7, a timestep schedule, the number of stochastic samples $X_{\text{inp}}$8, source and target CFG scales, and a step coefficient $X_{\text{inp}}$9, and decodes the edited latent with SAME. The paper stresses that the method requires no training, no paired edit data, no optimization, and no access to internal attention maps (Fu, 13 Jun 2026).

Experiments are reported on two real-audio benchmark sets: 227 sound-effect edits from FSD50K and 209 music edits from the Song Describer Dataset. The backbone is Stable Audio 3 medium, the codec is SAME, the sample rate is 44.1 kHz, default solver steps are 28, the source guidance scale is 1.5, the target guidance scale is 3.5, and $$\mathbf e_{\text{int}} := \eta \mathbf e_{\text{target}} + (1-\eta)\mathbf e_{\text{opt}},$$0 (Fu, 13 Jun 2026). On sound effects, AudEdit reports CLAP-T 0.52, CLAP-A 0.59, LSD 20.06, MCD 551.08, LPAPS 0.22, Structure 0.57, and FAD 50.37; the abstract highlights an improvement in target-text CLAP similarity from 0.42 to 0.52 and a reduction in FAD from 65.70 to 50.37 over the strongest baseline (Fu, 13 Jun 2026). On music, it reports CLAP-T 0.59, CLAP-A 0.72, LSD 18.90, MCD 474.84, LPAPS 0.19, Structure 0.91, and FAD 42.81 (Fu, 13 Jun 2026).

The paper is explicit that the method is best for controlled source-preserving edits, not for full regeneration. It struggles with acoustically incompatible replacements, multiple simultaneous role changes, vocal-to-instrumental transformations, solo-to-ensemble changes, extreme genre shifts, and broad semantic rewrites. It also does not provide explicit temporal masks, stem-level controls, beat-synchronous constraints, or pitch-contour preservation (Fu, 13 Jun 2026).

4. AudEdit in relation to other audio editing paradigms

AudEdit occupies one position within a broader methodological spectrum. AUDIT represents the supervised, task-aligned end of that spectrum. It constructs large triplet training data $$\mathbf e_{\text{int}} := \eta \mathbf e_{\text{target}} + (1-\eta)\mathbf e_{\text{opt}},$$1, learns the conditional distribution $$\mathbf e_{\text{int}} := \eta \mathbf e_{\text{target}} + (1-\eta)\mathbf e_{\text{opt}},$$2, and conditions the diffusion U-Net on the clean input latent to preserve unedited content. The paper reports state-of-the-art results across adding, dropping, replacement, inpainting, and super-resolution, including LSD 1.35, KL 0.92, FD 21.80 on adding and LSD 1.32, KL 0.75, FD 18.17 on inpainting (Wang et al., 2023).

At the training-free inversion end, AudioEditor adapts image-editing techniques such as DDIM inversion, Null-text Inversion, and EOT-suppression to audio using Auffusion as the backbone (Jia et al., 2024). The method is explicitly designed to preserve original audio features while executing accurate edits. Its ablation study reports that removing Null-text Optimization hurts preservation metrics significantly, while removing EOT-suppression lowers CLAP, indicating worse edit accuracy (Jia et al., 2024).

PPAE formulates precise editing as cross-attention control in latent diffusion. It supports Audio Replace, Audio Refine, Audio Reweight, and an Audio Refusion setting, using a Fuser that interpolates between source and edited attention maps over diffusion time and a global bootstrapping strategy over candidate guidance scales (Xu et al., 2024). On Replace, it reports FAD 2.15, LSD 1.51, FD 27.53, KL 1.30, and CLAP 0.62, and the paper attributes its behavior to smooth attention fusion rather than abrupt attention replacement (Xu et al., 2024).

A later diffusion-based inversion-free line is represented by DirectAudioEdit, which constructs a source-to-target editing path through shared-noise re-noising and diffusion prediction contrast. The paper reports that it reduces macro-averaged FAD and KL by 15.9% and 15.8% compared with DDPM inversion, while achieving up to 64.5% editing speedup (Ge et al., 5 Jun 2026). Relative to this landscape, the 2026 AUDEDIT paper can be read as the rectified-flow analogue of the same inversion-free impulse, but using a velocity-difference ODE rather than diffusion reverse-dynamics contrast. This suggests that the current field is organized less by prompt format than by how source preservation is enforced: supervised paired training, inversion and attention control, or direct source-to-target transport.

5. Evaluation, metrics, and the diagnostic role of AudEdit

Early AudEdit evaluation relied heavily on semantic-fidelity tradeoff metrics such as Text CLAP and Audio CLAP. The 2023 paper explicitly notes that Audio CLAP is not ideal for evaluating timing, because it measures semantic similarity rather than exact temporal structure (Paissan et al., 2023). More recent work generalizes that criticism. The 2025 AuditScore paper argues that common objective metrics such as CLAP, FAD, FD, and KL only partially capture editing quality, because editing must simultaneously satisfy Quality, Relevance, and Faithfulness (Jia et al., 16 Aug 2025). AuditScore introduces 6,360 original/edited audio pairs across 23 system configurations and 7 representative editing methods, each sample rated by five domain experts on those three dimensions; AuditEval is then trained as a MOS-style evaluator with three independently trained decoding heads (Jia et al., 16 Aug 2025).

The strongest benchmark-level critique comes from MMAE, which presents itself as the first comprehensive benchmark for general-purpose, instruction-based audio editing (Ma et al., 5 Jun 2026). MMAE spans 7 audio modalities, 6 levels of task complexity, 2 levels of granularity, and 8 operation types; it contains 2,000 samples and 17,741 rubrics produced through human-agent collaboration (Ma et al., 5 Jun 2026). Its rubric-based framework decomposes open-ended tasks into atomic verifiable checks and evaluates both Instruction Following (IF) and Consistency, with EMR defined as the proportion of samples where all rubrics are answered correctly (Ma et al., 5 Jun 2026). The paper’s main finding is that EMR stays below 5% for all models and drops to 0% in some complex mixed-modality settings, particularly Sound-Music-Speech (Ma et al., 5 Jun 2026). That result directly complicates any reading of AudEdit based solely on average CLAP, FAD, or related metrics: a model may appear competitive on scalar metrics and still almost never produce a completely correct edit.

A complementary evaluation line uses multimodal LLMs for interpretable judgment. "Interpretable Audio Editing Evaluation via Chain-of-Thought Difference-Commonality Reasoning with Multimodal LLMs" proposes the first natural-language-based automated evaluation framework for audio editing, built on Qwen2-Audio, and trains on 30,000 edited audio samples for Difference Captioning and Commonality Captioning (Jia et al., 21 Sep 2025). The reported Edit_score reaches LCC 0.7652, SRCC 0.7312, and KTAU 0.8460 on editing effectiveness, while Faith_score is weaker on preservation, which the paper attributes to the difficulty of capturing prosody, volume, and noise characteristics through semantics alone (Jia et al., 21 Sep 2025). In the context of AudEdit, these developments mark a shift from measuring plausibility to measuring exact compliance and preservation.

6. Limitations, misconceptions, and current research directions

A recurring misconception is that source-faithful audio editing is largely solved once a system yields plausible audio with good text alignment. The benchmark evidence does not support that view. MMAE reports that current systems are still far from reliable edits, with Exact Match Rate consistently below 5%, and identifies sharp degradation under increased complexity and mixed modalities (Ma et al., 5 Jun 2026). This suggests that AudEdit-style success on curated tasks such as addition, style transfer, or controlled source-preserving edits should not be conflated with robust general-purpose editing.

The limitations reported by AudEdit papers themselves are narrower but consistent. The 2023 latent-diffusion version requires per-sample optimization, is computationally expensive, and depends on a delicate fidelity–editability balance through $$\mathbf e_{\text{int}} := \eta \mathbf e_{\text{target}} + (1-\eta)\mathbf e_{\text{opt}},$$3; it also acknowledges that its evaluation is imperfect for timing and that only addition, style transfer, and inpainting are benchmarked (Paissan et al., 2023). The 2026 inversion-free version avoids inversion, paired data, and optimization, but remains strongest on controlled source-preserving edits and explicitly struggles on acoustically incompatible replacements, extreme genre shifts, and broad semantic rewrites, while lacking explicit temporal masks, stem-level controls, beat-synchronous constraints, and pitch-contour preservation (Fu, 13 Jun 2026).

The broader literature identifies several concrete directions. MMAE argues for improve atomic editing fidelity, support universal modality editing, strengthen structural robustness, integrate planning with stronger base editors, and adopt standardized rubric-based evaluation (Ma et al., 5 Jun 2026). Directly related method papers point toward different mechanisms for reaching those goals: supervised paired training in AUDIT, Null-text Inversion and EOT-suppression in AudioEditor, cross-attention fusion in PPAE, and inversion-free transport in AUDEDIT and DirectAudioEdit (Wang et al., 2023, Jia et al., 2024, Xu et al., 2024, Fu, 13 Jun 2026, Ge et al., 5 Jun 2026). A plausible implication is that future work on AudEdit will be judged less by whether it can generate plausible edits and more by whether it can execute small requested changes precisely while preserving unrelated content under a benchmark that measures both properties explicitly.