Music Adversarial Inpainting Attack (MAIA)

• The paper introduces MAIA, a framework that identifies and inpaints decision-critical music segments using generative models to mislead MIR systems.
• MAIA replaces global noise perturbations with localized edits, utilizing Grad-CAM in white-box and CMA-ES in black-box settings for precise targeting.
• Empirical results on cover song identification and genre classification demonstrate higher attack success and lower perceptual distortion compared to additive methods.

Music Adversarial Inpainting Attack (MAIA) is an adversarial attack framework for Music Information Retrieval (MIR) systems that replaces diffuse additive perturbation with localized content reconstruction. It first identifies decision-critical regions of a music signal, then masks and inpaints those regions with a generative model while guiding the reconstruction so that the attacked MIR model changes its prediction. The framework is defined for both white-box and black-box settings and is evaluated on cover song identification and music genre classification, where it is reported to achieve high attack success rates while maintaining lower perceptual distortion than additive baselines (Liu et al., 5 Sep 2025).

1. Conceptual basis and attack formulation

MAIA is motivated by the observation that many MIR attacks are noise-based: they add small perturbations directly to the waveform or spectrogram, often over the entire signal. MAIA instead assumes that MIR decisions are often driven by specific local patterns, such as timbral events, harmonic fragments, or short motifs, rather than every sample equally. On that basis, it edits only a few highly influential segments, removes or masks them, and reconstructs them with a generative inpainting model so that the resulting audio remains contextually plausible and musically coherent (Liu et al., 5 Sep 2025).

The framework is defined for both white-box and black-box threat models. In the white-box case, the attacker knows the target model’s architecture and parameters and has gradient access. In the black-box case, the attacker can only query the model and observe outputs such as predicted labels, logits, or confidence-based feedback. The adversarial goal described in the method is untargeted: given original music $x$, original or true label $y$, attacked MIR model $M$, binary mask $\mathbf{m}$, and inpainting model $\mathcal{G}_\theta$, MAIA seeks an edited signal $x_{\mathrm{adv}}$ such that

$M(x_{\mathrm{adv}}) \neq y.$

MAIA does not formulate this objective as a classic $L_p$-bounded perturbation problem. Instead, it constrains the attack through localized masking of only a few important segments, reconstruction consistency, and evaluation-time perceptual metrics such as FAD and LSD. In that sense, the method shifts the adversarial variable from “small perturbation everywhere” to “plausible replacement in selected regions.” This is the defining distinction between MAIA and earlier global additive music attacks (Liu et al., 5 Sep 2025).

2. Localization of decision-critical regions

The first stage of MAIA is importance analysis. Its purpose is to identify a small set of regions that most strongly support the model’s original decision and to use those regions as candidate adversarial zones.

In the white-box variant, importance is estimated with Grad-CAM on an internal convolutional layer of the target model. Let $F_k^l$ denote the $k$-th feature map at layer $y$0, and let $y$1 denote the class score or logit for class $y$2. The Grad-CAM weights are

$y$3

and the class activation map is

$y$4

The paper states that, for the attacked MIR model $y$5, the analyzed layer is

$y$6

The resulting heatmap is mapped back to spectrogram coordinates and normalized, or thresholded by taking the top $y$7 most active bins, and those regions are selected for masking and later inpainting (Liu et al., 5 Sep 2025).

In the black-box variant, MAIA uses a query-based coarse-to-fine segment search. The audio is first partitioned into coarse non-overlapping chunks

$y$8

with an example chunk length of $y$9 second. For each chunk $M$0, the method constructs a modified signal $M$1 by zero-masking that chunk. To reduce boundary artifacts, the masked region is tapered with a Tukey window with shape parameter $M$2. Importance is then defined as the increase in loss per unit duration:

$M$3

The most important coarse segment $M$4 is subdivided,

$M$5

rescored in the same way, and the refined segmentation is updated iteratively until round $M$6. Finally, the top $M$7 segments by importance are selected:

$M$8

This yields a query-efficient localization stage that searches for where to edit before searching how to edit (Liu et al., 5 Sep 2025).

3. White-box and black-box inpainting procedures

Once the mask $M$9 has been determined, MAIA edits only those regions. The generative model used in the experiments is GACELA, a Generative Adversarial Context Encoder for Long Audio Inpainting. In the MAIA description, GACELA is characterized as a conditional GAN for long-gap music inpainting with a generator operating on log-magnitude mel spectrogram context, a convolutional encoder-decoder structure, latent variables to model multimodal completion, and five discriminators at multiple time/frequency scales (Liu et al., 5 Sep 2025).

In the white-box setting, the masked region is first inpainted normally:

$\mathbf{m}$0

At iteration $\mathbf{m}$1, the total loss is

$\mathbf{m}$2

with untargeted attack term

$\mathbf{m}$3

where $\mathbf{m}$4 can be cross-entropy. The masked region is then updated by sign-gradient descent:

$\mathbf{m}$5

After the gradient step, MAIA re-inpaints the masked region,

$\mathbf{m}$6

which the paper explicitly describes as projecting the perturbed content back towards a realistic data manifold. The white-box implementation uses a maximum iteration count of $\mathbf{m}$7 and a grid search over

$\mathbf{m}$8

(Liu et al., 5 Sep 2025)

In the black-box setting, no gradient from $\mathbf{m}$9 is available, so MAIA combines importance-guided localization with gradient-free latent search using CMA-ES. Initial inpainting for a selected segment is

$\mathcal{G}_\theta$0

The latent code is then updated by

$\mathcal{G}_\theta$1

where $\mathcal{G}_\theta$2 denotes feedback obtained from querying the target model on the current inpainted sample. Candidate latent codes $\mathcal{G}_\theta$3 are sampled from the current CMA-ES distribution; for each candidate,

$\mathcal{G}_\theta$4

Candidates are ranked by whether they increase adversarial effect or cause

$\mathcal{G}_\theta$5

The next sample is formed as

$\mathcal{G}_\theta$6

The attack processes high-importance segments sequentially and is capped at $\mathcal{G}_\theta$7 queries (Liu et al., 5 Sep 2025).

4. Tasks, metrics, and reported results

MAIA is evaluated on two MIR tasks. For Cover Song Identification (CSI), the target model is CoverHunter and the dataset is the SHS100K test set. For Music Genre Classification (MGC), the target model is IDS-NMR and the dataset is GTZAN. White-box baselines are PGD and C&W; black-box baselines are NES and ZOO. The reported metrics are Attack Success Rate (ASR), task degradation measured by post-attack mAP for CSI and post-attack accuracy for MGC, FAD computed using MERT-V0 features, LSD, and a MOS-like listening score from a human listening test on a 5-point scale (Liu et al., 5 Sep 2025).

In white-box CSI, PGD achieves ASR $\mathcal{G}_\theta$8, mAP $\mathcal{G}_\theta$9, FAD $x_{\mathrm{adv}}$0, LSD $x_{\mathrm{adv}}$1, and MOS $x_{\mathrm{adv}}$2; C&W achieves ASR $x_{\mathrm{adv}}$3, mAP $x_{\mathrm{adv}}$4, FAD $x_{\mathrm{adv}}$5, LSD $x_{\mathrm{adv}}$6, and MOS $x_{\mathrm{adv}}$7. MAIA-White Box reports ASR $x_{\mathrm{adv}}$8, mAP $x_{\mathrm{adv}}$9, FAD $M(x_{\mathrm{adv}}) \neq y.$0, LSD $M(x_{\mathrm{adv}}) \neq y.$1, and MOS $M(x_{\mathrm{adv}}) \neq y.$2. In white-box MGC, PGD achieves ASR $M(x_{\mathrm{adv}}) \neq y.$3, accuracy $M(x_{\mathrm{adv}}) \neq y.$4, FAD $M(x_{\mathrm{adv}}) \neq y.$5, LSD $M(x_{\mathrm{adv}}) \neq y.$6, and MOS $M(x_{\mathrm{adv}}) \neq y.$7; C&W achieves ASR $M(x_{\mathrm{adv}}) \neq y.$8, accuracy $M(x_{\mathrm{adv}}) \neq y.$9, FAD $L_p$0, LSD $L_p$1, and MOS $L_p$2. MAIA-White Box reports ASR $L_p$3, accuracy $L_p$4, FAD $L_p$5, LSD $L_p$6, and MOS $L_p$7 (Liu et al., 5 Sep 2025).

In black-box CSI, NES achieves ASR $L_p$8, mAP $L_p$9, FAD $F_k^l$0, LSD $F_k^l$1, and MOS $F_k^l$2; ZOO achieves ASR $F_k^l$3, mAP $F_k^l$4, FAD $F_k^l$5, LSD $F_k^l$6, and MOS $F_k^l$7. MAIA-Black Box reports ASR $F_k^l$8, mAP $F_k^l$9, FAD $k$0, LSD $k$1, and MOS $k$2. In black-box MGC, NES achieves ASR $k$3, accuracy $k$4, FAD $k$5, LSD $k$6, and MOS $k$7; ZOO achieves ASR $k$8, accuracy $k$9, FAD $y$00, LSD $y$01, and MOS $y$02. MAIA-Black Box reports ASR $y$03, accuracy $y$04, FAD $y$05, LSD $y$06, and MOS $y$07 (Liu et al., 5 Sep 2025).

The listening test involves $y$08 participants who rate perturbation noticeability on a 5-point scale, where $y$09 means highly noticeable and $y$10 means no perceivable difference. Across all reported settings, MAIA receives higher listener scores than the additive baselines. The paper presents this as evidence that localized inpainting produces more natural edits than direct adversarial noise injection (Liu et al., 5 Sep 2025).

One textual inconsistency is explicitly noted in the reported results. The narrative paragraph states “93.5% for CSI and 94.5% for MGC” for white-box MAIA, whereas the quantitative table reports $y$11 for CSI and $y$12 for MGC. The table is identified as the more reliable source (Liu et al., 5 Sep 2025).

5. Relation to prior adversarial music and audio inpainting research

MAIA sits at the intersection of two earlier research lines: adversarial attacks on MIR systems and generative audio inpainting. Earlier music-adversarial work typically optimized global additive perturbations rather than localized replacement. One early example attacked magnitude-spectral deep genre classifiers with projected gradient descent and a Griffin–Lim-style consistency projection to obtain valid audio (Kereliuk et al., 2015). Later work on music instrument classification moved the optimization to the raw waveform and introduced end-to-end white-box attacks through differentiable torchaudio preprocessing, including a Multi-Scale C&W objective with

$y$13

over FFT sizes $y$14 (Prinz et al., 2020). Copyright-detection attacks likewise relied on global additive perturbations and demonstrated transfer from a differentiable Shazam-like surrogate to AudioTag and YouTube Content ID (Saadatpanah et al., 2019). A perception-aware attack on YouTube copyright detection replaced norm minimization with a learned perceptual proxy, qDev, trained from human ratings over pitch, rhythm, timbre, and loudness deviations (Duan et al., 2022).

The inpainting side of the lineage is different. GACELA addressed long-gap audio inpainting with conditioning on context before and after the gap, a $y$15-dimensional latent variable, and five parallel discriminators with increasing receptive fields; it was evaluated on gap durations from $y$16 ms to $y$17 ms and found that inpaintings were often detectable but usually between “not disturbing” and “mildly disturbing” in severity (Marafioti et al., 2020). Other generative work studied raw-waveform WGAN inpainting for approximately $y$18–$y$19 ms gaps with short-range and long-range border conditioning (Ebner et al., 2020), while AIRGen recast MusicGen as a token-space inpainting editor using EnCodec tokens, a prefix/prediction sequence design, heterogeneous adapters, and mask ratios sampled uniformly from $y$20 (Lin et al., 2024).

Against that background, MAIA’s novelty is not merely the use of a generator. Its distinguishing move is to combine decision-focused localization with generative replacement inside selected segments, then couple that replacement to the attacked model’s outputs in either gradient-guided or query-guided form. Earlier adversarial music work had already shown that MIR systems were vulnerable to imperceptible or high-SNR perturbations, but those systems were predominantly in the global additive regime; earlier inpainting work had shown that context-aware replacement was possible, but without adversarial objectives. MAIA combines the two strands into an attack in which the manipulated content is sparse, local, and generator-produced rather than globally noise-like (Liu et al., 5 Sep 2025).

6. Limitations, perceptual evaluation, and implications

Several limitations are explicit. MAIA depends on a pretrained inpainting model such as GACELA; if the generator produces poor completions, the attack may create audible artifacts or fail. The method is also computationally heavier than simple PGD or C&W because it must perform importance analysis, iterative re-inpainting in white-box mode, and CMA-ES-based latent search in black-box mode. The black-box variant is capped at $y$21 queries, which is more structured than naive zeroth-order search but still substantial (Liu et al., 5 Sep 2025).

Reproducibility is incomplete in several places. The paper leaves underspecified the number of selected segments $y$22, exact refinement depth $y$23, subdivision factor $y$24, the precise form of $y$25, and the exact black-box feedback function $y$26. The evaluation scope is also limited to CSI and genre classification; tasks such as tagging, transcription, recommendation, or source-oriented MIR are not tested (Liu et al., 5 Sep 2025).

Perceptual evaluation raises a separate issue. MAIA reports FAD and LSD, together with listener studies, as its principal perceptual indicators. However, later work on adversarial music perception argues that common metrics such as $y$27, SNR, LSD, PEMO-Q, and FAD correlate poorly with human judgments in music adversarial settings, and introduces PAMT, a Perceptually-Aligned MERT Transformer, which achieves a Spearman correlation coefficient of $y$28 with subjective scores (Liu et al., 5 Sep 2025). This suggests that FAD and LSD, while operationally useful, may not fully characterize imperceptibility for MAIA-style local replacement attacks.

The broader implication is that MIR systems may be vulnerable not only to low-amplitude additive perturbation but also to semantically coherent local content replacement. The defense consequence drawn in the MAIA paper is correspondingly broad: robustness methods aimed only at small additive noise are likely incomplete if they ignore localized content replacement, semantically coherent adversarial edits, and black-box query-guided manipulation. A plausible implication is that future defenses will need both locality-aware threat models and more human-aligned perceptual criteria than conventional signal-space distortion alone (Liu et al., 5 Sep 2025).