VoicePrivacy Attacker Challenge (VPAC)

• VoicePrivacy Attacker Challenge (VPAC) is an attacker-centric benchmark that tests systems recovering speaker identity from anonymized speech.
• It evaluates diverse attacker strategies using LibriSpeech data and an equal error rate (EER) metric to quantify privacy leakage.
• The challenge underscores that credible voice privacy claims require assessment against informed, adaptive attackers exploiting both acoustic and linguistic cues.

Searching arXiv for recent VPAC papers to ground the article. Searching arXiv for "VoicePrivacy Attacker Challenge" and related attacker-side evaluation work. The VoicePrivacy Attacker Challenge (VPAC) is an attacker-centric benchmark in which participants develop automatic speaker verification systems that attempt to recover speaker identity from anonymized speech. Introduced as the first challenge whose goal is not to design better anonymization, but to build the strongest possible attacker systems against anonymized speech, VPAC serves as the attacker-side counterpart to the VoicePrivacy 2024 Challenge and was organized as an ICASSP 2025 SP Grand Challenge (Tomashenko et al., 19 Apr 2025). Its central premise is that privacy in speech should be assessed against the best attacker available, because privacy can be overestimated when evaluation relies on a weak or fixed attacker (Tomashenko et al., 19 Apr 2025).

1. Historical and conceptual setting

VPAC emerged from the broader VoicePrivacy initiative, which was created in 2020 to promote privacy-preserving speech technology through common datasets, protocols, and metrics (Tomashenko et al., 2024). Earlier VoicePrivacy challenges concentrated on the design of anonymization systems that output speech waveforms, hide speaker identity, and preserve linguistic and paralinguistic content. By contrast, VPAC reverses the perspective: it evaluates attacker systems rather than anonymizers, and asks how much speaker information still leaks when the ASV side is optimized aggressively (Tomashenko et al., 2024).

This attacker-centric formulation has antecedents in earlier analyses of informed attackers. Work on voice conversion-based privacy protection had already shown that privacy can collapse when the attacker knows the transformation procedure and parameterization, and that evaluations under weak attacker assumptions can substantially overestimate protection (Srivastava et al., 2019). The VoicePrivacy 2022 evaluation plan reinforced this direction by introducing a stronger semi-informed attacker model retrained on utterance-level anonymized speech, using an x-vector + PLDA ASV system (Tomashenko et al., 2022). VPAC can therefore be understood as the institutionalization of a principle that had become increasingly clear across VoicePrivacy research: privacy claims are only credible under strong, adaptive, and informed attack models.

A common misconception is that VPAC is a challenge about anonymization design. In fact, its stated purpose is the opposite. VoicePrivacy 2024 primarily concerned the construction of anonymization systems, whereas VPAC explicitly focuses on breaking those systems. Together, the two benchmarks form a closed evaluation loop in which anonymization methods are developed on one side and challenged by stronger attack models on the other (Tomashenko et al., 19 Apr 2025).

2. Task definition and attacker model

The VPAC task is formulated as speaker verification under anonymization. Participants develop one or more attacker systems against one or more target anonymization systems, and each attacker is evaluated separately for each target system (Tomashenko et al., 2024). Operationally, the attacker receives a trial utterance and an enrollment speaker and must output an ASV score such that same-speaker pairs score higher and different-speaker pairs score lower. In the challenge framing, this corresponds to re-identifying the original speaker from anonymized trial utterances given enrollment utterances; in the attacker papers, it is also described as deciding whether anonymized speech instances originate from the same speaker (Tomashenko et al., 19 Apr 2025).

The challenge assumes a semi-informed attacker model. Participants have access to anonymized trial utterances, original and anonymized enrollment utterances, original and anonymized training data, other allowed public resources for ASV training, a description of the anonymization system, and the system code when available (Tomashenko et al., 2024). This attacker is not blind to the protection mechanism and may adapt its ASV strategy accordingly. The semi-informed assumption is a deliberate departure from weaker settings in which the attacker is evaluated only on original-trained models or is assumed not to know that anonymization has been applied.

This design reflects a privacy game already articulated in the VoicePrivacy literature: users anonymize speech before sharing it, and attackers attempt to infer identity from the released data. VPAC operationalizes the attacker side of that game by making the level of privacy depend on the lowest error rate achieved among competing attackers (Tomashenko et al., 19 Apr 2025). A plausible implication is that VPAC shifts privacy evaluation from a static benchmark of anonymizers to an adversarial search over practical attack strength.

3. Data resources, target systems, and baseline attacker

VPAC uses LibriSpeech, matching the VoicePrivacy 2024 setup. The attacker training set is LibriSpeech train-clean-360 with 921 speakers, comprising 439 female and 482 male speakers, and 104,014 utterances (Tomashenko et al., 19 Apr 2025). Development uses LibriSpeech dev-clean with 29 enrollment speakers and 343 enrollment utterances, together with 40 trial speakers and 1,978 trial utterances. Evaluation uses LibriSpeech test-clean with 29 enrollment speakers and 438 enrollment utterances, together with 40 trial speakers and 1,496 trial utterances. The number of verification trials is 1,348 same-speaker and 27,362 different-speaker for development, and 997 same-speaker and 20,653 different-speaker for evaluation (Tomashenko et al., 19 Apr 2025).

The challenge targets seven anonymization systems selected from VoicePrivacy 2024:

Category	System	Description
Baseline	B3	phonetic transcription, pitch and energy modification, and artificial pseudo-speaker embedding generation
Baseline	B4	neural audio codec language modeling
Baseline	B5	VQ-bottleneck features extracted from an ASR model and original pitch
Participant	T8-5	random choice per utterance between Whisper+VITS ASR-TTS and kNN voice conversion on WavLM features
Participant	T10-2	neural audio codec with disentanglement of linguistic content, speaker identity, and emotional state
Participant	T12-5	B5 plus pitch smoothing
Participant	T25-1	disentangles content and style using VQ-BN, GST features, and emotion transfer

The code for B3, B4, and B5 was available and could be used to generate extra or modified training data for attackers (Tomashenko et al., 19 Apr 2025).

The official baseline attacker is the same system used in VoicePrivacy 2024: an ECAPA-TDNN ASV model with 512-channel convolution frame layers, implemented by adapting the SpeechBrain VoxCeleb recipe to LibriSpeech and trained on anonymized training data (Tomashenko et al., 2024). For each enrollment speaker, the baseline averages the embeddings of all anonymized enrollment utterances and compares this centroid with the embedding of the anonymized trial utterance using cosine similarity. The baseline is described as reasonable but not necessarily the strongest attacker, because it does not exploit content cues, pseudo-speaker selection strategies, stronger ASV architectures, or other advanced attack ideas (Tomashenko et al., 2024).

4. Evaluation protocol, metric, and challenge organization

The sole official metric is equal error rate (EER), used throughout the VoicePrivacy Challenge series (Tomashenko et al., 19 Apr 2025). In the evaluation plan, if $P_\text{fa}(\theta)$ and $P_\text{miss}(\theta)$ denote the false alarm and miss rates at threshold $\theta$, then

$$\text{EER}=P_\text{fa}(\theta_\text{EER})=P_\text{miss}(\theta_\text{EER}).$$

Lower EER means a stronger attacker, because the attacker more reliably distinguishes same-speaker from different-speaker trials (Tomashenko et al., 2024).

Attackers are ranked separately for each anonymization system (Tomashenko et al., 19 Apr 2025). Development and evaluation trial lists are gender-segregated into libri_dev_trials_f, libri_dev_trials_m, libri_test_trials_f, and libri_test_trials_m, and participants submit scores for these four lists (Tomashenko et al., 2024). Each submission includes the EER, the corresponding ASV scores for development and evaluation data, and a single detailed system description.

The first edition attracted 41 registered teams from academia and industry in 11 countries. Of these, 11 teams successfully submitted results, producing 55 submissions across the 7 anonymization systems (Tomashenko et al., 19 Apr 2025). Results were presented at an ICASSP 2025 special session, and five selected top-ranked participants were invited to submit and present their challenge systems (Tomashenko et al., 2024). This scale matters because privacy evaluation in VPAC depends not only on a single strong system but also on the diversity of independently designed attacks.

5. Representative attacker systems and challenge outcomes

The main empirical result of the first VPAC is that many attackers substantially outperformed the baseline attacker. The best systems reduced EER by 7–18 percentage points absolute and by 25–44% relative with respect to the baseline (Tomashenko et al., 19 Apr 2025). Team A.5 achieved the best attacker for T8-5, while Team A.20 achieved the best attacker for every other anonymization system. A.20 adapted a pretrained ResNet34 ASV model from WeSpeaker using LoRA on the provided anonymized data. A.5 proposed ECAPA-PLDA-Mix, combining an ECAPA-TDNN feature extractor trained on mixed datasets, a PLDA scoring module trained on anonymized data, and SpecAugment (Tomashenko et al., 19 Apr 2025).

A concrete instance of this attacker design space is the DA-SID system, proposed in “Attacking Voice Anonymization Systems with Augmented Feature and Speaker Identity Difference” (Zhang et al., 2024). DA-SID combines data augmentation with a PLDA back end in order to address two effects emphasized in the VPAC setting: anonymized speech shifts away from the original feature distribution, and anonymization tends to reduce inter-speaker distinctiveness. Its augmentation stage includes data fusion,

$\mathcal{D}_{\text{fused} = \mathcal{D}_{\text{orig} \cup \mathcal{D}_{\text{anon},$

and SpecAugment, with embeddings extracted as

$$\mathbf{e} = \mathcal{F}(\mathbf{X} \odot \mathbf{M}_{tf}),$$

where $\mathcal{F}(\cdot)$ is ECAPA-TDNN trained with additive angular margin loss. Its scoring stage uses PLDA,

$$s(\mathbf{e}_i, \mathbf{e}_j) = \log p(\mathbf{e}_i, \mathbf{e}_j \mid H_0) - \log p(\mathbf{e}_i, \mathbf{e}_j \mid H_1),$$

with $H_0$ denoting the same-speaker hypothesis and $H_1$ the different-speaker hypothesis (Zhang et al., 2024).

On LibriSpeech, DA-SID outperformed the official baseline across B3, B4, B5, T8-5, T12-5, and T25-1, with total average EER improving from 26.28 to 24.01 for B3, 31.49 to 23.38 for B4, 34.36 to 28.82 for B5, 40.76 to 26.05 for T8-5, 43.23 to 28.96 for T12-5, and 41.75 to 33.07 for T25-1 (Zhang et al., 2024). The paper highlights T8-5, where DA-SID reduced EER by 14.71% and was reported as the most effective attacker system for that anonymization system among all challenge systems. An ablation study showed that removing either the augmentation component or the PLDA-based speaker identity difference component degraded performance, and that SID contributed more strongly than DA. Because details about T10-2 were missing, DA-SID used a pretrained TitaNet-Large embedding model with cosine similarity for that case and still beat the official baseline (Zhang et al., 2024).

Later work extended attacker design beyond purely acoustic verification. VoxATtack introduced a dual-branch multimodal attacker combining ECAPA-TDNN on anonymized speech with a pretrained BERT over transcriptions, using confidence-weighted fusion (Aloradi et al., 16 Jul 2025). On the VPAC corpus it outperformed the top-ranking attackers on five out of seven benchmarks, and after anonymized-speech augmentation and SpecAugment it achieved state of the art on all VPAC benchmarks, including 20.6% average EER on T10-2 and 27.2% average EER on T25-1 (Aloradi et al., 16 Jul 2025). This suggests that the attacker design space in VPAC extends beyond stronger acoustic embeddings and includes multimodal exploitation of information that anonymization intentionally preserves.

6. Interpretation, limitations, and later developments

The significance of VPAC is twofold. First, it showed that the privacy protection of the best VoicePrivacy 2024 anonymization systems had been overestimated when assessed only with the baseline attacker (Tomashenko et al., 19 Apr 2025). Second, it did not imply that anonymization is useless: the challenge paper emphasizes that the best anonymized systems still had EER $P_\text{miss}(\theta)$0 against the best attackers, so they still offered moderate protection (Tomashenko et al., 19 Apr 2025). The challenge therefore sharpened, rather than eliminated, the privacy–leakage trade-off.

One recurring controversy concerns what exactly VPAC measures. A common interpretation is that it measures only residual acoustic speaker information. Subsequent work complicates this view. “You Are What You Say: Exploiting Linguistic Content for VoicePrivacy Attacks” adapted BERT-base uncased as a text-only ASV-style attacker and reported a mean EER of 35%, with certain speakers as low as 1.60% EER using only transcripts (Gaznepoglu et al., 11 Jun 2025). That study attributes part of the attack success to intra-speaker linguistic content similarity in LibriSpeech and recommends speaker-level EER analysis together with clipped averaging via $P_\text{miss}(\theta)$1 (Gaznepoglu et al., 11 Jun 2025). VoxATtack independently reported that text alone reached 35.8% EERavg and argued that linguistic content preserved for downstream tasks can leak speaker-linked patterns (Aloradi et al., 16 Jul 2025). These findings do not negate VPAC; rather, they indicate that privacy evaluation on anonymized speech may be confounded by content regularities in the evaluation corpus.

Post-challenge attacker research has continued to use VPAC as a benchmark for stronger attack models. SegReConcat proposed word-level segmentation, rearrangement, and concatenation as attacker-side augmentation and improved de-anonymization on five of seven systems (Arefeen et al., 26 Aug 2025). DAST proposed a dual-stream attacker with staged training over spectral and WavLM features, reported that Stage II training on diverse voice conversion data was the primary driver of cross-system generalization, and found that fine-tuning on only 10% of the target anonymized data surpassed prior state of the art on all seven systems (Arefeen et al., 13 Mar 2026). A plausible implication is that VPAC has evolved from a single challenge event into a continuing benchmark for attacker robustness, cross-system transfer, and multimodal privacy auditing.

In the broader VoicePrivacy landscape, VPAC is best understood as an adversarial audit of anonymization rather than a replacement for anonymization research. It formalizes a strong semi-informed threat model, standardizes the use of LibriSpeech-based anonymized corpora and EER-based ranking, and has demonstrated that attacker optimization materially changes the measured privacy of speech anonymization systems (Tomashenko et al., 2024). Its enduring contribution is methodological: voice privacy cannot be credibly claimed unless anonymization is evaluated against strong, adaptive attackers, including attackers that exploit both acoustic residuals and preserved linguistic content (Tomashenko et al., 19 Apr 2025).