VoxATtack: Multimodal Speaker De-anonymization

• VoxATtack is a multimodal de-anonymization system that exploits preserved acoustic and textual cues to link voice data to the original speaker.
• The dual-branch architecture integrates ECAPA-TDNN for audio and BERT for text, using confidence-weighted fusion to enhance speaker verification.
• Benchmark results on VPAC datasets reveal significant EER reductions, underscoring the impact of linguistic leakage on voice privacy.

VoxATtack is a multimodal speaker de-anonymization system for attacking voice anonymization methods by combining anonymized speech with its transcript. It was introduced as a dual-branch model in which an acoustic branch based on ECAPA-TDNN is paired with a textual branch based on pretrained BERT, with the stated premise that anonymization systems usually preserve linguistic content and that this preserved content can itself leak speaker-linked information such as lexical preferences, habitual phrases, discourse markers, topic consistency, and broader idiolect. Operationally, VoxATtack does not attempt to reconstruct an original waveform; it frames de-anonymization as anonymized automatic speaker verification, asking whether anonymized utterances come from the same original speaker (Aloradi et al., 16 Jul 2025).

1. Position within voice anonymization research

VoxATtack is situated in the VoicePrivacy line of work, and specifically in the evaluation regime formalized by the First VoicePrivacy Attacker Challenge. In that setting, the attacker is an automatic speaker verification system operating in a semi-informed model, with access to anonymized trial utterances, original and anonymized enrollment utterances, original and anonymized training data, descriptions of the anonymization system, and software implementation when available. The challenge evaluates attackers on seven anonymization systems—B3, B4, B5, T8-5, T10-2, T12-5, and T25-1—using equal error rate, with lower EER indicating a stronger attacker and therefore weaker privacy protection (Tomashenko et al., 19 Apr 2025).

Within that framework, VoxATtack advances a specific argument: the ceiling of de-anonymization performance is not set only by acoustic modeling. The preserved text of anonymized speech can also act as an attack surface. This is especially salient in the VPAC corpus, which is built from LibriSpeech. In the reported setup, development and test sets are gender-segregated and partitioned into ASV trials with roughly 95.4% negative trials and 4.6% positive trials, and the aggregate score is reported as Dev EER, Test EER, and $\mathrm{EER}_{\mathrm{avg}}$, the average of development and test EERs. VoxATtack therefore belongs to a class of deanonymization attackers that exploit not only residual speaker cues in acoustics but also speaker-linked regularities in linguistic content (Aloradi et al., 16 Jul 2025).

2. Task formulation and leakage model

The VoxATtack formulation is standard verification in the anonymized domain. Given enrollment utterances and a trial utterance, the attacker predicts whether they belong to the same speaker. During inference, embeddings are extracted from anonymized utterances, enrollment embeddings are averaged into speaker centroids, and trial utterances are scored by cosine similarity to those centroids. The benchmark score is effectively

$$\mathrm{EER}_{\mathrm{avg}} = \frac{\mathrm{EER}_{\mathrm{dev}} + \mathrm{EER}_{\mathrm{test}}}{2}.$$

The conceptual novelty lies in the leakage model. VoxATtack treats anonymization systems as deliberately preserving the words while altering vocal traits, and argues that the preserved words are themselves informative. The paper explicitly associates this with lexical preferences, filler words, discourse markers, syntactic style, topic consistency, and idiolectal regularities. One of its most consequential empirical observations is that a text-only branch, using transcript embeddings as speaker representations, reaches an average EER of 35.8%, exceeding the official acoustic baseline on all but B3, B4, and B5. This result is used to argue that linguistic leakage is not merely auxiliary but structurally important in the VPAC setting (Aloradi et al., 16 Jul 2025).

The de-anonymization task is therefore narrower than waveform reconstruction and broader than conventional speaker embedding recovery. It asks whether anonymized speech still retains enough information—acoustic, textual, or both—to link utterances back to the same source speaker. A plausible implication is that VoxATtack shifts the privacy discussion from “how much voice identity remains in the signal” to “how much speaker-linked information remains in the full multimodal utterance representation.”

3. Dual-branch architecture and fusion

VoxATtack uses a dual-branch architecture with an acoustic encoder and a textual encoder. The acoustic branch takes anonymized speech $X_{\text{audio}}$ and uses SpeechBrain’s pretrained ECAPA-TDNN, initialized from VoxCeleb 1 and 2, to extract a 192-dimensional speaker embedding. This output is passed through an audio projection module consisting of LayerNorm, a fully connected layer $192 \rightarrow 512$, GELU, and dropout with $p = 0.3$, producing $e_{\text{audio}}\in\mathbb{R}^{512}$. A confidence estimator maps this embedding to a scalar $y_{\text{audio}}\in(0,1)$ through a $512 \rightarrow 256 \rightarrow 1$ MLP with ReLU and sigmoid.

The textual branch takes a transcription $X_{\text{text}}$. It uses BERT-base-uncased from Hugging Face, with weights kept fixed, and uses the 768-dimensional pooled $[CLS]$ representation as a sentence-level embedding. This is projected through LayerNorm, a fully connected layer $$\mathrm{EER}_{\mathrm{avg}} = \frac{\mathrm{EER}_{\mathrm{dev}} + \mathrm{EER}_{\mathrm{test}}}{2}.$$0, GELU, and dropout with $$\mathrm{EER}_{\mathrm{avg}} = \frac{\mathrm{EER}_{\mathrm{dev}} + \mathrm{EER}_{\mathrm{test}}}{2}.$$1, producing $$\mathrm{EER}_{\mathrm{avg}} = \frac{\mathrm{EER}_{\mathrm{dev}} + \mathrm{EER}_{\mathrm{test}}}{2}.$$2. A structurally identical confidence estimator yields $$\mathrm{EER}_{\mathrm{avg}} = \frac{\mathrm{EER}_{\mathrm{dev}} + \mathrm{EER}_{\mathrm{test}}}{2}.$$3. The equal-dimensional projections are a deliberate design choice, removing the dimensional imbalance between the 192-dimensional ECAPA output and the 768-dimensional BERT output (Aloradi et al., 16 Jul 2025).

Fusion is confidence-weighted and per-utterance: $$\mathrm{EER}_{\mathrm{avg}} = \frac{\mathrm{EER}_{\mathrm{dev}} + \mathrm{EER}_{\mathrm{test}}}{2}.$$4 The model also includes separate classifiers for the audio, text, and fused embeddings, plus an ensemble over their logits: $$\mathrm{EER}_{\mathrm{avg}} = \frac{\mathrm{EER}_{\mathrm{dev}} + \mathrm{EER}_{\mathrm{test}}}{2}.$$5 with $$\mathrm{EER}_{\mathrm{avg}} = \frac{\mathrm{EER}_{\mathrm{dev}} + \mathrm{EER}_{\mathrm{test}}}{2}.$$6, $$\mathrm{EER}_{\mathrm{avg}} = \frac{\mathrm{EER}_{\mathrm{dev}} + \mathrm{EER}_{\mathrm{test}}}{2}.$$7, and $$\mathrm{EER}_{\mathrm{avg}} = \frac{\mathrm{EER}_{\mathrm{dev}} + \mathrm{EER}_{\mathrm{test}}}{2}.$$8. These weights are produced by a gating network taking the concatenation of the two embeddings and the two confidence scalars, with input dimension $$\mathrm{EER}_{\mathrm{avg}} = \frac{\mathrm{EER}_{\mathrm{dev}} + \mathrm{EER}_{\mathrm{test}}}{2}.$$9, then $X_{\text{audio}}$0, and softmax at the output. The architecture is intentionally not exotic; its contribution is the demonstration that a standard ECAPA-TDNN plus a standard BERT materially strengthens deanonymization.

4. Training objective, scoring, and augmentation

Training uses a multi-task additive angular margin objective: $X_{\text{audio}}$1 with $X_{\text{audio}}$2 and $X_{\text{audio}}$3. The AAM parameters are scale $X_{\text{audio}}$4 and margin $X_{\text{audio}}$5. During inference, all classifier heads are discarded and the fused embedding is used as the speaker representation. For an enrolled speaker,

$X_{\text{audio}}$6

and embeddings are $X_{\text{audio}}$7-normalized before cosine scoring: $X_{\text{audio}}$8 Scores are normalized using adaptive score normalization with the top 1000 cohort embeddings selected from 40k random training samples, except for T8-5, where AS-norm is skipped because that anonymizer randomly applies one of two anonymization methods (Aloradi et al., 16 Jul 2025).

Implementation details are unusually explicit. Training speech is cropped to at most 10 seconds, mean-centered, normalized to $X_{\text{audio}}$9 dB RMS, and clipped to maximum amplitude 1.0. Transcriptions are not truncated. ECAPA is fine-tuned, while BERT remains fixed. Parameter counts are reported as 22.2M for the audio embedding model, 0.1M for the audio projection, 0.1M for the audio confidence estimator, 109.0M for the fixed BERT text model, 0.4M for the text projection, 0.1M for the text confidence estimator, 0.5M for the fusion gating module, and 2.1M for the classifiers, for a total of 25.5M trainable parameters (Aloradi et al., 16 Jul 2025).

Augmentation is central to the method’s later gains. Two families are used: SpecAugment-style augmentation with time masking, frequency masking, and speed perturbation of $192 \rightarrow 512$0, and anonymized-speech augmentation using anonymized speech from other anonymization systems. In the standard pipeline, each original sample generates three additional samples, so a batch of 32 consists of 8 original anonymized samples and 24 augmented samples. For T10-2, T12-5, and T25-1, the paper augments training with anonymized speech from B3, B4, or B5 and reports that this substantially boosts performance, especially on T10-2 (Aloradi et al., 16 Jul 2025).

5. Benchmark performance on VPAC

In its base form, VoxATtack outperforms prior attackers on five of the seven VPAC benchmarks—B3, B4, B5, T8-5, and T12-5. The main reported $192 \rightarrow 512$1 values are: B3 20.8, B4 20.6, B5 26.1, T10-2 29.0, T12-5 26.8, T25-1 31.1, and T8-5 24.1. Against the official baseline and strong prior systems such as A.5 and A.20, these values show that text contributes beyond acoustics alone. The paper also defines an audio-only control, ECAPAours, trained under the same general regime but without text or fusion. ECAPAours is already much stronger than the official ECAPAbaseline, and VoxATtack improves over ECAPAours on every benchmark (Aloradi et al., 16 Jul 2025).

The relative gains are uneven across anonymizers. The improvement over ECAPAours is modest on B4, but much larger on T8-5. T10-2 and T25-1 remain the hardest anonymizers in the unaugmented setup. After augmentation, however, VoxATtack is reported to achieve state of the art on all seven VPAC benchmarks, with $192 \rightarrow 512$2 on T10-2 and $192 \rightarrow 512$3 on T25-1. The largest gain is on T10-2, where augmentation reduces EERavg from 29.0 to 20.6; T25-1 improves from 31.1 to 27.2; T12-5 improves more modestly, to around 26.2. The authors note an especially large 8.4% drop in EERavg on T10-2 when augmented with B4 and speculate that a shared neural audio-codec characteristic might partly explain the transfer (Aloradi et al., 16 Jul 2025).

A separate cross-system experiment trains a single model simultaneously on all seven anonymization systems plus original LibriSpeech. The resulting EERavg values are B3 21.5, B4 21.9, B5 26.1, T10-2 21.4, T12-5 26.7, T25-1 28.9, T8-5 24.6, and LibriSpeech 4.2. This contradicts earlier reports that all-systems training causes near-random performance. The most striking anomaly is T10-2, which appears relatively strong under single-system training but becomes markedly easier to attack in the all-systems regime.

6. Interpretation, limitations, and later developments

VoxATtack’s main interpretation is that voice anonymization should not be treated as a purely acoustic privacy problem. If anonymizers preserve content for downstream ASR or speech processing, they may also preserve speaker-linked semantic and stylistic regularities. The method therefore reveals a structural vulnerability in benchmarks derived from LibriSpeech: the privacy evaluation can underestimate leakage if it ignores transcripts or transcript-recoverable information. The authors also explicitly raise the possibility that the dataset itself contributes to the observed effect, since LibriSpeech’s audiobook structure may encode speaker-specific topical or stylistic regularities not representative of spontaneous speech (Aloradi et al., 16 Jul 2025).

Several limitations follow directly from the reported setup. VoxATtack uses ground-truth transcripts rather than ASR output, which isolates linguistic leakage but may overestimate attacker strength in settings where only noisy ASR transcripts are available. The method does not quantify which textual features are most responsible for re-identification. Its strong cross-system results, especially on T10-2, are presented as surprising and in need of further investigation. More generally, the paper implies that privacy conclusions can be highly sensitive to attacker implementation details and to whether evaluation includes multimodal leakage channels.

The name also invites confusion. VoxATtack is unrelated to “VoxAtnNet,” a 3D point-cloud convolutional attention network for face presentation attack detection on smartphones; the latter concerns facial biometrics rather than voice anonymization (Ramachandra et al., 2024). In subsequent attacker research, DAST cited VoxATtack as a prior multimodal deanonymization system and reported lower EER on all seven VPAC anonymizers, attributing its gains chiefly to a three-stage training curriculum with voice-conversion-based transfer and lightweight target adaptation. In that later comparison, VoxATtack serves as a reference point for multimodal attacks that exploit both acoustics and text, while DAST is presented as a stronger audio-only attacker with broader cross-system generalization (Arefeen et al., 13 Mar 2026).

Taken together, VoxATtack is best understood as a benchmark-defining argument as much as a model. Its empirical contribution is a dual-branch ECAPA-TDNN–BERT attacker with confidence-weighted fusion and selective augmentation. Its broader significance is the demonstration that preserved linguistic content can materially strengthen deanonymization, forcing voice anonymization research to confront privacy leakage outside the conventional acoustic speaker-embedding channel.