Same-Gender TSA for Speaker Anonymization

• Same-Gender TSA is an utterance-level rule that selects a target speaker from candidates sharing the same gender, thereby enforcing a gender constraint during voice conversion.
• It plays a critical role in anonymization pipelines by influencing metrics such as Equal Error Rate (EER) and validation error rate (VER) through the mixing of source and target speaker attributes.
• Adversarial training with a Gradient Reversal Layer is used to suppress target identity cues, ensuring that evaluators do not erroneously rely on target information during privacy assessments.

Searching arXiv for the cited paper and closely related speaker anonymization work. Using the arXiv search tool to retrieve relevant papers. The same-gender Target Selection Algorithm (TSA) is an utterance-level target-choice rule used in speaker anonymization systems that rely on voice conversion. For each utterance from a source speaker $s$, it selects a target speaker $t$ uniformly from the subset of candidate targets that share the same binary gender label as $s$. In the evaluation setting studied by Franzreb et al., this apparently simple constraint has disproportionate methodological importance: under VoicePrivacy Challenge-style evaluation, same-gender TSA can produce near-random Equal Error Rates (EERs) that overestimate privacy, even though the recognizer knows the source’s gender. The reported explanation is that anonymized speech contains information from both the source and the target, so an evaluator trained only to predict sources can be confounded by target identity cues; the proposed remedy is to quantify and adversarially suppress those cues during evaluation (Franzreb et al., 13 Aug 2025).

1. Formal definition

Let $S$ denote the set of source speakers in the development set, $T$ a pool of candidate target speakers, $G:S\cup T\rightarrow\{\mathrm{M},\mathrm{F}\}$ a binary gender map, and $\mathcal{A}$ the anonymization function that transforms an utterance $x$ from source $s$ toward a target $t$. The same-gender TSA is defined by sampling

$t$0

No embedding-distance or scoring criterion is used; selection is uniform within the same-gender subset. The contrast condition is the random TSA, which samples

$t$1

This definition isolates gender conditioning as the sole difference between the two TSAs. In the reported experiments, the target pool consists of 100 LibriTTS train-other-500 targets, equally split by gender, so the constraint operates by reducing the target pool available to each source speaker rather than by reweighting targets within a shared pool (Franzreb et al., 13 Aug 2025).

2. Role in the anonymization pipeline

The operational procedure is dataset-level but utterance-specific. For a dataset $t$2, one first precomputes, for each source speaker $t$3, the subset $t$4 of same-gender targets. Then, for each pair $t$5, one draws $t$6, computes $t$7, and appends $t$8 to the anonymized dataset. The source label is retained because evaluation is framed as speaker verification against the original source identity.

A defining property of the protocol is that target selection is performed at utterance level during both training and evaluation. Each utterance of a given speaker may therefore be paired with an independently drawn target. In the training phase, the anonymizer is applied utterance-level to the training data under either same-gender or random TSA. In the evaluation phase, enrollment and trial utterances are anonymized utterance-level with the same TSA used in training. This design matters because it increases target variability at fixed source identity while preserving the evaluation’s source-centric labeling scheme, thereby creating the conditions under which target leakage can distort privacy estimates (Franzreb et al., 13 Aug 2025).

3. Evaluation protocol and privacy metrics

The evaluation follows the VoicePrivacy Challenge 2024 plan as implemented in the SpAnE framework. A speaker recognizer with an ECAPA-TDNN backbone from SpeechBrain maps speech to embeddings $t$9. During training, a source classifier $s$0 is attached and optimized with cross-entropy loss to predict the source label. During evaluation, the source classifier is removed. For each claimed source $s$1, enrollment embeddings are averaged to form a template $s$2, and each trial embedding is scored by cosine similarity:

$s$3

A binary threshold is swept to compute False Acceptance Rate (FAR) and False Rejection Rate (FRR), and the Equal Error Rate (EER) is the operating point where $s$4. Under this convention, $s$5 EER means perfect speaker recovery and $s$6 means random guessing.

The paper also introduces validation error rate (VER) as a diagnostic for whether a classifier has learned its intended label from anonymized speech. With classifier posteriors $s$7, the cross-entropy on held-out anonymized data is

$s$8

and VER is the fraction of misclassified samples. In this setting, source VER and target VER measure different residual identity channels in the anonymized embeddings. That distinction is central to the analysis of same-gender TSA, because EER alone can make the evaluation appear privacy-preserving even when target identity remains highly recoverable (Franzreb et al., 13 Aug 2025).

4. Adversarial target-classifier formulation

To measure and suppress target leakage, the recognizer is augmented with a second head $s$9 that predicts the target speaker. A Gradient Reversal Layer (GRL) is inserted before $S$0. In the forward pass, the GRL acts as the identity; in the backward pass, it multiplies the gradient by $S$1. The resulting architecture jointly supports source discrimination and adversarial removal of target information.

For anonymized triples $S$2 with embedding $S$3, the losses are

$S$4

$S$5

With GRL weight schedule $S$6, the batch objective is

$S$7

Operationally, $S$8 is trained to minimize target-classification loss, while the backbone is trained to maximize that loss and thus remove target information without ceasing to minimize source loss. The weight schedule follows GANin and Lempitsky: $S$9 for the first $T$0 epochs, then a linear increase from epoch $T$1 to epoch 10 until $T$2. Franzreb et al. report that $T$3 minimizes source VER on the validation split. This formulation reframes same-gender TSA from a mere data-selection heuristic into a stress test for whether the evaluation network is exploiting target identity rather than measuring residual source identity (Franzreb et al., 13 Aug 2025).

5. Empirical behavior under same-gender constraints

The experimental setup uses LibriSpeech train-clean-360 for training, test-clean for evaluation, and a 10% per-speaker hold-out from training data for validating VER. The recognizer is an ECAPA-TDNN in the standard SpeechBrain configuration, trained for 10 epochs with default Adam at learning rate $T$4, weight decay $T$5, and batch size 64. Three random seeds are used for significance assessment.

Several empirical observations are specific to same-gender TSA. First, recognizer capacity matters. The smaller ECAPA-TDNN used in the VoicePrivacy Challenge yields EERs of approximately $T$6 for same-gender TSA, whereas the full-size SpAnE recognizer reports more consistent EERs below $T$7. Second, when both source and target classifiers are trained without gradient reversal, the recognizer learns targets much more effectively than sources: final target VERs are $T$8 for random TSA and $T$9 for same-gender TSA, while final source VERs are $G:S\cup T\rightarrow\{\mathrm{M},\mathrm{F}\}$0 and $G:S\cup T\rightarrow\{\mathrm{M},\mathrm{F}\}$1, respectively. The stated conclusion is that anonymized embeddings carry strong target identity cues and that the recognizer latches onto them.

A concise subset of the reported EER results is shown below.

Configuration	EER$G:S\cup T\rightarrow\{\mathrm{M},\mathrm{F}\}$2	EER$G:S\cup T\rightarrow\{\mathrm{M},\mathrm{F}\}$3
private kNN-VC $G:S\cup T\rightarrow\{\mathrm{M},\mathrm{F}\}$4, random TSA	$G:S\cup T\rightarrow\{\mathrm{M},\mathrm{F}\}$5	$G:S\cup T\rightarrow\{\mathrm{M},\mathrm{F}\}$6
private kNN-VC $G:S\cup T\rightarrow\{\mathrm{M},\mathrm{F}\}$7, same-gender TSA	$G:S\cup T\rightarrow\{\mathrm{M},\mathrm{F}\}$8	$G:S\cup T\rightarrow\{\mathrm{M},\mathrm{F}\}$9
private kNN-VC $\mathcal{A}$0, same-gender TSA	$\mathcal{A}$1	$\mathcal{A}$2

For random TSA, adversarial removal of target information produces negligible EER changes. For same-gender TSA, by contrast, adversarial training reduces EER by $\mathcal{A}$3 to $\mathcal{A}$4 relative, and target VER rises from single digits to approximately $\mathcal{A}$5. The paper interprets this as evidence that the baseline evaluator was partly “cheating” by relying on target cues, especially under same-gender constraints. A common misreading is therefore corrected: a near-random EER under same-gender TSA does not necessarily indicate stronger privacy; it may instead indicate evaluator confusion induced by mixed source-target information in anonymized speech (Franzreb et al., 13 Aug 2025).

6. Methodological significance and terminological scope

Within speaker anonymization, the same-gender TSA is not an anonymization model by itself but a target-selection policy that conditions the behavior of voice-conversion-based anonymizers and, crucially, the behavior of privacy evaluators. Its methodological significance comes from the mismatch it can create between what the evaluator is trained to predict and what the anonymized signal actually contains. The paper’s recommendation is correspondingly procedural: when baseline target VER exceeds approximately $\mathcal{A}$6, especially under constrained TSAs such as same-gender selection, the evaluation should be augmented with a GRL and target classifier, the adversarial weight should be ramped from 0 to 1 after a warm-up of 6–8 epochs, target VER should be monitored, and EER should be recomputed under the adversarially trained recognizer. A jump of target VER above $\mathcal{A}$7 is presented as evidence of effective removal of target cues (Franzreb et al., 13 Aug 2025).

The acronym “TSA” is also used in unrelated research areas. In influence maximization on social networks, “Targeted Influence with Community and Gender-Aware Seeding” introduces a community- and gender-aware seeding method that has been recast as a Target Selection Algorithm for selecting seed users under a target-gender spread constraint, using community detection, Gender-Aware Potential Influence, and a swap-based local search procedure (Styczen et al., 2022). This suggests that the acronym is domain-dependent: in speaker anonymization it denotes a policy for choosing voice-conversion targets, whereas in social diffusion it denotes a policy for choosing seed nodes. The same-gender TSA discussed here belongs specifically to the former usage and is best understood as an evaluation-critical design choice in privacy-preserving speech processing.