---
title: 'TST-Bench: Target Speaker Tagging Benchmark'
url: https://www.emergentmind.com/topics/tst-bench
type: topic
---

# TST-Bench: Target Speaker Tagging Benchmark

TST-Bench is a benchmark for **target speaker tagging (TST)**, a task defined for long multi-speaker recordings with a set of pre-enrolled target speakers. In this setting, a system must detect and label speech segments belonging to known speakers while rejecting speech from unknown speakers. The benchmark was introduced to support evaluation of this integrated problem at scale, with a focus on long recordings, global speaker identities across sessions, open-set conditions, and end-to-end evaluation from diarization to target-speaker assignment [2606.14091].

## 1. Task definition and relation to adjacent speaker tasks

Target speaker tagging combines elements of **speaker diarization**, **speaker verification**, and **speaker identification** into a single workflow. The input is a long recording with multiple speakers, together with a subset of pre-enrolled speakers. The required output is a segmentation of the recording into single-speaker regions, assignment of globally meaningful speaker identities to regions spoken by enrolled speakers, and rejection of all other speech as non-target [2606.14091].

This formulation differs from conventional speaker diarization because diarization normally produces only anonymous, session-local labels such as local speaker clusters. TST instead requires **global identity assignment across sessions** for enrolled speakers, and it explicitly includes **unknown-speaker rejection**. It also differs from speaker verification and standard speaker identification because those tasks are usually formulated on pre-segmented utterances, whereas TST must first operate on long conversational recordings and then perform open-set assignment on the resulting segments.

A central point in the task definition is that TST is not merely diarization followed by identification. The paper emphasizes an asymmetry between clustering errors: **under-clustering**, where multiple true speakers are merged into one cluster, can irreversibly corrupt downstream speaker representation and severely harm identification, whereas **over-clustering**, where one true speaker is split across multiple clusters, is often less damaging because multiple clusters may still be assigned to the same enrolled speaker. This suggests that optimization for TST is not equivalent to optimization for diarization metrics alone.

## 2. Benchmark structure and dataset composition

TST-Bench is a **large-scale synthetic benchmark** built from real single-speaker source recordings and mixed into multi-speaker sessions with global annotations. Its stated purpose is to provide a benchmark that supports global speaker labeling across sessions, a large enrolled population, long-form multi-speaker recordings, and an evaluation protocol for both diarization and the full TST pipeline [2606.14091].

The benchmark statistics reported in the paper are as follows.

| Aspect | Value |
|---|---|
| Source corpus | MLS (English) |
| Total speakers | 350 |
| Enrolled speakers | 150 |
| Unknown speakers | 200 |
| Sessions | 300 |
| Session duration | 20–60 minutes |
| Speakers per session | 8–30 |
| Unknown speakers per session | 0–10 |
| Enrollment duration | 20 s per enrolled speaker |
| Audio format | 16 kHz mono |
| Annotation format | RTTM with global speaker labels |
| Total audio | approximately 200 hours |

The benchmark uses **global speaker labels**, so the same person retains the same identifier across sessions. This is a defining property of the resource, since TST aims to recognize known speakers globally rather than assign only session-local anonymous labels. Unknown speakers are included explicitly: there are 200 unknown speakers in the global pool, each session includes 0 to 10 unknown speakers, and their speech must be tagged as non-target rather than matched to the enrolled gallery.

The paper also stresses the importance of per-session speaker count. TST-Bench supports **8–30 speakers per session**, which is intended to create a more demanding open-set identification regime than typical meeting corpora with fewer participants. This suggests that TST-Bench is designed not only as a larger dataset, but as a benchmark whose session structure amplifies the interaction between diarization errors and open-set tagging.

## 3. Construction pipeline

The benchmark is synthesized from **single-speaker MLS recordings**. To ensure sufficient source material, the authors randomly select 350 English speakers, each with at least one hour of speech. Because MLS recordings are crowdsourced and may encode speaker-specific channel conditions, the source recordings are enhanced with **Resemble Enhance** denoising before synthesis [2606.14091].

Word-level time boundaries are obtained using the **Montreal Forced Aligner (MFA)**. Source speech chunks are then formed by grouping consecutive voiced intervals, using silence gaps greater than **0.3 s** as boundaries. The full speaker inventory is partitioned into **150 enrolled speakers** and **200 unknown speakers**. For each enrolled speaker, approximately **20 seconds** of speech-only audio is selected as enrollment material and excluded from evaluation.

Session generation proceeds in four stages. In **session planning**, each session duration is sampled uniformly from **20 to 60 minutes**, the number of speakers is sampled from **8 to 30**, each session includes at least one enrolled speaker, each enrolled speaker appears in **10 to 30 sessions**, and each session includes **0 to 10 unknown speakers**. In **turn layout**, turns are arranged sequentially with inter-turn gaps of **0.15 to 2.5 s** and turn durations of **0.5 to 15 s**. Speaker participation proportions are sampled from a symmetric Dirichlet distribution with concentration parameter $\alpha = 3.0$.

In **audio mixing**, speech segments are drawn from the source corpus without replacement so that no speech content is reused across sessions. Each session is mixed onto a continuous ambient-noise track, with session-level SNR sampled uniformly from **0 to 10 dB**. Ambient noises are collected from **Freesound**, restricted to candidate recordings of at least **3 minutes**, filtered with an in-house VAD to keep recordings with **less than 5% detected speech**, and repeated with fade-in and fade-out when shorter than session duration. In **annotation generation**, each session is labeled with an **RTTM file** containing onset time, duration, and a global speaker label, with each label additionally marked as enrolled or unknown.

The paper is explicit that TST-Bench is synthetic rather than a real-conversation corpus. It also states that this synthesis does **not** fully simulate reverberation, crosstalk, far-field microphone effects, or realistic meeting-room acoustics. A plausible implication is that the benchmark prioritizes controlled scale, exact labeling, and reproducibility over full acoustic realism.

## 4. Evaluation protocol and metrics

TST-Bench defines two evaluation scenarios: **speaker diarization** and the **full TST pipeline** [2606.14091].

For the full pipeline, the paper defines **Detection and Identification Rate (DIR)** and **False Alarm Rate (FAR)**. Let $\mathbf{D}$ denote the set of evaluated single-speaker segments, $\mathbf{D}_T$ the target segments from enrolled speakers, and $\mathbf{D}_N$ the non-target segments from unknown speakers. With threshold $\theta$, the thresholded tagging decision is

$$
\text{id}_{\text{tag}(d, \theta)} =
\begin{cases}
\widetilde{\text{id}_{\text{tag}(d)}} & \text{if } \text{score}_{\max}(d) \ge \theta \\
\text{non-target} & \text{otherwise.}
\end{cases}
$$

The benchmark then reports

$$
\text{DIR}(\theta) =
\frac{
\left|
\left\{
d_t \mid \text{id}_{\text{tag}(d_t,\theta)} = \text{id}_{\text{ans}(d_t)},\ d_t \in \mathbf{D}_T
\right\}
\right|
}{|\mathbf{D}_T|}
$$

and

$$
\text{FAR}(\theta) =
\frac{
\left|
\left\{
d_n \mid \text{score}_{\max}(d_n) \ge \theta,\ d_n \in \mathbf{D}_N
\right\}
\right|
}{|\mathbf{D}_N|}.
$$

The principal reported end-to-end metric is therefore **DIR@FAR** at several operating points.

A notable methodological contribution is the **reference-anchored evaluation protocol**. Because different diarization systems produce different segmentations, the benchmark first selects a fixed set of **reference evaluation segments** from the reference RTTM: these must be **non-overlapping**, **single-speaker**, and have duration at least

$$
\tau_{\min} = 1 \text{ s}.
$$

For each reference segment, the system-generated segment with the longest temporal overlap is identified, and its label is transferred to the reference segment for scoring. This design makes system comparisons segmentation-independent in a controlled way. It also naturally penalizes under-clustering: if one long system segment spans several reference regions, only those matching the assigned identity are correct. If a reference segment has no overlapping system output, it remains in the denominator and lowers DIR.

For diarization-only evaluation, the benchmark reports **DER** and also **JER** in the main diarization results.

## 5. Baseline system design

The paper evaluates TST-Bench using a pipeline that combines a diarization frontend with open-set speaker identification [2606.14091].

The diarization system is built on a **High-Resolution Embedding Extractor (HEE)** trained on **VoxCeleb1 and VoxCeleb2**, followed by dimensionality reduction, attention-based aggregation, and **spectral clustering**. Open-set speaker identification uses **ECAPA-TDNN**, also trained on **VoxCeleb1 and VoxCeleb2**, with a **4-second window**, **1.5-second shift**, and **256-dimensional** speaker embeddings. If a segment or enrollment utterance yields multiple embeddings, score-level average pooling is used. Similarity scoring is performed with **cosine similarity** and **adaptive symmetric normalization (AS-Norm)**, using a cohort of **2,000 speakers** randomly selected from **VoxBlink2** and adaptation size **20**.

The paper characterizes this as a largely **naive integration** of diarization and open-set identification, and then studies several TST-specific modifications. These include **clustering tendency adjustment** with baseline, under-clustered, and over-clustered configurations; **segment margin extension**, where diarized segments are symmetrically expanded before embedding extraction; and **short-utterance compensation**, which aggregates evidence from other segments with the same diarization label using strategies such as Top-1, Top-2, Top-3, and label-based aggregation.

This suggests that the benchmark is not only a dataset release but also a framework for testing how speaker-tagging performance depends on choices that are largely invisible in traditional verification-style benchmarks.

## 6. Empirical findings

On TST-Bench diarization, the baseline system achieves **DER = 8.70%**, decomposed into **Miss = 6.23%**, **False alarm = 0.35%**, and **Speaker confusion = 2.12%**, with **JER = 15.66%** [2606.14091]. The dominant error source is missed speech, which the paper attributes mainly to overlapping speech and the lack of explicit overlap handling in the baseline diarization system.

For the full TST pipeline on TST-Bench, the baseline results are:

| Operating point | DIR |
|---|---:|
| FAR = 0.5% | 88.79% |
| FAR = 1% | 93.00% |
| FAR = 5% | 96.80% |
| FAR = 10% | 97.61% |

The clustering-tendency experiment yields one of the paper’s main conclusions. On TST-Bench, the baseline gives **DIR@FAR=0.5% = 88.79%**, the **under-clustered** configuration gives **86.75%**, and the **over-clustered** configuration gives **89.46%**. The paper therefore concludes that **over-clustering gives the best TST result**, whereas **under-clustering hurts most**, even though over-clustering increases speaker confusion relative to the baseline. This directly supports the claim that TST imposes design pressures not captured by conventional diarization objectives.

The segment-margin experiment shows only modest changes. On TST-Bench, **0.1 s** margin gives **89.05%** at FAR 0.5%, compared with **88.79%** for no margin, while larger margins yield little additional gain. The interpretation given is that tightly cropped diarization segments can lose useful speaker information near boundaries.

For short-utterance compensation, **Top-N** aggregation improves results consistently on TST-Bench. The reported values at FAR 0.5% are **88.95%** for Top-1, **88.94%** for Top-2, and **89.03%** for Top-3, compared with **88.79%** without compensation. The **label-based** strategy performs poorly at strict operating points, dropping to **81.82%** at FAR 0.5%, which the paper attributes to contamination from diarization errors, especially under-clustering.

The paper also evaluates on real **ICSI** meetings. The direction of the main findings remains consistent, but the effects are smaller. A plausible implication, explicitly suggested in the paper, is that TST-Bench’s larger speaker counts per session make the benchmark more sensitive to interactions between diarization and identification than smaller real-meeting corpora.

## 7. Scope, limitations, and significance

TST-Bench is presented as a response to the absence of suitable evaluation resources for target speaker tagging, but the paper is also explicit about its limitations [2606.14091]. The source speech comes from **MLS**, which consists of read speech rather than spontaneous conversation. Session turn-taking is algorithmically generated and does not fully reproduce backchannels, interruptions, floor-holding behavior, or richer multi-party conversational dynamics. The acoustic mixing process also does not fully simulate reverberation, crosstalk, far-field capture, or realistic meeting-room acoustics. In addition, the paper does **not** specify an official train/dev/test split in the conventional sense; the benchmark is framed primarily as an evaluation resource.

Within those limits, the benchmark’s significance lies in two areas. First, it supplies a controlled large-scale resource with **global speaker labels**, **150 enrolled speakers**, **200 unknown speakers**, **300 sessions**, and a **reference-anchored protocol** for evaluating both diarization and end-to-end TST. Second, it exposes properties of the task that conventional speaker benchmarks do not make salient, especially the asymmetry between under-clustering and over-clustering and the importance of handling short diarization-driven segments under open-set conditions.

In that sense, TST-Bench is best understood not simply as a new dataset, but as an attempt to define a distinct evaluation regime for **multi-session, multi-speaker, open-set target speaker tagging with global speaker identities**.

Source: https://www.emergentmind.com/topics/tst-bench