Time-Constrained cpWER (tcpWER) Metric

• tcpWER is defined as the cpWER metric modified with a time-constrained Levenshtein distance, ensuring temporal plausibility in word alignment.
• It prevents aligning words that are temporally distant by assigning an infinite cost, effectively converting some substitutions into insertions and deletions.
• tcpWER offers practical benefits for evaluating long meeting transcriptions and diarization by balancing speaker attribution with realistic timing constraints.

Searching arXiv for the specified paper and closely related metrics. Time-Constrained cpWER (tcpWER) is the time-constrained minimum-permutation Word Error Rate, defined as the time-constrained version of cpWER for long-form multi-talker speech recognition. It retains the speaker-attributed, minimum-permutation structure of cpWER, but replaces the unconstrained Levenshtein alignment with a time-constrained Levenshtein distance so that words that are temporally too far apart cannot be aligned as correct matches or substitutions. In the formulation of "Word Error Rate Definitions and Algorithms for Long-Form Multi-talker Speech Recognition" (Neumann et al., 4 Aug 2025), tcpWER is introduced to prevent unreasonable matchings with a time-constraint when working with long word sequences and thereby to make evaluation more realistic for long-form meeting transcription and diarization.

1. Conceptual role in long-form multi-talker evaluation

tcpWER addresses a failure mode of ordinary Levenshtein-based WER alignment in long recordings with multiple speakers. In standard cpWER, the system first finds the best speaker permutation and then aligns speaker-wise concatenated transcripts with ordinary edit distance. That alignment ignores timestamps completely. For long-form multi-talker speech recognition, this can become implausible because standard edit-distance alignment always prefers a substitution over an insertion+deletion pair whenever it lowers edit cost, even if the two words are far apart in time. The associated study argues that, in such cases, a human would count the outcome as two errors rather than one substitution, or would not consider it a reasonable match at all (Neumann et al., 4 Aug 2025).

This makes tcpWER a metric for settings in which temporal consistency matters in addition to lexical correctness and speaker attribution. Its intended scope is long-form meeting transcription and diarization, especially when timestamps exist and the evaluation should reflect whether recognized words occur at plausible times. The metric therefore evaluates transcription and diarization jointly in a stronger sense than cpWER, because the timestamps attached to words must correspond to their actual temporal positions.

2. Formal definition

The starting point is cpWER, defined as

$\distance^\text{cp} = \min_{\perm} \sum_{\ispk=1}^\nspk \lev\!\left(\select(\ind_{\{i:\rlble_i = \ispk\}})\rv,\; \select(\ind_{\{i:\hlble_i=\pi_\ispk\}})\hv\right).$

Here, $\rv$ denotes reference segments, $\hv$ denotes hypothesis segments, $\rlbl$ denotes reference speaker labels, $\hlbl$ denotes hypothesis speaker labels, and $\pi$ is a permutation, or one-to-one mapping, between reference speakers and hypothesis streams. The operator $\select(\ind_{\{i:\cdot\}})$ selects all segments of a given speaker or stream. If the number of reference and hypothesis labels differs, dummy labels with empty transcripts are inserted so that every reference transcript is matched with exactly one hypothesis transcript (Neumann et al., 4 Aug 2025).

tcpWER is obtained by replacing $\lev$ with the time-constrained Levenshtein distance $\tclev$. In explicit form, tcpWER is cpWER using the time-constrained Levenshtein alignment. The key mechanism is the time-constrained transition cost

$C_\text{C/S}^\text{tc}(r, h) = \begin{cases} \infty, & \text{if } \rtbege_r - \htende_h > \collar \vee \htbege_h - \rtende_r > \collar, \ 0, & \text{if } \rtrne_r = \htrne_h, \ 1, & \text{otherwise.} \end{cases}$

The parameter $\rv$0 is the allowed time tolerance. If the reference word $\rv$1 and hypothesis word $\rv$2 are too far apart in time in either direction, then a correct match or substitution is forbidden by assigning cost $\rv$3. Otherwise, an exact word match costs $\rv$4 and a different word costs $\rv$5. This is the formal device by which tcpWER prevents implausible long-range substitutions.

3. Alignment semantics and error accounting

Operationally, tcpWER proceeds by grouping reference and hypothesis segments by speaker or stream labels, searching over speaker permutations as in cpWER, and computing the Levenshtein alignment within each candidate pairing using the time-constrained cost function. Any word pair farther apart than the collar cannot be aligned as a correct match or substitution. The metric still counts the usual WER error types—insertions, deletions, and substitutions—but it prevents implausible substitution or correct-match decisions across large temporal distances (Neumann et al., 4 Aug 2025).

The practical consequence is that some alignments that appear as substitutions under cpWER become insertion+deletion structures under tcpWER. This is not a change in the error inventory but a change in admissible alignments. In the reported experiments, cpWER is always a lower bound on tcpWER, expressed as

$\rv$6

When cpWER and tcpWER differ substantially, the difference is interpreted as a sign of temporal misalignment or implausible matching. This makes tcpWER particularly informative when a system produces approximately correct words and speech activity but places those words in temporally inconsistent segments.

4. Relation to other long-form WER variants

Within the same framework, tcpWER belongs to the speaker-attributed family of long-form WERs. ORC-WER and MIMO-WER are speaker-agnostic alternatives, whereas DI-cpWER is a diarization-invariant diagnostic variant of cpWER. The time constraint used in tcpWER is also extended to these related metrics (Neumann et al., 4 Aug 2025).

Metric	Speaker handling	Distinctive property
cpWER	Speaker-attributed	Best speaker permutation; no time constraint
tcpWER	Speaker-attributed	cpWER with $\rv$7
ORC-WER	Speaker-agnostic	Assigns reference utterances to output streams; preserves global utterance order
tcORC-WER	Speaker-agnostic	ORC-WER with $\rv$8
MIMO-WER	Speaker-agnostic	Allows changing the global order; preserves order within each speaker; requires a globally consistent partial order
tcMIMO-WER	Speaker-agnostic	MIMO-WER with $\rv$9
DI-cpWER	Diarization-invariant analysis	Optimizes hypothesis labels to minimize WER
DI-tcpWER	Diarization-invariant analysis	DI-cpWER with $\hv$0

ORC-WER assigns reference utterances to output streams, preserves the global utterance order, and imposes utterance-consistency, meaning that each utterance must go wholly to one stream. MIMO-WER relaxes ORC-WER further: it allows changing the global order, preserves the order within each speaker, and requires a globally consistent partial order. tcpWER differs from both because it is not a speaker-agnostic relaxation of ordering constraints; instead, it keeps the cpWER speaker-permutation framework and constrains alignment by timestamps.

DI-cpWER serves a different role. It fixes the hypothesis speaker labels to minimize WER, thereby estimating the WER as if the hypothesis speaker labels were correct. Its stated purpose is analysis of the impact of speaker confusions, not system ranking. DI-tcpWER applies the same idea under the time constraint. A common misconception is to treat DI-cpWER or DI-tcpWER as interchangeable with tcpWER for benchmarking; the formulation explicitly distinguishes analytical use from ranking use.

5. Collar semantics, timestamp requirements, and computational behavior

The collar in tcpWER is analogous in name, but not in effect, to the collar used in diarization error rate (DER). In DER, a collar means regions around segment boundaries are not scored. In time-constrained WERs, the collar means that words farther apart than the collar cannot be matched as correct or substitution. The limiting behavior is therefore different: DER with collar $\hv$1 tends to zero, whereas tcpWER with collar $\hv$2 tends to cpWER,

$\hv$3

The study also notes that, unlike DER, WER needs word-level timing accuracy for the collar to be meaningful (Neumann et al., 4 Aug 2025).

Because tcpWER depends on word times, the availability and quality of timestamps matter directly. When word timestamps are not directly available, they can be approximated from segment-level timestamps using pseudo word-level timestamps. A practical method mentioned is to divide a segment into word-level subsegments proportional to word character count and to use center points of those pseudo-timestamps on the hypothesis side to prevent gaming the metric. This preserves the applicability of tcpWER when only segment-level timing is available.

The time constraint also affects runtime. The reported computational observation is that the time constraint improves runtime substantially because it prunes impossible alignments. It is enough to compute a bounded band of the Levenshtein matrix where $\hv$4. Under the assumption that only a limited number of words overlap in time, the computation becomes linear in the number of words in the reference and hypothesis. In this respect, tcpWER is presented not only as more plausible for long recordings but also as often much faster than the non-time-constrained variant.

6. Empirical behavior and practical use

The reported examples illustrate why tcpWER was proposed. One timeline example shows cpWER matching many words as substitutions across large temporal distances even though the transcript is mostly correct in place; the interpretation given is that cpWER can be fooled by a far-away substitution if it reduces total edit cost, whereas tcpWER prevents this and yields a more plausible alignment. Another example concerns a system with roughly correct speech activity and word content but with words placed in wrong segments: cpWER is lower because it can match words across distant times, while tcpWER is higher because those time-inconsistent alignments are penalized (Neumann et al., 4 Aug 2025).

The same study reports a scatter plot comparison over many examples in which cpWER is a lower bound on tcpWER, most differences are under 10 percentage points, and large gaps indicate severe temporal mismatch. This suggests that tcpWER is especially informative when temporal placement, not merely lexical identity, is central to the evaluation target.

The practical guidance given is correspondingly task-specific. tcpWER is appropriate when evaluating systems that produce diarization plus transcription, when word timestamps are available or can be estimated, when the placement of words at the correct times matters, and when implausibly good scores from distant substitutions should be avoided. Non-time-constrained cpWER remains preferable when timestamps are unavailable or unreliable, or when the task is speaker-attributed transcript scoring without temporal concern. For speaker-agnostic evaluation with temporal plausibility, tcORC-WER and tcMIMO-WER are the relevant extensions. For analysis rather than ranking, DI-cpWER and DI-tcpWER are used to estimate how much of cpWER is caused by speaker confusion.

In the broader architecture of long-form multi-talker evaluation, tcpWER therefore occupies a specific niche: it preserves the speaker-permutation logic of cpWER while excluding alignments that are lexically economical but temporally implausible. Its defining contribution is not a new error taxonomy, but a restriction on admissible alignments that better reflects human judgment in long recordings.