SEGALE: Long-Form Translation Evaluation

• SEGALE is an evaluation framework that extends sentence-level metrics to long-document translation by segmenting and aligning continuous texts.
• It employs multilingual sentence embeddings and an adaptive skip-penalty search to achieve many-to-many, monotonic alignments, mitigating under- and over-translation issues.
• In long-form speech-to-speech translation, SEGALE enables both quality and latency analysis through target transcript alignment, while its use in LEGEND-200 highlights acronym ambiguity.

SEGALE most prominently denotes an evaluation scheme for long-document machine translation that extends sentence-level metrics to continuous texts by SEGment, ALign, and Evaluate operations, thereby enabling document-level scoring under boundary mismatch, under-translation, and over-translation (Wang et al., 21 Sep 2025). In subsequent work on long-form simultaneous speech-to-speech translation, SEGALE functions as a sentence-embedding-based aligner that maps target speech-derived text to source sentences so that latency and quality can be computed over aligned spans (Xue et al., 13 Jun 2026). The term also appears, in a separate and unrelated usage, as an informal label for the LEGEND-200 signal readout electronics, although that acronym is not explicitly defined in the corresponding LEGEND paper (Willers, 2019).

1. Definition and scope

In its machine-translation usage, SEGALE is designed for settings in which both source and translation are treated as continuous text without guaranteed sentence-boundary agreement. Its core objective is to recover a monotonic, contiguous, many-to-many alignment between automatically segmented source and target spans, permit null alignments on either side, and then apply existing sentence-level metrics such as COMET, MetricX, BLEU, chrF, or BERTScore to the aligned blocks (Wang et al., 21 Sep 2025). This design directly addresses three constraints emphasized in the literature: dataset annotations are often sentence-level, automatic metrics have token limits, and long-context translations generated with document-level prompts may exhibit under- and over-translation as well as arbitrary sentence-boundary variation (Wang et al., 21 Sep 2025).

The formal setting introduces a source document $D_{\mathrm{src}}$ and a translation $D_{\mathrm{tgt}}$, sentence segmentation functions that produce sequences $S=\{s_1,\dots,s_N\}$ and $T=\{t_1,\dots,t_M\}$, and an alignment objective over contiguous spans on both sides (Wang et al., 21 Sep 2025). The aligned spans are then concatenated locally before metric application. This local-span strategy is central because it sidesteps the global token-count restrictions of metrics such as COMET and MetricX while preserving document-level coverage (Wang et al., 21 Sep 2025).

A distinct but closely related usage arises in long-form simultaneous speech-to-speech translation. There, source speech, pre-segmented source transcripts, reference translations, and generated target speech are combined in a pipeline that performs target-side ASR, forced alignment, sentence boundary detection, and SEGALE-based alignment before computing group-level latency and quality scores (Xue et al., 13 Jun 2026). In that setting, SEGALE is not itself the entire evaluator but the alignment backbone that makes sentence- or span-level aggregation practical over long continuous speech (Xue et al., 13 Jun 2026).

This suggests that SEGALE is best understood not as a single metric, but as an alignment-and-normalization layer that makes pre-existing metrics usable in long-form evaluation regimes.

2. Segmentation and alignment methodology

SEGALE begins with automatic sentence segmentation. The documented implementations use off-the-shelf segmenters: ersatz, described as a language-agnostic rule-based segmenter, and spaCy, with language-specific pipelines including en_core_web_sm, de_core_news_sm, es_core_news_sm, ja_core_news_sm, and zh_core_web_sm (Wang et al., 21 Sep 2025). In the reported experiments, spaCy produced finer-grained segmentation than ersatz, but downstream evaluation correlation was similar; ersatz was the default choice because its segmentation granularity was closer to WMT test sets and its null-alignment behavior was slightly better aligned with gold annotations (Wang et al., 21 Sep 2025).

Alignment is built on Vecalign, modified for SEGALE with many-to-many monotonic alignment over contiguous spans, multilingual sentence embeddings for block similarity, explicit skip costs for null alignments, and an adaptive search over the skip threshold $B_{\mathrm{skip}}$ (Wang et al., 21 Sep 2025). The aim is to avoid two complementary failure modes: a skip penalty that is too high forces semantically unrelated blocks to align, whereas a skip penalty that is too low yields excessive null alignments (Wang et al., 21 Sep 2025). The search therefore starts at $B_{\mathrm{skip}}=0.2$ and decreases in steps of $0.005$, while monitoring average alignment cost and null-alignment ratio. The reported stopping heuristics are: stop when average alignment cost $<0.3$, stop when the null-alignment ratio exceeds $0.15$, and early-stop if average cost increases or exceeds $0.7$; the selected solution is the last safe alignment before over-deletion is triggered (Wang et al., 21 Sep 2025).

The conceptual dynamic program can be written as

$D_{\mathrm{tgt}}$0

with monotonicity and contiguous-block extensions for many-to-many matches (Wang et al., 21 Sep 2025). In practice, Vecalign implements this efficiently through bounded block search rather than exhaustive quadratic dynamic programming (Wang et al., 21 Sep 2025).

For document-level MT evaluation, SEGALE uses multilingual sentence embeddings, with LASER and BGE-M3 as baselines and a fine-tuned BGE-M3 variant as the preferred encoder (Wang et al., 21 Sep 2025). That fine-tuned model was trained on 130,436 synthetic triplets from News Commentary v18.1 for $D_{\mathrm{tgt}}$1 using InfoNCE over adjacent-sentence concatenations and hard negatives that either drop a target sentence or substitute a nearby non-parallel sentence (Wang et al., 21 Sep 2025). The overlap size for candidate span alignments is set to $D_{\mathrm{tgt}}$2, limiting candidate blocks by $D_{\mathrm{tgt}}$3 (Wang et al., 21 Sep 2025).

In long-form simultaneous speech-to-speech translation, the alignment principle is similar but the inputs differ. Generated target speech is first transcribed and force-aligned into tokens with timestamps, then split into target sentences with spaCy, and finally aligned to source sentence spans using SEGALE, again allowing one-to-one, one-to-many, many-to-one, many-to-many, and null alignments (Xue et al., 13 Jun 2026). The reported cost uses cosine similarity over sentence embeddings,

$D_{\mathrm{tgt}}$4

with span cost

$D_{\mathrm{tgt}}$5

and an adaptive skip-penalty search analogous to the MT setting (Xue et al., 13 Jun 2026).

3. Extension of sentence-level metrics to long documents

Once an alignment is obtained, SEGALE converts any sentence-level metric into a document-level evaluator by scoring aligned spans after concatenation. For an aligned source span $D_{\mathrm{tgt}}$6 and target span $D_{\mathrm{tgt}}$7, the block score is

$D_{\mathrm{tgt}}$8

where $D_{\mathrm{tgt}}$9 may be COMET, MetricX, BLEU, chrF, BERTScore, or another sentence-level metric (Wang et al., 21 Sep 2025). Null alignments explicitly encode missing or hallucinated content. Under-translation corresponds to unmatched source spans, and over-translation to unmatched target spans; both receive a fixed worst-case penalty $S=\{s_1,\dots,s_N\}$0 (Wang et al., 21 Sep 2025). The documented values are $S=\{s_1,\dots,s_N\}$1 for COMET and $S=\{s_1,\dots,s_N\}$2 for MetricX (Wang et al., 21 Sep 2025).

The document-level SEGALE score is then

$S=\{s_1,\dots,s_N\}$3

where $S=\{s_1,\dots,s_N\}$4 is the set of aligned span pairs and $S=\{s_1,\dots,s_N\}$5 the set of null alignments (Wang et al., 21 Sep 2025). SEGALE also reports the null alignment ratio

$S=\{s_1,\dots,s_N\}$6

which acts as an explicit diagnostic for under- and over-translation (Wang et al., 21 Sep 2025).

This null-aware formulation is essential because direct document concatenation was found inadequate: a preliminary study using MetricX-24 and MetricX-24-QE showed limited sensitivity to injected under- and over-translation even when token limits were not violated (Wang et al., 21 Sep 2025). By contrast, SEGALE converts these failure modes into explicit alignment structure, after which any local metric can penalize them through null spans (Wang et al., 21 Sep 2025).

In the speech-to-speech setting, the same alignment outputs support both quality and latency. For each aligned group $S=\{s_1,\dots,s_N\}$7, sentence-level quality is computed as

$S=\{s_1,\dots,s_N\}$8

with $S=\{s_1,\dots,s_N\}$9 instantiated as xCOMET-XL in the reported experiments, while null groups receive the metric minimum $T=\{t_1,\dots,t_M\}$0 (Xue et al., 13 Jun 2026). This makes SEGALE compatible with both reference-based and quality-estimation-style scorers, provided they can operate on aligned local spans (Wang et al., 21 Sep 2025, Xue et al., 13 Jun 2026).

4. Empirical evaluation on long-form machine translation

SEGALE was meta-evaluated primarily on the WMT 2024 Metrics Shared Task for language pairs en-de, en-es, and ja-zh, using document-level Kendall’s $T=\{t_1,\dots,t_M\}$1 against document-aggregated MQM z-scores (Wang et al., 21 Sep 2025). The evaluation included the original data and three synthetic anomaly settings: over-translate, created by randomly removing 10% of sentences from source and reference; under-translate, created by removing 10% from system translations; and flex-boundary, created by merging 10% of adjacent source sentences using GPT-4o while preserving semantics with a BLEURT $T=\{t_1,\dots,t_M\}$2 filter (Wang et al., 21 Sep 2025).

On the original setting, SEGALE achieved correlations close to gold sentence alignments. Averaged over the three language pairs, the reported scores were: COMET $T=\{t_1,\dots,t_M\}$3, COMET-QE $T=\{t_1,\dots,t_M\}$4, MetricX $T=\{t_1,\dots,t_M\}$5, and MetricX-QE $T=\{t_1,\dots,t_M\}$6, compared with gold-alignment values of COMET $T=\{t_1,\dots,t_M\}$7, COMET-QE $T=\{t_1,\dots,t_M\}$8, MetricX $T=\{t_1,\dots,t_M\}$9, and MetricX-QE $B_{\mathrm{skip}}$0 (Wang et al., 21 Sep 2025). The null-alignment ratio for SEGALE in this setting was $B_{\mathrm{skip}}$1, whereas gold had $B_{\mathrm{skip}}$2 (Wang et al., 21 Sep 2025).

Under explicit perturbation, SEGALE retained substantially stronger behavior than mwerSegmenter. In the over-translate setting, SEGALE reported NA $B_{\mathrm{skip}}$3, close to the gold value of $B_{\mathrm{skip}}$4, while mwerSegmenter remained at $B_{\mathrm{skip}}$5 and therefore failed to flag hallucinated content as null structure (Wang et al., 21 Sep 2025). In the under-translate setting, SEGALE reported NA $B_{\mathrm{skip}}$6 versus gold $B_{\mathrm{skip}}$7, whereas mwerSegmenter reported only $B_{\mathrm{skip}}$8 and exhibited a marked drop in correlation, including COMET $B_{\mathrm{skip}}$9 and MetricX $B_{\mathrm{skip}}=0.2$0 (Wang et al., 21 Sep 2025). In the flex-boundary setting, SEGALE remained close to gold with minimal NA inflation, showing that many-to-many alignment absorbs moderate sentence-boundary disagreement without major degradation (Wang et al., 21 Sep 2025).

The ablation results further identify the fine-tuned BGE-M3 encoder as consistently better than LASER and untuned BGE-M3 in both correlation and NA-ratio behavior, while the segmenter choice between ersatz and spaCy had only minor impact (Wang et al., 21 Sep 2025). A plausible implication is that alignment quality is limited more by cross-lingual semantic matching under granularity mismatch than by the initial sentence splitter, provided the splitter does not produce pathological boundaries.

SEGALE was also applied to book-length translation in the WMT 2024 discourse-level literary translation setting for zh→en. Using segment sizes of 1k, 2k, 4k, and 8k tokens, it revealed that many open-weight LLMs degrade sharply at 4k–8k context, with document-level MetricX dropping and null-alignment ratio rising markedly (Wang et al., 21 Sep 2025). The paper highlights EuroLLM as summarizing instead of translating at 4k, whereas Qwen2.5-72B-Instruct remained comparatively robust, with stable or slightly improved quality up to 4k (Wang et al., 21 Sep 2025).

5. Adaptation to long-form simultaneous speech-to-speech translation

In long-form SimulS2ST, SEGALE is embedded in a broader evaluation pipeline that operates on generated target speech rather than exposed target text (Xue et al., 13 Jun 2026). The pipeline begins with target-side ASR and forced alignment. Specifically, generated target speech $B_{\mathrm{skip}}=0.2$1 is chunked into consecutive 180-second segments, transcribed with Qwen3-ASR-1.7B, and token-aligned with Qwen3-ForcedAligner-0.6B to obtain target text $B_{\mathrm{skip}}=0.2$2 and token-level end timestamps $B_{\mathrm{skip}}=0.2$3 (Xue et al., 13 Jun 2026). Chunk-relative timestamps are then mapped back to the global timeline and concatenated (Xue et al., 13 Jun 2026). Sentence boundary detection is performed with spaCy, and SEGALE aligns the recovered target sentences to source sentences and reference translations (Xue et al., 13 Jun 2026).

This alignment supports YAAL-style latency computation over sentence groups. For an aligned group $B_{\mathrm{skip}}=0.2$4 spanning source times $B_{\mathrm{skip}}=0.2$5 to $B_{\mathrm{skip}}=0.2$6 and target token end times $B_{\mathrm{skip}}=0.2$7, the ideal schedule is

$B_{\mathrm{skip}}=0.2$8

and the group latency is

$B_{\mathrm{skip}}=0.2$9

Groups with no source-target content are excluded from latency, and target tokens generated after the end of the full source stream are also excluded, following LongYAAL-style practice (Xue et al., 13 Jun 2026).

The final reported system-level latency and quality are simple averages over aligned groups (Xue et al., 13 Jun 2026). On ACL 60/60 dev, the reported latency/xCOMET-XL pairs were: SeamlessStreaming En→De $0.005$0, En→Ja $0.005$1, En→Zh $0.005$2; and Seed LiveInterpret 2.0 En→De $0.005$3, En→Ja $0.005$4, En→Zh $0.005$5 (Xue et al., 13 Jun 2026). On Audio-NTREX-L, the reported values were: SeamlessStreaming Fr→En $0.005$6, De→En $0.005$7, Pt→En $0.005$8, Es→En $0.005$9; Seed LiveInterpret 2.0 Fr→En $<0.3$0, De→En $<0.3$1, Pt→En $<0.3$2, Es→En $<0.3$3; and Hibiki-Zero Fr→En $<0.3$4, De→En $<0.3$5, Pt→En $<0.3$6, Es→En $<0.3$7 (Xue et al., 13 Jun 2026).

A key empirical result is that SEGALE achieved 90.9% segmentation accuracy versus SoftSegmenter’s 79.1% on ACL 60/60 En→Zh, which the paper attributes to explicit boundary modeling and robust semantic alignment (Xue et al., 13 Jun 2026). The same experiments also revealed pronounced latency accumulation on long talks: ending offsets of aligned sentences tend to grow over time, with especially strong accumulation for En→Ja, where target speech is longer than source and offsets can exceed 200 seconds (Xue et al., 13 Jun 2026). This suggests that long-form simultaneous speech evaluation requires alignment methods that remain robust under drift, missing content, and variable sentence duration rather than assuming pre-segmented utterances.

6. Limitations, interpretive caveats, and unrelated usage in LEGEND-200

SEGALE inherits several limitations from its components. In machine translation, the underlying metrics remain sentence-level and do not themselves model discourse; SEGALE extends them to long documents by alignment and aggregation rather than by introducing new discourse-sensitive semantics (Wang et al., 21 Sep 2025). Its adaptive skip-penalty search is tuned to current metric behavior and to Vecalign’s explicit null-penalty framework, so the documented mechanism is not readily portable to aligners without comparable skip controls (Wang et al., 21 Sep 2025). The embedding fine-tuning is described as proof-of-concept on limited data and languages, implying that broader language coverage would likely improve robustness (Wang et al., 21 Sep 2025).

In speech-to-speech translation, the evaluation depends on target-side ASR and forced alignment accuracy. The reported target-side ASR comparison on ACL 60/60 dev gave Qwen3-ASR-1.7B versus WhisperX WER/CER values of De: $<0.3$8 vs $<0.3$9, Ja: $0.15$0 vs $0.15$1, and Zh: $0.15$2 vs $0.15$3 (Xue et al., 13 Jun 2026). The paper notes that the high Japanese error rates suggest quality problems in the synthesized Japanese speech rather than a difference driven by ASR choice (Xue et al., 13 Jun 2026). This indicates that SEGALE can expose system-level pathologies, but it cannot by itself separate synthesis errors from translation-policy errors.

The term “SEGALE” also appears in an unrelated nuclear-instrumentation context. The paper “Signal Readout Electronics for LEGEND-200” describes the LEGEND-200 signal readout electronics as a low-mass, low-background, two-stage charge-sensitive amplifier operated inside liquid argon, with a first-stage Low Mass Front End near each HPGe detector and a second-stage differential amplifier located approximately $0.15$4–$0.15$5 cm away (Willers, 2019). The data explicitly note that the acronym SEGALE is not explicitly used or defined in that paper, even though it may be used informally to refer to the readout electronics (Willers, 2019). Because this usage is separate from the MT and SimulS2ST literature, the two meanings should not be conflated.

Overall, the dominant technical meaning of SEGALE in current arXiv literature is the long-form alignment-and-evaluation framework for machine translation and its reuse as the sentence-embedding-based aligner in long-form simultaneous speech-to-speech translation (Wang et al., 21 Sep 2025, Xue et al., 13 Jun 2026).