SlideASR-Bench: Slide-Aware ASR Benchmark

• The paper introduces SlideASR-Bench, a benchmark that fuses synchronized slide and speech data to improve transcription of domain-specific entities using metrics like WER, NE-WER, and NE-FNR.
• SlideASR-Bench is a multimodal evaluation framework where the synthetic component (SlideASR-S) precisely aligns slide text with speech to support controlled named-entity assessment.
• The real-world component (SlideASR-R) tests model robustness with low-resolution slides and authentic academic content, highlighting challenges in end-to-end slide-aware transcription.

Searching arXiv for the benchmark and closely related source corpus papers to ground the article in current paper metadata. SlideASR-Bench is an entity-rich benchmark for the SlideASR task, introduced in "Look before Transcription: End-to-End SlideASR with Visually-Anchored Policy Optimization" (Hu et al., 8 Oct 2025). It is designed for automatic speech recognition settings in which domain-specific terminology is present both in speech and in synchronized presentation slides. The benchmark combines a synthetic bilingual dataset, SlideASR-S, with a real-world evaluation set, SlideASR-R, and evaluates model outputs with Word Error Rate (WER), Named-Entity WER (NE-WER), and Named-Entity False-Negative Ratio (NE-FNR). Its immediate research setting is slide-aware ASR for academic and technical content, where pipeline methods are described as complex and underperforming, and where omni-modal LLMs may degenerate into simple optical character recognition systems (Hu et al., 8 Oct 2025).

1. Research context and antecedents

SlideASR-Bench emerged from a line of work on multimodal ASR that uses non-acoustic side information to improve recognition of specialized terminology. In "SlideSpeech: A Large-Scale Slide-Enriched Audio-Visual Corpus" (Wang et al., 2023), the slide modality was operationalized as synchronized visual context extracted from conference presentations. That corpus contains 1,705 videos and 1,000+ hours, with 473 hours of speech in the $L95$ subset, and it introduced baseline contextual-ASR methods that used OCR-derived slide text and keyword extraction to improve recognition, with gains that were described as modest but consistent, especially on rare/proper nouns.

SlideASR-Bench inherits the basic premise that slides contain useful domain-specific textual signals, but it changes the unit of analysis. Rather than centering on corpus construction and contextual biasing over long-form online presentations, it centers on entity-rich benchmark design and explicit evaluation of named-entity recovery (Hu et al., 8 Oct 2025). This suggests a shift from slide text as auxiliary biasing context toward slide-conditioned end-to-end transcription as a primary benchmarked task.

The benchmark also reflects a methodological critique stated in its companion paper: existing pipeline methods for SlideASR tend to be complex and underperform, while end-to-end OLLM-based systems frequently fail by collapsing into OCR behavior. SlideASR-Bench is therefore not merely a dataset collection; it is a benchmark specification intended to expose whether a model actually fuses speech and visual slide content, especially for domain-specific entities (Hu et al., 8 Oct 2025).

2. Synthetic component: SlideASR-S

SlideASR-S is the synthetic portion of SlideASR-Bench. It is built on top of ContextASR-Bench, which provides colloquial utterances rich in named entities across multiple domains, including medicine, culture, and ecology (Hu et al., 8 Oct 2025). Its stated goal is to create slides whose visual text aligns perfectly with the spoken entities.

The generation process proceeds in three stages. First, from each ContextASR-Bench sample, the system extracts the domain label $L_{\mathrm{domain}}$ and the entity list

$E = \{e_1,\dots,e_k\}.$

Second, a large LLM, Qwen2.5-14B-Instruct, is prompted to generate slide-style textual content:

$E = \{e_1,\dots,e_k\}.$9

The specified output format is a title plus a paragraph containing $E$. Third, Matplotlib is used to typeset the title and paragraph into an image $I$, yielding the slide visual used in the benchmark (Hu et al., 8 Oct 2025).

SlideASR-S includes both English and Chinese samples. Its reported composition is as follows:

Split	Samples / Duration	Entity Mentions / AvgEnt
SlideASR-S (train)	6,413 / 67.3 h	44,240 / 6.90
SlideASR-S (test)	2,054 / 18.5 h	13,895 / 6.76
SlideASR-R (real)	60 / 0.35 h	200 / 3.33

For SlideASR-S specifically, the training split contains 6,413 samples, 44,240 entity mentions, and 67.3 hours, while the test split contains 2,054 samples, 13,895 entity mentions, and 18.5 hours (Hu et al., 8 Oct 2025). The per-sample entity density is approximately $44{,}240/6{,}413 \approx 6.90$ for training and $13{,}895/2{,}054 \approx 6.76$ for test. The vocabulary size is not explicitly reported. The entity types are domain-specific named entities inherited from the upstream NER seed data.

A key design property is exact cross-modal alignment at the entity level: the speech and slide text share exactly the same entity sets. This is one of the benchmark's central controlled conditions and is important for disentangling acoustic difficulty from multimodal grounding effects.

3. Real-world component: SlideASR-R

SlideASR-R is the real-world evaluation set within SlideASR-Bench. It consists of 60 clips culled from publicly-available academic report videos, spanning Chemistry, Medicine, Biology, and Artificial Intelligence (Hu et al., 8 Oct 2025). For each clip, the synchronized slide image and accompanying speech audio are extracted.

The annotation protocol is fully manual. Each clip is labeled with the ground-truth transcript and with the set of domain-specific entities that are both visible on the slide and spoken in the audio. The resulting dataset contains 60 samples, 200 entity mentions, and 0.35 hours of audio, corresponding to a per-sample entity density of approximately $200/60 \approx 3.33$ entities per sample (Hu et al., 8 Oct 2025).

Functionally, SlideASR-R is intended as a challenging robustness evaluation set rather than a large-scale training resource. The paper characterizes it as small but highly challenging; its low-resolution slides and specialized terminology expose OCR failures and test a model's true multimodal fusion ability (Hu et al., 8 Oct 2025). This makes SlideASR-R structurally different from SlideASR-S: the synthetic set maximizes controlled alignment and scale, whereas the real-world set introduces realistic visual degradation and domain-specific ambiguity.

The real-world subset also narrows the domain distribution relative to the synthetic portion. SlideASR-S inherits broader domain coverage from the upstream seed data, while SlideASR-R focuses on academic scientific presentation content. A plausible implication is that the benchmark jointly measures in-domain multimodal grounding and synthetic-to-real transfer.

4. Evaluation protocol and metric design

SlideASR-Bench adopts three metrics defined over a model output $\hat Y$ and reference $Y$ (Hu et al., 8 Oct 2025). The first is the standard Word Error Rate:

$L_{\mathrm{domain}}$0

where $L_{\mathrm{domain}}$1 is the number of substitutions, $L_{\mathrm{domain}}$2 the number of deletions, $L_{\mathrm{domain}}$3 the number of insertions, and $L_{\mathrm{domain}}$4 the total number of words, or Chinese characters, in the reference.

The second metric is Named-Entity WER:

$L_{\mathrm{domain}}$5

computed after restricting both reference and hypothesis to the subsequences corresponding to the ground-truth named entities. Here $L_{\mathrm{domain}}$6 is the total entity-word count in the reference, and $L_{\mathrm{domain}}$7, $L_{\mathrm{domain}}$8, and $L_{\mathrm{domain}}$9 are the corresponding error counts (Hu et al., 8 Oct 2025).

The third metric is Named-Entity False-Negative Ratio. It is defined as one minus the ratio between the number of distinct ground-truth entities correctly recognized under fuzzy matching and the total ground-truth entity count (Hu et al., 8 Oct 2025). Unlike NE-WER, which measures token-level entity transcription errors, NE-FNR emphasizes entity recovery at the level of distinct named entities.

Taken together, these metrics make the benchmark explicitly entity-sensitive. Standard WER measures overall transcription fidelity, NE-WER isolates error behavior on the benchmark's central lexical targets, and NE-FNR emphasizes recall-oriented entity capture. This metric triad is particularly suited to SlideASR, where the main performance bottleneck is often not generic speech recognition but correct transcription of visually available domain terminology.

5. Association with VAPO and end-to-end SlideASR

SlideASR-Bench was introduced together with Visually-Anchored Policy Optimization, a post-training method for end-to-end SlideASR (Hu et al., 8 Oct 2025). VAPO enforces a structured "Look before Transcription" procedure using a > <answer> format. In this procedure, the model first performs OCR on the slide content within the think step, then generates the transcription by referencing the recognized visual information in the answer step.

The reinforcement-learning objective scores each output with four rewards: format compliance, OCR accuracy, ASR quality, and visual anchoring consistency, with default coefficients $E = \{e_1,\dots,e_k\}.$0. The format reward is binary:

$E = \{e_1,\dots,e_k\}.$1

The OCR reward uses the WER between the text in the <think> block, $E = \{e_1,\dots,e_k\}.$2, and the true slide text, $E = \{e_1,\dots,e_k\}.$3:

$E = \{e_1,\dots,e_k\}.$4

The ASR reward uses the WER between the text in the <answer> block, $E = \{e_1,\dots,e_k\}.$5, and the ground-truth transcript, $E = \{e_1,\dots,e_k\}.$6:

$E = \{e_1,\dots,e_k\}.$7

The visual-anchoring reward extracts the set of entities correctly recognized in <think> and computes the $E = \{e_1,\dots,e_k\}.$8 score of those entities in the <answer> text (Hu et al., 8 Oct 2025). This reward formulation makes the benchmark operationally relevant not only for evaluation but also for post-training regimes that explicitly control multimodal reasoning structure.

In this configuration, SlideASR-S serves as a large, diverse, entity-rich synthetic corpus for training and testing in both English and Chinese, while SlideASR-R provides the real-world stress test. The benchmark is therefore coupled to an end-to-end learning paradigm rather than being limited to pipeline contextual biasing.

6. Strengths, limitations, and interpretation

Several strengths of SlideASR-Bench are explicit in its design summary (Hu et al., 8 Oct 2025). The first is entity density: each synthetic slide packs approximately seven entities, while real slides contain approximately three entities. The second is bimodal alignment: speech and slide text share exactly the same entity sets. The third is domain breadth: the synthetic side covers domains ranging from academia, including chemistry, biology, medicine, and AI, to culture and ecology.

These properties make the benchmark especially suitable for evaluating recognition of domain-specific terms. Because the visual modality is entity-aligned by construction in SlideASR-S, failures on NE-WER and NE-FNR can be attributed more directly to cross-modal utilization problems than in unconstrained multimodal corpora. In the real-world set, by contrast, low-resolution slides and specialized terminology intentionally reintroduce practical OCR and grounding difficulties.

The benchmark also foregrounds three remaining challenges. The first is OCR reliability on low-quality slides. The second is the domain gap between synthetic and real slide styles. The third is the inference-latency trade-off created by enforcing explicit <think>/<answer> chains (Hu et al., 8 Oct 2025). These challenges are not incidental; they define much of the difficulty of SlideASR as a research problem.

A common misconception is to treat SlideASR-Bench as an OCR benchmark with audio attached. The associated paper argues against that view in two ways: it identifies degeneration into simple OCR systems as a failure mode of OLLMs, and it positions SlideASR-R as a test of true multimodal fusion ability rather than text extraction alone (Hu et al., 8 Oct 2025). The benchmark's metric design reinforces that distinction by measuring not just transcription error, but named-entity transcription and entity recovery.

In the broader trajectory from "SlideSpeech" (Wang et al., 2023) to end-to-end SlideASR, SlideASR-Bench represents a more task-specific and entity-centered evaluation framework. SlideSpeech established that synchronized slides can improve contextual ASR through OCR-derived biasing; SlideASR-Bench reformulates the problem as benchmarked multimodal transcription with explicit named-entity criteria and a paired synthetic-real evaluation regime. This suggests an evolution from auxiliary-context ASR toward fully slide-aware ASR systems whose success depends on robust visual anchoring, not merely on external keyword injection.