Serialized Output Prompts (SOP)

• Serialized Output Prompts (SOP) are techniques that linearize complex outputs into ordered representations for clear conditioning and control.
• They are applied in multi-talker ASR and streaming ASR/ST to manage talker order and language switches, yielding measurable improvements in WER and latency.
• In prompt engineering, SOP enforces deterministic output contracts through explicit schema tokens, ensuring reliable format compliance and reducing error modes.

Serialized Output Prompts (SOP) denotes a family of prompting and serialization mechanisms that linearize complex outputs into an ordered representation for conditioning or control. In recent arXiv literature, the term is used in three closely related but technically distinct ways. In LLM-based multi-talker automatic speech recognition (MT-ASR), SOP is a structured prompt extracted from serialized CTC outputs and supplied to an LLM decoder to represent overlapping talkers in first-speaking-first-out order (Shi et al., 1 Sep 2025). In joint streaming ASR and speech translation (ST), SOP is realized through explicit in-vocabulary task or language tags embedded in a single transducer output sequence, such as #ASR#, #ST#, or <st_l> (Papi et al., 2023); (Papi et al., 2023). In prompt-engineering systems, SOP denotes prompt designs that require deterministic, machine-parseable outputs such as JSON, XML, CSV, or strict Markdown tables, with output contracts elevated to first-class prompt components (Zhang et al., 19 Aug 2025); (Dovdon, 15 Jun 2026).

1. Terminological scope and common structure

Across these papers, SOP is not a single standardized algorithm. The term is used for serialization strategies that make output structure explicit, whether the structure is talker order, task order, language order, or schema order. A common misconception is that SOP refers only to text formatting. In the MT-ASR literature, SOP is derived from acoustic processing and is part of the decoder input; in streaming transducer work, it is part of the target token stream itself; in prompt-engineering systems, it is an explicit contract for downstream parsers and validators.

Research setting	Serialized unit	Prompt mechanism
LLM-based MT-ASR	Talker-wise CTC outputs ordered by speaking start time	Concatenated SOP embeddings supplied with speech and text inputs
Joint streaming ASR/ST	ASR and ST tokens in one target sequence	In-vocabulary tags such as #ASR#, #ST#, <st_es>
Structured LLM output	Keys, rows, fields, or artifact names	Output contracts, serializers, and schema-constrained prompt directives

This suggests that SOP is best understood as an interface between latent model computation and an explicitly serialized output policy. The policy may encode temporal order, speaker order, task switching, or schema conformance, depending on the application.

2. SOP in LLM-based multi-talker speech recognition

In "Serialized Output Prompting for LLM-based Multi-Talker Speech Recognition" (Shi et al., 1 Sep 2025), SOP is defined as the greedy-decoded output of serialized CTC branches corresponding to individual talkers ordered by speaking start time. The architecture comprises a speech encoder, three CNN downsampling layers, a Separator, serialized CTC layers, a projector, and an LLM decoder adapted with LoRA. The speech path is defined as

$$H_e = \text{Enc}(y), \qquad H_d = \text{Down}(H_e), \qquad H_p = \text{Projector}(H_d).$$

In the proposed configuration, the second CNN output is fed to the Separator,

$$H^{(2)} = \text{Conv}_2(\text{Conv}_1(H_e)), \qquad H^{(3)} = \text{Conv}_3(H^{(2)}), \qquad H_d = H^{(3)},$$

$$H_{sep} = \text{Separator}(H^{(2)}), \qquad H_{sep}^{s} = \text{ReLU}(\text{Linear}^{s}(\text{LayerNorm}(H_{sep}))).$$

Each talker-specific branch is greedily decoded,

$$\bar{C}^s = \text{Greedy}(H_{sep}^{s}), \qquad \bar{C}_{sop} = \text{Concat}(\bar{C}^1, \bar{C}^2, ..., \bar{C}^s),$$

and the resulting sequence is embedded as

$E_{\text{sop}} = \text{Embedding}(\bar{C}_{sop}).$

The LLM input is then

$$H_{\text{sop}} = [E_{\text{sop}}; H_p ; E_t], \qquad T_e = \text{LLM}(H_{\text{sop}}, \text{LoRA}).$$

The design is motivated by a limitation of prior LLM-based MT-ASR systems: earlier systems either used no prompt or a simple task-definition prompt of the form

$T_e = \text{LLM}([P], H_p, E_t),$

which does not encode overlap structure or talker-wise content. The paper contrasts SOP with Serialized Output Training (SOT), where labels are serialized by speaker start time and speaker changes are marked with ⟨sc⟩, but no serialized auxiliary content is explicitly provided to the LLM. The SOT target example for two talkers is

$$T_{sot} = \{t_1^{1}, \dots, t_1^{N^{1}}, \langle sc \rangle, t_2^{1}, \dots, t_2^{N^{2}}\},$$

with

$$\mathcal{L}_{\text{SOT}} = \text{CE}(T_e, T_{sot}).$$

Training is organized into three stages to avoid negative interference between CTC training and LLM decoding. Stage 1 performs SOT fine-tuning of the speech encoder and LLM-based decoder. Stage 2 jointly optimizes the speech encoder, downsampling CNNs, Separator, and serialized CTC layers with

$$\mathcal{L}_{\text{EncSep}} = \alpha \mathcal{L}_{\text{CTC-EncSep}} + (1 - \alpha)\mathcal{L}_{\text{SOT}},$$

where

$$H^{(2)} = \text{Conv}_2(\text{Conv}_1(H_e)), \qquad H^{(3)} = \text{Conv}_3(H^{(2)}), \qquad H_d = H^{(3)},$$0

Stage 3 freezes the encoder, Separator, and CTC layers and adds another set of LoRA modules to adapt the LLM to SOP-based prompts.

On LibriMix, the paper reports that SOP materially improves WER under both two- and three-talker conditions. For the 1B LLM, the baseline SOT model achieved 11.3 WER on Libri2Mix eval and 39.1 on Libri3Mix eval, while SOP-based MT-ASR achieved 10.5 and 28.5. For the 3B LLM, baseline SOT achieved 9.8 and 31.7, while SOP-based MT-ASR achieved 9.2 and 28.1; clean-condition eval WER was 3.6 on Libri2Mix and 16.5 on Libri3Mix. For the 8B LLM, baseline SOT achieved 14.3 and 47.5, while SOP-based MT-ASR achieved 12.7 and 38.1. The paper states that the 3B model performed best and that the 8B model did not improve further, hypothesized to be due to the limited amount of data (Shi et al., 1 Sep 2025).

The study also identifies important failure modes. Removing mixed speech encoding and feeding only SOP was highly unstable: for the 3B model, Libri2Mix eval rose to 85.1 and Libri3Mix eval to 154.0. Stage 2 training initially degraded performance because CTC ⟨blank⟩-induced sparsity interacted poorly with multi-layer CNN downsampling. Regions with high overlap tended to result in more prediction errors, although the serialized CTC output still provided complete and well-aligned speech content for different talkers. The paper therefore treats SOP as an auxiliary prompt that must complement, rather than replace, acoustic evidence.

3. SOP in serialized streaming ASR and speech translation

In streaming ASR and ST, SOP is implemented through explicit tags that serialize multiple outputs into one token stream. "Token-Level Serialized Output Training for Joint Streaming ASR and ST Leveraging Textual Alignments" (Papi et al., 2023) introduces a streaming Transformer-Transducer with a single decoder that jointly generates ASR and ST. Two vocabulary items are added, <asr> and <st>, instantiated as #ASR# and #ST#. These are ordinary tokens trained with the same RNN-T loss as other subwords. The target sequence is an interleaving of ASR and ST chunks,

$$H^{(2)} = \text{Conv}_2(\text{Conv}_1(H_e)), \qquad H^{(3)} = \text{Conv}_3(H^{(2)}), \qquad H_d = H^{(3)},$$1

and the model is optimized with

$$H^{(2)} = \text{Conv}_2(\text{Conv}_1(H_e)), \qquad H^{(3)} = \text{Conv}_3(H^{(2)}), \qquad H_d = H^{(3)},$$2

where

$$H^{(2)} = \text{Conv}_2(\text{Conv}_1(H_e)), \qquad H^{(3)} = \text{Conv}_3(H^{(2)}), \qquad H_d = H^{(3)},$$3

The paper studies several interleaving policies. INTER 0.0 emits all ASR and then all ST; INTER 1.0 emits all ST and then all ASR; INTER 0.5 alternates word by word; and the proposed INTER ALIGN uses word alignments from awesome-align. The alignment-driven rules explicitly handle unique 1–1 alignment, many-to-one mappings, non-monotonicity, unaligned runs, and exhaustion of one side. On multilingual {de, es, it} \rightarrow en, the paper reports that joint t-SOT with INTER ALIGN achieves the best quality-latency balance, with an average ASR latency of approximately 1.0 s, an ST latency of approximately 1.3 s, and an average improvement of 1.1 WER and 0.4 BLEU over separate multilingual ASR and ST models (Papi et al., 2023).

"Leveraging Timestamp Information for Serialized Joint Streaming Recognition and Translation" (Papi et al., 2023) extends this line of work by using timestamp-informed serialization and by enabling one-to-many joint outputs with a single decoder for the first time. Here, the serialized unit is a tuple (time, tag, word), where the tag may be <asr>, <st_es>, <st_de>, and so forth. The basic INTER [TIME](https://www.emergentmind.com/topics/tomographic-ionized-carbon-mapping-experiment-time) rule concatenates all timestamped words, sorts them by time, inserts a tag whenever the tag changes, and uses the resulting sequence as the RNN-T target. The paper also introduces time-step grouping, with $$H^{(2)} = \text{Conv}_2(\text{Conv}_1(H_e)), \qquad H^{(3)} = \text{Conv}_3(H^{(2)}), \qquad H_d = H^{(3)},$$4, to reduce switch frequency. On the many-to-English setting, INTER TIME with 500 ms grouping reports, for example, it→en WER 21.22 with $$H^{(2)} = \text{Conv}_2(\text{Conv}_1(H_e)), \qquad H^{(3)} = \text{Conv}_3(H^{(2)}), \qquad H_d = H^{(3)},$$5 ms and BLEU 22.05 with $$H^{(2)} = \text{Conv}_2(\text{Conv}_1(H_e)), \qquad H^{(3)} = \text{Conv}_3(H^{(2)}), \qquad H_d = H^{(3)},$$6 ms; es→en WER 19.74 with $$H^{(2)} = \text{Conv}_2(\text{Conv}_1(H_e)), \qquad H^{(3)} = \text{Conv}_3(H^{(2)}), \qquad H_d = H^{(3)},$$7 ms and BLEU 24.09 with $$H^{(2)} = \text{Conv}_2(\text{Conv}_1(H_e)), \qquad H^{(3)} = \text{Conv}_3(H^{(2)}), \qquad H_d = H^{(3)},$$8 ms; and de→en WER 21.17 with $$H^{(2)} = \text{Conv}_2(\text{Conv}_1(H_e)), \qquad H^{(3)} = \text{Conv}_3(H^{(2)}), \qquad H_d = H^{(3)},$$9 ms and BLEU 20.81 with $$H_{sep} = \text{Separator}(H^{(2)}), \qquad H_{sep}^{s} = \text{ReLU}(\text{Linear}^{s}(\text{LayerNorm}(H_{sep}))).$$0 ms. In the English-to-many setting, the joint model with 1000 ms grouping produces ASR and three ST outputs in a single inference step, whereas the cited baseline requires four steps; the paper states that this reduces inference cost 4× while maintaining comparable accuracy with some degradations mitigated by grouping (Papi et al., 2023).

A key technical point in both papers is that the tags are not external control signals injected from outside the model. They are part of the learned target sequence. The decoder therefore learns to emit tag tokens at switching points and then continue with the corresponding stream, making SOP an in-stream control mechanism rather than a separate prompting channel.

4. SOP as schema-constrained prompt engineering

In prompt-engineering research, SOP refers to explicit prompt designs that require deterministic, schema-conformant outputs. "Prompt Orchestration Markup Language" (Zhang et al., 19 Aug 2025) introduces POML, a component-based markup language for prompt authoring. POML uses intention components such as <role>, <task>, <example>, and <output-format>; data components such as <document>, <table>, <img>, <folder>, and <conversation>; a CSS-like <stylesheet> for decoupling content from presentation; and templating constructs such as <let>, for, if/else, and {...} substitution. SOP is realized by making the output contract explicit in the prompt, for example through <output-format syntax="json">, or through region-level serialization controls such as env presentation="serialize" serializer="json". The paper emphasizes failure modes such as format drift, hallucinated or missing keys, column reordering, extraneous commentary, incorrect escaping, and table misalignment. It also specifies SDK- and IDE-assisted repair loops in which the returned output is parsed, validated, and, if necessary, regenerated with the explicit output contract reinserted (Zhang et al., 19 Aug 2025).

"PromptMN: Pseudo Prompting Language" (Dovdon, 15 Jun 2026) advances a related but distinct approach. PromptMN is a pseudo-prompting DSL with typed, %-prefixed directives for roles, goals, requirements, priorities, constraints, plans, inputs, and outputs. SOP is encoded through %out plus %format json, strengthened by directives such as %mustnot, %rule, %showplan, and %trace. The paper formalizes order-independent semantic resolution: directives are interpreted by function rather than source order, and numbered plan steps %1, %2, and so on are assembled in ascending order. It also states a precedence hierarchy in which %mustnot dominates %should and %could, while %newreq overrides earlier requirements. Concrete examples include JSON extraction, prime-checking with %repeat, %if, and %return, and SDLC-oriented generation of spec.json, tests.json, and docs.md as explicit serialized artifacts (Dovdon, 15 Jun 2026).

Both systems treat serialization as a first-class prompt concern rather than a post-processing convenience. POML does so through markup, stylesheets, and validators; PromptMN does so through typed directives and a lightweight execution-oriented syntax. In both cases, the intended output format is part of the prompt’s semantics, not merely a prose instruction appended at the end.

5. Formalization, metrics, and empirical reliability

The formal treatment of SOP varies by domain. In MT-ASR, the central formal object is the serialized prompt derived from greedy CTC decoding, together with the composite loss used to train the Separator and serialized CTC layers (Shi et al., 1 Sep 2025). In streaming ASR/ST, the formal object is the single RNN-T target sequence containing tags and subwords, with latency assessed by Length-Adaptive Average Lagging (LAAL) and quality assessed by WER and BLEU (Papi et al., 2023); (Papi et al., 2023). In prompt-engineering systems, formalization focuses on schema conformance and style sensitivity.

POML provides explicit metrics for serialized-output evaluation. In TableQA, accuracy is

$$H_{sep} = \text{Separator}(H^{(2)}), \qquad H_{sep}^{s} = \text{ReLU}(\text{Linear}^{s}(\text{LayerNorm}(H_{sep}))).$$1

relative improvement is

$$H_{sep} = \text{Separator}(H^{(2)}), \qquad H_{sep}^{s} = \text{ReLU}(\text{Linear}^{s}(\text{LayerNorm}(H_{sep}))).$$2

and style ranking stability is measured by Spearman correlation,

$$H_{sep} = \text{Separator}(H^{(2)}), \qquad H_{sep}^{s} = \text{ReLU}(\text{Linear}^{s}(\text{LayerNorm}(H_{sep}))).$$3

Across 100 sampled styles over 283 WikiTQ validation samples, GPT-3.5-turbo ranged from 6% to 61.8% accuracy, Phi-3 Medium from 0.7% to 32.2%, while DeepSeek V3 and Gemini 2.0 Flash varied by approximately 21% and 16%. From a single 30-line base POML template, the system generated 73,926 unique style combinations. The paper also reports Spearman self-correlation values such as 0.931 for Phi-3 Medium and 0.866 for Claude 3 Haiku, indicating stable style orderings across data splits (Zhang et al., 19 Aug 2025).

PromptMN proposes compliance-oriented metrics for serialized outputs. If $$H_{sep} = \text{Separator}(H^{(2)}), \qquad H_{sep}^{s} = \text{ReLU}(\text{Linear}^{s}(\text{LayerNorm}(H_{sep}))).$$4 is the target schema and $$H_{sep} = \text{Separator}(H^{(2)}), \qquad H_{sep}^{s} = \text{ReLU}(\text{Linear}^{s}(\text{LayerNorm}(H_{sep}))).$$5 is the observed output, compliance is

$$H_{sep} = \text{Separator}(H^{(2)}), \qquad H_{sep}^{s} = \text{ReLU}(\text{Linear}^{s}(\text{LayerNorm}(H_{sep}))).$$6

It also defines

$$H_{sep} = \text{Separator}(H^{(2)}), \qquad H_{sep}^{s} = \text{ReLU}(\text{Linear}^{s}(\text{LayerNorm}(H_{sep}))).$$7

and

$$H_{sep} = \text{Separator}(H^{(2)}), \qquad H_{sep}^{s} = \text{ReLU}(\text{Linear}^{s}(\text{LayerNorm}(H_{sep}))).$$8

The paper evaluates feasibility across Claude Fable 5, Claude Opus 4.8, Gemini 3.1 Pro, and GPT-5.5, reporting that the models correctly resolved PromptMN instructions, including repetition, conditionals, methods, and a prime-checking task, without fine-tuning (Dovdon, 15 Jun 2026).

A notable empirical theme is that serialized prompting does not simply improve correctness by adding more text. It changes the structure of the target space or output contract. In MT-ASR, the improvement comes from supplying talker-aligned auxiliary content. In transducer-based streaming work, it comes from learning a single tagged sequence rather than coordinating multiple decoders. In prompt engineering, it comes from explicit contracts, stable renderings, and validator-compatible schemas.

6. Limitations, failure modes, and open directions

The current literature also makes clear that SOP is not a universal remedy. In MT-ASR, training all parameters jointly in a single stage was suboptimal, Stage 2 caused degradation due to CTC ⟨blank⟩-induced sparsity, SOP-only inputs were unstable, and evaluation was limited to up to three talkers; broader scalability and domain shifts were not explicitly addressed (Shi et al., 1 Sep 2025). In serialized streaming ASR/ST, alignment errors, unaligned function words, non-monotonic reordering, timestamp noise, and frequent language switching can all degrade quality or latency; too large a grouping interval can also hurt quality (Papi et al., 2023); (Papi et al., 2023).

In prompt-engineering systems, POML explicitly states that strong output contracts and examples cannot fully guarantee perfect serialization. Token budget overhead, nested serialization complexity, large-document IDE performance, and model non-compliance remain practical concerns. PromptMN likewise notes that large-scale validation, formal parsers, linters, and certified interpreters remain future work, and that prompt injection and cross-model variance require further empirical study (Zhang et al., 19 Aug 2025); (Dovdon, 15 Jun 2026).

These limitations suggest that SOP should be treated as a design discipline rather than a single mechanism. Where the task is acoustically complex, SOP must complement acoustic evidence. Where the task is streaming and multilingual, SOP must balance switch frequency against latency. Where the task is schema-constrained generation, SOP must be paired with validators, repair loops, and model-specific style profiles. The recent literature therefore presents SOP not as a monolithic prompt recipe, but as a broader strategy for making output structure explicit, serialized, and operationally reliable across speech and language systems.