StreamFusion: Efficient Streaming Fusion

• StreamFusion is a family of methods that fuse multiple data streams or operator pipelines to produce efficient, abstraction-free code and high-performance streaming systems.
• It leverages formal models, like co-algebraic stream abstractions and normalization-by-evaluation, to normalize combinator pipelines into optimized imperative loops.
• In machine learning and ASR, StreamFusion enhances system robustness and throughput by integrating early, middle, and late fusion strategies as well as neural-symbolic reasoning.

StreamFusion encompasses a family of techniques—both theoretical and practical—that realize efficient, expressive, and high-performance fusion of multiple data streams or operator pipelines in streaming computation. The term is contextually overloaded: in programming language implementation, StreamFusion denotes code transformation and normalization techniques that generate single-loop, abstraction-free code from high-level compositional stream pipelines; in modern machine learning, StreamFusion (and related fusion methods) refers to algorithmic frameworks that combine multiple data, feature, or inference streams within DNN/transformer architectures—typically to improve robustness, throughput, or latency for streaming tasks.

1. Theoretical Foundations and Models

The core theoretical underpinning of StreamFusion in programming languages is the formalization of stream combinators and their normalization into highly optimized imperative state machines. A central model is the co-algebraic (pull-style) stream abstraction, defined as:

$$\mathit{Stream}\,\alpha \;=\;\exists s.\;\bigl(s\times\bigl(s\to(\alpha,s)\,\mathrm{stream\_shape}\bigr)\bigr)$$

where $$\mathrm{stream\_shape} = \mathit{Nil}\mid\mathit{Cons}(\alpha,s)$$, and combinator pipelines (map, filter, take, flatMap/concatMap, zipWith) are described by explicit mathematical equations operating over the stepper functions and hidden state variables (Kiselyov et al., 2016, Kiselyov et al., 2024).

Modern equational theories, such as that for Strymonas, extend the model to stateful skipping streams, achieving a unique normal form for every well-formed pipeline. The normalization process is realized via Normalization-by-Evaluation (NbE), yielding an imperative state machine with no abstraction overhead (Kiselyov et al., 2024). This formal foundation guarantees correctness (bisimilarity with the high-level semantics), termination, and confluence.

2. Code Generation and Pipeline Fusion in Programming Languages

StreamFusion systems for programming languages (OCaml+MetaOCaml, Scala+LMS) compile user pipeline expressions—involving arbitrary combinations and nestings of stream combinators—into a single fused imperative loop. The multi-stage approach proceeds as follows (Kiselyov et al., 2016):

1. Pipeline Staging: User code is written in a two-level DSL, supporting all primitive combinators (map, filter, take, drop, flat_map, zip_with, fold), with all intermediate values stage-annotated for code generation.
2. Partial Evaluation and Fusion: All combinators are inlined, closures and temporary tuples eliminated, and stream-shape constructors are represented via CPS (eliminating dynamic pattern matches). State threading is removed via let-insertion and in-place mutation.
3. Imperative Code Emission: The final output is a tight, abstraction-free loop (or nested loops for nested streams) that matches or outperforms hand-written code. No dynamic allocation or virtual calls occur except where user code introduces them.

Recent advances extend fusion to stateful streams, supporting combinators such as zip, flatMap, map-accumulate, sliding window, and arbitrary nested pipelines (Kiselyov et al., 2024). The normalization algorithm is fully deterministic, drives all combinators into normal form by the equational theory, and ultimately generates statically guaranteed high-performance code (OCaml, C, Scala).

3. Operator and Feature Stream Fusion in Machine Learning and ASR

In modern machine learning, StreamFusion techniques generalize to the fusion of heterogeneous data or feature streams within DNN-centric architectures. In end-to-end ASR and speech recognition, “stream fusion” (often called system combination) realizes the combination of parallel feature representations (e.g., magnitude + phase, audio + video) at different architectural levels:

• Early Fusion: Fusing stacked features at the front-end (feature concatenation before the encoder); generally shows little WER gain.
• Middle Fusion: Fusing the outputs of multiple encoder branches inside the decoder by weighted sum, concatenation/projection, or parameter tying. Example: given parallel transformer encoders ($H_{\mathrm{mag}}, H_{\mathrm{phase}}$), the decoder combines both streams (e.g., $$h^\mathrm{middle}_\ell = \alpha h^\mathrm{mag}_\ell + (1-\alpha)h^\mathrm{phase}_\ell$$), with a bias toward one “primary” stream (Lohrenz et al., 2021).
• Late Fusion: Combining the token posteriors (during beam search) from multiple independently trained models via log-linear interpolation:

$$\log P_\ell^\mathrm{late}(c) = \beta\,\log P_\ell^\mathrm{mag}(c) + (1-\beta)\,\log P_\ell^\mathrm{phase}(c)$$

Late fusion is robust and achieves substantial WER improvements when two multi-encoder-learned models are combined at inference, but at the cost of doubled inference time.

4. Communication- and Topology-Aware Stream Fusion for Distributed Inference

In distributed inference for Diffusion Transformers (DiTs) and generative models, StreamFusion refers to topology-aware sequence parallelism (SP) that exploits hardware characteristics for efficient multi-GPU inference (Yang et al., 28 Jan 2026). Key innovations include:

• Topology-aware partitioning: Shards the GPU pool into intra-machine (high bandwidth, e.g., NVSwitch) and inter-machine (lower bandwidth) rings, with Ulysses-style all-to-all inside machines and Ring attention across machines.
• Torus Attention: Decomposes all-to-all (Ulysses) operations into pipelined micro-stages, overlapping inter-machine data movement with local computation, thus minimizing latency bubbles.
• One-sided Communication: Uses NVSHMEM primitives to avoid handshakes and per-step SM contention on GPUs.
• Performance Gains: Achieves 1.35× average (up to 1.77×) end-to-end speedup over prior unified SP baselines, with robust scalability across batch sizes and head sharding.

These architectural techniques are necessary to enable real-time inference for large-scale image/video generation at high resolution or long sequence lengths, by minimizing both compute and communication bottlenecks.

5. Neural-Symbolic Stream Fusion and Declarative Reasoning

Semantic StreamFusion addresses the fusion of multimodal input streams (e.g., DNN detections, temporal facts) within logic programming frameworks, as in CQELS 2.0 (Le-Tuan et al., 2022). Key elements:

• Neural-symbolic pipeline: DNN feature extractors emit symbolic (RDF-star) streams, which are continually queried and fused via hard and soft logic rules.
• Probabilistic Soft Rule Fusion: Each soft rule $r_i$ is assigned a learnable weight $w_i$, defining a Gibbs distribution over answer sets. Training proceeds via cross-entropy on ground-truth data, approximating expectations by single best-answer sets.
• Platform-agnostic Federated Execution: StreamFusion is distributed across heterogeneous hardware (embedded devices, cloud clusters) using an adaptive federator that decomposes queries and assigns subqueries for load balancing, leveraging cost models for CPU, memory, and network resources.
• Declarative Construction: Hybrid pipelines (e.g., multi-object tracking-by-detection) are fully described in CQELS-QL, blending tracking logic (SORT, DeepSORT) and windowed detection soft rules, all with trainable weights.

CQELS 2.0 experimentally demonstrates linear scaling of throughput with node count, sub-100 ms end-to-end latency, and increased tracking accuracy (precision/recall improvements and reduced ID switches) relative to classic non-fused pipelines.

6. Fusion Techniques in Streaming ASR and Robust Multimodal Fusion

For streaming ASR and audio-visual speech recognition, StreamFusion encompasses both architectural and algorithmic strategies for integrating multiple information streams, with a focus on low-latency, online inference:

• FusionFormer: A variant of the Conformer ASR model that replaces runtime-heavy LayerNorms with BatchNorm + ReLU, folded into each conv/linear, resulting in ∼10% lower inference latency while maintaining WER within 0.3% of LN-based Conformer (Song et al., 2022).
• Streaming LLM Fusion: In RNNT architectures, shallow and cold fusion (and early variants) combine external LMs with the RNNT backbone via scoring, gating, or DNN-projection. Cold fusion delivers up to 8.5% WER reduction, especially in moderate-resource regimes, with negligible additional latency (Cabrera et al., 2021).
• Decision Fusion for Multimodal ASR: StreamFusion modules implement temporal gating over acoustic and visual encoder representations, using small learnable gating networks and LSTM fusion nets on top of per-stream logits and reliability signals. Fusion is rewarded with entropy penalties that encourage confident stream selection; significant WER gains (up to 43% relative to audio-only, 31% over AV baselines) are observed on LRS2/LRS3 under noisy conditions, as the fusion module emphasizes more reliable streams (Yu et al., 2021).

7. Experimental Benchmarks and Impact

Across domains, StreamFusion techniques deliver measurable improvements in latency, throughput, and accuracy:

• Programming language fusion frameworks (Strymonas, MetaOCaml/LMS StreamFusion) consistently match or exceed hand-written loop performance (up to 35× faster than mainstream OCaml/Scala Streams), eliminating all abstraction overhead by construction (Kiselyov et al., 2016, Kiselyov et al., 2024).
• In distributed transformer inference, StreamFusion achieves 1.35–1.77× speedup with robust scaling in both model and hardware dimensions (Yang et al., 28 Jan 2026).
• For ASR/AVSR, best-in-class WER reductions are achieved via multi-level (early/middle/late) fusion, MEL strategies, and decision-fusion nets, all with no or modest increases in compute (Lohrenz et al., 2021, Yu et al., 2021, Song et al., 2022, Cabrera et al., 2021).

StreamFusion unifies a class of fundamental approaches—ranging from code normalization and theoretical foundations, to hardware-aware distributed operator scheduling, to robust multi-stream machine learning architectures—under the objective of achieving high-throughput, compositionality, and minimal latency in streaming computation settings.