Complete Fusion for Stateful Streams: Equational Theory of Stateful Streams and Fusion as Normalization-by-Evaluation (2412.15768v1)

Abstract: Processing large amounts of data fast, in constant and small space is the point of stream processing and the reason for its increasing use. Alas, the most performant, imperative processing code tends to be almost impossible to read, let alone modify, reuse -- or write correctly. We present both a stream compilation theory and its implementation as a portable stream processing library Strymonas that lets us assemble complex stream pipelines just by plugging in simple combinators, and yet attain the performance of hand-written imperative loops and state machines. The library supports finite and infinite streams and offers a rich set of combinators: from map, filter, take(while) to flat-map (nesting), zip, map-accumulate and sliding windowing. The combinators may be freely composed, and yet the resulting convoluted imperative code contains no traces of combinator abstractions: no closures, intermediate objects or tuples. The high-performance is portable and statically guaranteed, without relying on compiler or black-box optimizations. We greatly exceed in performance the available stream processing libraries in OCaml. The library exists in two versions, OCaml and Scala 3, and supports pluggable backends for code generation (currently: C, OCaml and Scala). Strymonas has been developed in tandem with the equational theory of stateful streams. Our theoretical model can represent all desired pipelines and also guarantees the existence of unique normal forms, which are mappable to (fused) state machines. We describe the normalization algorithm, as a form of normalization-by-evaluation. Stream pipeline compilation and optimization are represented as normalization, and are hence deterministic and terminating, with the guaranteed outcome. The equational theory lets us state and prove the correctness of the complete fusion optimization.

• The paper establishes an equational theory for stateful streams that enables complete fusion and deterministic pipeline compilation.
• It introduces normalization-by-evaluation to eliminate intermediate data structures and optimize complex operations like flat_map and zip.
• Implementation via the strymonas library demonstrates competitive performance against hand-optimized code, confirming practical efficiency.

Overview of "Complete Fusion for Stateful Streams"

The paper "Complete Fusion for Stateful Streams" presents a thorough exploration of optimizing stream processing through the development of a theoretical foundation and its practical implementation in the form of the {strymonas\ library. This endeavor focuses primarily on achieving complete fusion in stream processing, eradicating intermediate data structures and closures that typically add overhead and complexity. The authors provide both a theory and an implementation that addresses complex pipelines, including those with combinations of flat_map and zip, thus exceeding the limitations of previous approaches in this domain.

Stream Processing Challenges and Goals

The motivation for this work stems from the growing need for efficient stream processing to handle large volumes of data, such as those encountered in automated trading and big data applications. The paper identifies key challenges in developing high-performance stream processing systems, notably the trade-off between abstraction and performance. While imperative state-machines deliver the best performance, they are not reusable or easy to modify. On the contrary, declarative approaches offer ease of use but suffer performance drawbacks due to the overhead of abstractions.

The central ambition of the research is to support complex stream operations with complete fusion, aiming to generate efficient, low-level, imperative code from high-level, declarative stream pipelines. This goal encompasses operations like zip and flat_map without imposing restrictions or leaving performance improvements to compiler optimizations.

Theoretical Contribution

At the core of their contribution is an equational theory of stateful streams, which introduces the notion of unique normal forms mappable to state machines. This theoretical model confirms the existence of normal forms for stream pipelines and enables deterministic and terminating pipeline compilation. By representing stream fusion as normalization-by-evaluation (NBE), the approach ensures predictable outcomes and avoids the pitfalls of conventional optimization techniques, which rely on non-confluent rewriting rules.

The paper's equational theory provides a structured basis to express and handle stateful streams. Laws like linearization and innovative strategies for eliminating zip through techniques such as zip-linear transformation ensure the broad applicability of the theoretical model to complex stream configurations.

Implementation and Practical Realization

Implemented as a portable library, {strymonas\ actualizes the theory for OCaml and Scala languages, with code generation backends for C, OCaml, and Scala. The distinction between host and target languages is deftly managed to allow extensive use of abstractions in the host language without infiltrating the generated target code, thereby ensuring complete fusion.

A remarkable aspect of the implementation is the treatment of stream state implicitly, optimizing it for imperative, typed, and hygienically assured code generation. The staging approach utilized in this implementation ensures that user-defined functions within stream operations are effectively inlined, eliminating function calls in the resultant code.

The {strymonas\ library also maintains a separation of concerns by implementing the API in the raw API format, enabling greater expressivity and direct mapping to the theoretical core API when needed. It demonstrates versatility by incorporating examples like run-length coding and nested grouping/aggregation.

Performance Evaluation

Empirical evaluations and benchmarks underscore the effectiveness of {strymonas\ in demonstrating significant performance gains over existing libraries. The library’s ability to comprehensively support and fuse complex, real-world stream operations like FM Radio reception validates its practical implications. Performance assessments exhibit the library closely matching or even surpassing hand-optimized code in certain scenarios, reaffirming the efficacy of the implemented fusion approach.

Conclusion

"Complete Fusion for Stateful Streams" contributes substantially to the domain of stream processing by offering a robust theoretical framework and an effective library implementation for achieving complete fusion. The paper sets a precedent by proving that advanced streaming operations can be optimized deterministically and efficiently without compromising the elegance of high-level stream abstractions. Looking forward, the authors speculate on extending their work to encompass broader domains such as ETL pipelines, highlighting potential expansions to the scope and impact of their research in artificial intelligence and data-intensive computing.