Styx: Transactional Stateful Functions on Streaming Dataflows (2312.06893v3)

Abstract: Developing stateful cloud applications, such as low-latency workflows and microservices with strict consistency requirements, remains arduous for programmers. The Stateful Functions-as-a-Service (SFaaS) paradigm aims to serve these use cases. However, existing approaches either provide serializable transactional guarantees at the level of individual functions, or separate application logic from the state and use inefficient transactional protocols. These design choices increase the execution latency, limiting the adoption of SFaaS systems. In this paper, we present Styx, a novel SFaaS runtime that executes serializable transactions across functions with exactly-once guarantees. Styx extends a deterministic transactional protocol to support an arbitrary call graph of stateful functions. It introduces a transaction-execution acknowledgment scheme that allows tracking a transactional workflow's SFaaS calls, guaranteeing atomicity and exactly-once processing. Finally, Styx features a function-execution caching mechanism and early transactional commit replies for optimized performance. Experiments with the YCSB-T, TPC-C, and Deathstar benchmarks show that Styx outperforms state-of-the-art approaches by achieving at least one order of magnitude higher throughput while exhibiting near-linear scalability and low latency.