Reactive Liquid: Optimized Liquid Architecture for Elastic and Resilient Distributed Data Processing

Abstract: Today's most prominent IT companies are built on the extraction of insight from data, and data processing has become crucial in data-intensive businesses. Nevertheless, the size of data which should be processed is growing significantly fast. The pace of the data growing has changed the nature of data processing. Today, data-intensive industries demand highly scalable and fault tolerant data processing architectures which can handle the massive amount of data. In this paper, we presented a distributed architecture for elastic and resilient data processing based on the Liquid which is a nearline and offline big data architecture. We used the Reactive Manifesto to design the architecture highly reactive to workload changes and failures. We evaluate our architecture by drawing some numerical comparisons between our architecture prototype and the Liquid prototype. The performed evaluation shows that our architecture can be more scalable against workload and more resilient against failures than the Liquid architecture is.