Solving Big Data Challenges for Enterprise Application Performance Management (1208.4167v1)

Abstract: As the complexity of enterprise systems increases, the need for monitoring and analyzing such systems also grows. A number of companies have built sophisticated monitoring tools that go far beyond simple resource utilization reports. For example, based on instrumentation and specialized APIs, it is now possible to monitor single method invocations and trace individual transactions across geographically distributed systems. This high-level of detail enables more precise forms of analysis and prediction but comes at the price of high data rates (i.e., big data). To maximize the benefit of data monitoring, the data has to be stored for an extended period of time for ulterior analysis. This new wave of big data analytics imposes new challenges especially for the application performance monitoring systems. The monitoring data has to be stored in a system that can sustain the high data rates and at the same time enable an up-to-date view of the underlying infrastructure. With the advent of modern key-value stores, a variety of data storage systems have emerged that are built with a focus on scalability and high data rates as predominant in this monitoring use case. In this work, we present our experience and a comprehensive performance evaluation of six modern (open-source) data stores in the context of application performance monitoring as part of CA Technologies initiative. We evaluated these systems with data and workloads that can be found in application performance monitoring, as well as, on-line advertisement, power monitoring, and many other use cases. We present our insights not only as performance results but also as lessons learned and our experience relating to the setup and configuration complexity of these data stores in an industry setting.

• The paper demonstrates that Apache Cassandra achieves linear scalability and high throughput for write-heavy APM tasks, albeit with increased latencies.
• The study reveals that while HBase and VoltDB perform well in specific scenarios, HBase faces high read latency and VoltDB struggles with distributed scalability.
• The evaluation uses YCSB benchmarks on both memory-bound and disk-bound clusters to provide practical insights for optimizing data store configurations in APM systems.

Evaluation of Data Stores for Application Performance Management

The proliferation of complex enterprise systems has underscored the necessity for robust application performance management (APM) tools. This paper details an empirical performance evaluation of six open-source data stores: Apache Cassandra, Apache HBase, Project Voldemort, Redis, VoltDB, and MySQL Cluster, within the context of application performance monitoring. The main challenges addressed include sustaining high data rates and providing up-to-date views of infrastructure with minimal resource overhead.

Methodology and Setup

The evaluation employed the Yahoo! Cloud Serving Benchmark (YCSB) to simulate workloads reflective of APM tasks. Each system's performance was quantified based on its throughput and latency across a range of workloads, emphasizing varying proportions of read and write operations and incorporating scan operations where applicable. The paper utilized two distinct cluster configurations: a memory-bound cluster (Cluster M) and a disk-bound cluster (Cluster D), allowing for comprehensive insights into both memory-centric and disk-centric processing dynamics.

Results and Analysis

Scalability and Throughput

Cassandra consistently demonstrated superior scalability, exhibiting a linear increase in throughput with the number of nodes, albeit with elevated latencies. Its architecture, designed for write-heavy operations, proved beneficial in scenarios involving massive data ingestion. HBase, while achieving the lowest throughput per single node, also exhibited linear scalability; however, its read latency was significantly higher, suggesting a trade-off between read performance and scalability.

Latency

The paper revealed that Project Voldemort offered a balanced performance with stable latencies for both read and write operations, though with modest throughput improvements as compared to Cassandra. Redis, in its standalone configuration, exceeded others in read-heavy workloads, thanks to its in-memory data handling, yet its sharded operation lagged due to suboptimal data distribution handled by the client-side library. MySQL, similarly sharded, matched Cassandra's throughput under certain workloads but suffered from performance degradation in scan operations, primarily due to inefficient SQL query translations for scans.

VoltDB

VoltDB, an ACID-compliant in-memory database, showcased high single-node throughput but failed to scale effectively across nodes, indicating potential inefficiencies in concurrent transaction handling across distributed nodes.

Implications for APM Systems

The findings highlight that while Cassandra and HBase provide robust solutions for high-scale environments typical in modern enterprises, considerations around latency become crucial, particularly in latency-sensitive applications. The trade-offs observed suggest that careful configuration and possibly hybrid approaches (utilizing multiple data store types) could optimize both scalability and access latency.

Future Directions

For practical deployment, further investigation into the impacts of replication and data compression on these data stores is warranted. Additionally, expanding the paper to include emerging storage architectures and hybrid models could offer deeper insights for APM systems dealing with next-generation enterprise applications.

In conclusion, this paper accentuates the nuanced performance characteristics of prevalent key-value and relational data stores in the field of APM, offering valuable benchmarks and insights that can guide system architects in selecting and tuning data management solutions for enterprise environments.