A Benchmark for Databases with Varying Value Lengths (2508.07551v1)

Abstract: The performance of database management systems (DBMS) is traditionally evaluated using benchmarks that focus on workloads with (almost) fixed record lengths. However, some real-world workloads in key/value stores, document databases, and graph databases exhibit significant variability in value lengths, which can lead to performance anomalies, particularly when popular records grow disproportionately large. Existing benchmarks fail to account for this variability, leaving an important aspect of DBMS behavior underexplored. In this paper, we address this gap by extending the Yahoo! Cloud Serving Benchmark (YCSB) to include an "extend" operation, which appends data to record fields, simulating the growth of values over time. Using this modified benchmark, we have measured the performance of three popular DBMS backends: MongoDB, MariaDB with the InnoDB storage engine, and MariaDB with the MyRocks storage engine. Our experiments alternate between extending values and executing query workloads, revealing significant performance differences driven by storage engine design and their handling of variable-sized values. Our key contribution is the introduction of a novel benchmarking approach to evaluate the impact of growing value sizes and isolate the effect of querying data with a distribution of data sizes from any cost associated with accessing data after a history of updates. This highlights the need for more representative benchmarks that capture the dynamic nature of real-world workloads, providing valuable guidance for both practitioners and researchers.