LinGBM: A Performance Benchmark for Approaches to Build GraphQL Servers (Extended Version)

Abstract: GraphQL is a popular new approach to build Web APIs that enable clients to retrieve exactly the data they need. Given the growing number of tools and techniques for building GraphQL servers, there is an increasing need for comparing how particular approaches or techniques affect the performance of a GraphQL server. To this end, we present LinGBM, a GraphQL performance benchmark to experimentally study the performance achieved by various approaches for creating a GraphQL server. In this article, we discuss the design considerations of the benchmark, describe its main components (data schema; query templates; performance metrics), and analyze the benchmark in terms of statistical properties that are relevant for defining concrete experiments. Thereafter, we present experimental results obtained by applying the benchmark in three different use cases, which demonstrates the broad applicability of LinGBM.