Impact of etcd Deployment on Kubernetes, Istio, and Application Performance

Abstract: By intrinsic necessity, Kubernetes is a complex platform. Its complexity makes conducting performance analysis in that environment fraught with difficulties and emergent behavior. Applications leveraging more "moving parts" such as the Istio service mesh makes the platform strictly more complex, not less. In this paper we study how underlying platform constitution and deployment affects application performance, specifically in Kubernetes-based environments. We alter platform constitution via use of native Kubernetes networking or Istio. Platform deployment is altered via etcd data storage location at two extremes on the performance spectrum: network disk and RAM disk. Our results show that etcd performance has a large impact on that of Kubernetes and its ability to perform orchestration actions, and thereby indirectly on the performance of the application. The implication is that systems researchers conducting performance evaluations cannot just consider their specific application as being under test, but must also take the underlying Kubernetes platform into account. To conduct experiments of scientific rigor, we developed an experiment framework for conducting repeatable and reproducible experiments. Our framework and resulting data set are openly available for the research community to build upon and reason about.