Benchmarking Distributed Transactional Database Systems

With the attractive characteristics of scalability, strong consistency, and high availability, distributed databases have attracted much attention. Moreover, application-oriented database development promotes the fast evolution and development of various distributed databases. Then there is a growing need for more precise and comprehensive evaluations to facilitate the selection and deployment of distributed databases. Though there have already been a bunch of benchmarks, we observe that existing benchmarks often fall short in addressing the technical challenges posed by distributed systems and particularly in providing quantitative control to workload for a fair comparison. To address these gaps, we identify five critical evaluation scenarios, focusing on distributed transaction processing, dynamic data scheduling, distributed lock management, distributed clock management, and fault recovery. Based on this analysis, we design and implement Sherry, an evaluation benchmark tool specifically tailored to these scenarios. Through extensive experiments conducted on OceanBase, we expose Sherry’s effectiveness in assessing the key design and optimization of OceanBase which are the inherent challenges for distributed transaction processing. Our findings validate Sherry’s feasibility as a robust benchmarking tool in this domain.