Dynamic Environment Simulation for Database Performance Evaluation

The wide popularity and the maturity of cloud platform promote the development of Cloud Native database systems. On-demand resource configuration is an attractive feature of cloud platforms, but its complexity in resource management challenges the benchmarking of database performance, which is no longer in a stand-alone test environment. Sharing or contending of resources aggravates the dynamics of environment, which can influence database performance much. In order to expose the real performance in production environment, environment simulation is prerequisite for benchmarking databases. Although Docker Containers have been promoted to isolate resources, we still cannot achieve the true resource isolation. In this paper, we first define four kinds of workload generators corresponding to the key environmental dimensions, then builds a multi-order polynomial linear regression model to calculate the correlation among workloads, and simulates the dynamical changes of environment. It is the first work to provide a complete and dynamic simulation to environment. We conduct comprehensive experiments on the open-source DBMS by running the standard benchmarks to verify the effectiveness of our work.