For end-to-end performance testing, workload simulation is an important method to enhance the real workload while protecting user privacy. To ensure the effectiveness of the workload simulation, it is necessary to dynamically evaluate the similarity of system inner status using key performance indicators(KPIs), which provide a comprehensive record of the system status, between the simulated workload and real workload by injecting workload into the system. However, due to the characteristics of KPIs, including large data size, amplitude differences, phase shifts, non-smoothness, high dimension, and Large numerical span, it is unpractical to evaluation on the full volume of KPIs and is challenging to measure the similarity between KPIs. In this paper, we propose a similarity metric algorithm for KPIs, ESBD, which describes both shape and intensity similarity. Around ESBD, a KPIs-based quality evaluation of workload simulation(KEWS) was proposed, which consists of four steps: KPIs preprocessing, KPIs screening, KPIs clustering, and KPIs evaluation. These techniques help mitigate the negative impact of the KPIs characteristics and give a comprehensive evaluation result. The experiments conducted on Hipstershop, an open-source microservices application, show the effectiveness of the ESBD and KEWS.

翻译：对于端到端的绩效测试,工作量模拟是提高实际工作量,同时保护用户隐私的一个重要方法。为确保工作量模拟的有效性,有必要使用关键业绩指标(KPI)动态地评估系统内部状况的相似性,这些指标通过向系统注入工作量,全面记录系统模拟工作量和实际工作量,全面记录系统状况,但是,由于KPI的特征,包括数据大小大、振幅差异、阶段转移、非抽取性、高尺寸和大数字范围,因此,对KPI的全部数量进行评价是不切实际的,而且难以衡量KPI之间的相似性。在本文件中,我们提议为KPI、ESBD提出一个类似性指标算法,描述形状和强度相似性。在ESBD周围,提出了基于KPI对工作量模拟的质量评价,其中包括四个步骤:KPI的预处理、KPI的筛选、KPI的集群和KPIs的评价。这些技术有助于减轻KPI特性的消极影响,并对KPI的相似性进行衡量。在本文件中,我们提议为KPB、ESA的开放应用软件和EBA的效益。