The correct use of cryptography is central to ensuring data security in modern software systems. Hence, several academic and commercial static analysis tools have been developed for detecting and mitigating crypto-API misuse. While developers are optimistically adopting these crypto-API misuse detectors (or crypto-detectors) in their software development cycles, this momentum must be accompanied by a rigorous understanding of their effectiveness at finding crypto-API misuse in practice. This paper presents the MASC framework, which enables a systematic and data-driven evaluation of crypto-detectors using mutation testing. We ground MASC in a comprehensive view of the problem space by developing a data-driven taxonomy of existing crypto-API misuse, containing $105$ misuse cases organized among nine semantic clusters. We develop $12$ generalizable usage-based mutation operators and three mutation scopes that can expressively instantiate thousands of compilable variants of the misuse cases for thoroughly evaluating crypto-detectors. Using MASC, we evaluate nine major crypto-detectors and discover $19$ unique, undocumented flaws that severely impact the ability of crypto-detectors to discover misuses in practice. We conclude with a discussion on the diverse perspectives that influence the design of crypto-detectors and future directions towards building security-focused crypto-detectors by design.

翻译：正确使用加密技术对于确保现代软件系统的数据安全至关重要,因此,已经开发了若干学术和商业静态分析工具,用于检测和减少对API的滥用;虽然开发者乐观地在其软件开发周期中采用这些加密API滥用检测器(或加密检测器),但这一势头必须伴之以对其在实际中发现加密应用API滥用的有效性的严格理解;本文件介绍了MASC框架,它使得能够利用突变测试系统对加密检测器进行系统化和数据驱动的评价;我们将MASC放在对问题空间的全面认识之下,方法是对现有的加密API滥用进行数据驱动的分类,其中包括由9个语义组组成的105万美元的滥用案件;我们开发了1,200美元的一般基于使用的突变操作器和3个突变范围,能够瞬间地表达数千个可比较的误用案例变量,以便利用突变测试系统对加密检测器检测器进行彻底评价;我们利用MASC,评估了9个主要加密检测器对空间问题的全面认识,对现有的加密API误用空间进行了数据分析,并发现了19美元的误用税项,并发现了安全设计上的独特性设计影响。