Existing studies in black-box optimization for machine learning suffer from low generalizability, caused by a typically selective choice of problem instances used for training and testing different optimization algorithms. Among other issues, this practice promotes overfitting and poor-performing user guidelines. To address this shortcoming, we propose in this work a benchmark suite, OptimSuite, which covers a broad range of black-box optimization problems, ranging from academic benchmarks to real-world applications, from discrete over numerical to mixed-integer problems, from small to very large-scale problems, from noisy over dynamic to static problems, etc. We demonstrate the advantages of such a broad collection by deriving from it Automated Black Box Optimizer (ABBO), a general-purpose algorithm selection wizard. Using three different types of algorithm selection techniques, ABBO achieves competitive performance on all benchmark suites. It significantly outperforms previous state of the art on some of them, including YABBOB and LSGO. ABBO relies on many high-quality base components. Its excellent performance is obtained without any task-specific parametrization. The OptimSuite benchmark collection, the ABBO wizard and its base solvers have all been merged into the open-source Nevergrad platform, where they are available for reproducible research.

翻译：用于机器学习的黑箱优化现有研究具有较低的通用性,其原因是对用于培训和测试不同优化算法的问题进行典型的选择性选择,除其他问题外,这种做法促进超配和业绩不佳的用户准则。为解决这一缺陷,我们提议在这项工作中采用一个基准套件 " 优化 ",它涵盖从学术基准到现实应用等广泛的黑箱优化问题,从数字过大到混杂问题,从小到大问题,从小到大问题,从动态过急到静态问题等等。我们通过从中产生一个通用算法选择精华的自动黑箱优化师(ABBO),展示了这种广泛收集的优势。使用三种不同的算法选择技术,ABBO在所有基准套件上都取得了竞争性的绩效。它大大超越了包括YABBOBOB和LSGO在内的某些应用程序的以往状态。ABBO依靠许多高质量的基础组件。它的业绩是出色的,而没有任何具体任务分解问题的。我们从中获得了良好的业绩。我们从它得到的出色业绩,它是一个通用的黑箱优化的算法基础基础收集,ABABBA和ABO的所有基础基础数据库,在那里都有了它可以用来进行综合研究。