Recent advances in many domains require more and more complicated experiment design. Such complicated experiments often have many parameters, which necessitate parameter tuning. Tree-structured Parzen estimator (TPE), a Bayesian optimization method, is widely used in recent parameter tuning frameworks. Despite its popularity, the roles of each control parameter and the algorithm intuition have not been discussed so far. In this tutorial, we will identify the roles of each control parameter and their impacts on hyperparameter optimization using a diverse set of benchmarks. We compare our recommended setting drawn from the ablation study with baseline methods and demonstrate that our recommended setting improves the performance of TPE. Our TPE implementation is available at https://github.com/nabenabe0928/tpe/tree/single-opt.

翻译：近年来，许多领域的最新进展需要更复杂的实验设计。这些复杂的实验通常具有许多参数，需要参数调优。树状Parzen估计器（TPE），一种贝叶斯优化方法，广泛应用于最近的参数调整框架中。尽管其受欢迎程度很高，但至今未讨论每个控制参数及其直觉的作用。在本教程中，我们将使用多样的基准测试来确定每个控制参数的作用及其对超参数优化的影响。我们将建立与基准方法的比较，证明我们从擦除研究中得出的推荐设置优化了TPE的性能。我们的TPE实现可在https://github.com/nabenabe0928/tpe/tree/single-opt上获得。