We formulate selecting the best optimizing system (SBOS) problems and provide solutions for those problems. In an SBOS problem, a finite number of systems are contenders. Inside each system, a continuous decision variable affects the system's expected performance. An SBOS problem compares different systems based on their expected performances under their own optimally chosen decision to select the best, without advance knowledge of expected performances of the systems nor the optimizing decision inside each system. We design easy-to-implement algorithms that adaptively chooses a system and a choice of decision to evaluate the noisy system performance, sequentially eliminates inferior systems, and eventually recommends a system as the best after spending a user-specified budget. The proposed algorithms integrate the stochastic gradient descent method and the sequential elimination method to simultaneously exploit the structure inside each system and make comparisons across systems. For the proposed algorithms, we prove exponential rates of convergence to zero for the probability of false selection, as the budget grows to infinity. We conduct three numerical examples that represent three practical cases of SBOS problems. Our proposed algorithms demonstrate consistent and stronger performances in terms of the probability of false selection over benchmark algorithms under a range of problem settings and sampling budgets.

翻译：我们提出了选择最佳优化系统（SBOS）问题的数学框架，并为此类问题提供了解决方案。在SBOS问题中，存在有限个候选系统。每个系统内部，一个连续决策变量会影响该系统的期望性能。SBOS问题通过比较各系统在自身最优决策下的期望性能来选择最佳系统，且事先既不知道各系统的期望性能，也不了解各系统内部的优化决策。我们设计了易于实现的算法，该算法自适应地选择待评估的系统及其决策变量以获取带噪声的系统性能观测值，通过序贯淘汰劣质系统，最终在消耗用户指定预算后推荐一个系统作为最优选择。所提出的算法结合了随机梯度下降方法与序贯淘汰策略，以同时利用各系统内部结构并进行跨系统比较。对于所提算法，我们证明了当预算趋于无穷时，误选概率以指数速率收敛至零。我们通过三个数值算例展示了SBOS问题的三种实际应用场景。在不同问题设置和采样预算下，与基准算法相比，我们提出的算法在误选概率方面均表现出更稳定且更优越的性能。