We propose a new adaptive empirical Bayes framework, the Bag-Of-Null-Statistics (BONuS) procedure, for multiple testing where each hypothesis testing problem is itself multivariate or nonparametric. BONuS is an adaptive and interactive knockoff-type method that helps improve the testing power while controlling the false discovery rate (FDR), and is closely connected to the "counting knockoffs" procedure analyzed in Weinstein et al. (2017). Contrary to procedures that start with a $p$-value for each hypothesis, our method analyzes the entire data set to adaptively estimate an optimal $p$-value transform based on an empirical Bayes model. Despite the extra adaptivity, our method controls FDR in finite samples even if the empirical Bayes model is incorrect or the estimation is poor. An extension, the Double BONuS procedure, validates the empirical Bayes model to guard against power loss due to model misspecification.

翻译：我们提出了一个新的适应性经验性贝耶斯框架,即Bag-of-Null-Statistics(BONuS)程序,用于多次测试,其中每个假设测试问题本身都是多变量或非参数的。BONuS是一种适应性和互动性登门型方法,有助于在控制虚假发现率的同时提高测试力(FDR),并与Weinstein等人(2017年)分析的“计算淘汰”程序密切相关。与从每套假设的美元价值开始的程序相反,我们的方法分析整个数据集,以适应性地估算以经验性贝耶斯模型为基础的最佳值值变换。尽管存在额外适应性,但我们的方法在有限的样本中控制FDR,即使经验性贝雅模型不正确或估计不准确。一个扩展,即双倍BONUS程序,验证了经验性海湾模型防止因模型错误确定而导致的权力损失。