An important limitation of standard multiple testing procedures is that the null distribution should be known. Here, we consider a null distribution-free approach for multiple testing in the following semi-supervised setting: the user does not know the null distribution, but has at hand a sample drawn from this null distribution. In practical situations, this null training sample (NTS) can come from previous experiments, from a part of the data under test, from specific simulations, or from a sampling process. In this work, we present theoretical results that handle such a framework, with a focus on the false discovery rate (FDR) control and the Benjamini-Hochberg (BH) procedure. First, we provide upper and lower bounds for the FDR of the BH procedure based on empirical $p$-values. These bounds match when $\alpha (n+1)/m$ is an integer, where $n$ is the NTS sample size and $m$ is the number of tests. Second, we give a power analysis for that procedure suggesting that the price to pay for ignoring the null distribution is low when $n$ is sufficiently large in front of $m$; namely $n\gtrsim m/(\max(1,k))$, where $k$ denotes the number of ``detectable'' alternatives. Third, to complete the picture, we also present a negative result that evidences an intrinsic transition phase to the general semi-supervised multiple testing problem {and shows that the empirical BH method is optimal in the sense that its performance boundary follows this transition phase}. Our theoretical properties are supported by numerical experiments, which also show that the delineated boundary is of correct order without further tuning any constant. Finally, we demonstrate that our work provides a theoretical ground for standard practice in astronomical data analysis, and in particular for the procedure proposed in \cite{Origin2020} for galaxy detection.

翻译：标准多重测试程序的一个重要限制是, 不存在分配 。 在此, 我们考虑在以下半监督环境下对多重测试采用无效分配方法 : 用户不知道无分配, 但手头有从此无分配的样本 。 在实际情况下, 这个无效培训样本可以来自先前的实验, 来自测试中数据的一部分, 来自特定模拟, 或来自抽样过程 。 在此工作中, 我们展示处理此框架的理论结果, 重点是虚假的发现率( FDR) 控制和 Benjami- Hochberg (BH) 程序。 首先, 我们为基于实证 $- p$ 的 BH 程序的 FDR 提供上下边框 。 在实际情况下, 当 $\ alpha (n+1) / m 美元是整数时, 也就是NTS 样本大小, 或 $20 美元是提议的测试数量 。 其次, 我们为这个程序提供无视无效分配的价格是低的 $n- O) 和 bential 的内值 。 。 在 解算法中, 也显示直数的轨中, 。 直数 直径 。 直值 。 。 直值 直值 。 直值 直值 直值 直值 直值 解到直到直到直到直到直到直值 。