A recent report of Littmann [Commun. ACM '21] outlines the existence and the fatal impact of collusion rings in academic peer reviewing. We introduce and analyze the problem Cycle-Free Reviewing that aims at finding a review assignment without the following kind of collusion ring: A sequence of reviewers each reviewing a paper authored by the next reviewer in the sequence (with the last reviewer reviewing a paper of the first), thus creating a review cycle where each reviewer gives favorable reviews. As a result, all papers in that cycle have a high chance of acceptance independent of their respective scientific merit. We observe that review assignments computed using a standard Linear Programming approach typically admit many short review cycles. On the negative side, we show that Cycle-Free Reviewing is NP-hard in various restricted cases (i.e., when every author is qualified to review all papers and one wants to prevent that authors review each other's or their own papers or when every author has only one paper and is only qualified to review few papers). On the positive side, among others, we show that, in some realistic settings, an assignment without any review cycles of small length always exists. This result also gives rise to an efficient heuristic for computing (weighted) cycle-free review assignments, which we show to be of excellent quality in practice.
翻译:Littmann[Commun. ACM '21]最近的报告概述了学术同侪审查中串通集团的存在及其致命影响。我们介绍和分析了无周期审查的问题,目的是找到一个没有以下串通圈的审查任务:每审查一份按顺序由下个审查者编写的文件(与上一个审查者审查第一份文件),从而创造一个审查周期,每个审查者都给予有利的审查。结果,该周期中的所有文件都具有很高的被接受的机会,而不受各自的科学价值的影响。我们注意到,使用标准线性规划方法计算的审查任务通常接受许多短期审查周期。在负面方面,我们显示,在各种受限制的案件中,无周期审查是困难的(即,每个作者都有资格审查所有文件,人们想防止作者审查对方或他们自己的文件,或者每个作者只有一份文件,而且只有很少有资格审查论文)。在正面方面,我们发现,在某些现实环境中,使用标准的线性规划方法计算出来的审查任务通常包括许多短期的审查周期。在各种受限制的案件中,我们显示无周期审查是难的(即当每个作者都有资格审查所有文件时,这种审查周期都具有很高的质量)。