We consider the straggler problem in decentralized learning over a logical ring while preserving user data privacy. Especially, we extend the recently proposed framework of differential privacy (DP) amplification by decentralization by Cyffers and Bellet to include overall training latency--comprising both computation and communication latency. Analytical results on both the convergence speed and the DP level are derived for both a skipping scheme (which ignores the stragglers after a timeout) and a baseline scheme that waits for each node to finish before the training continues. A trade-off between overall training latency, accuracy, and privacy, parameterized by the timeout of the skipping scheme, is identified and empirically validated for logistic regression on a real-world dataset.

翻译：我们认为,在维护用户数据隐私的同时,在逻辑环上的分散学习中,分散学习是分流的难题。 特别是,我们扩大了最近提出的差异隐私框架(DP)扩大范围,由Cyffers和Bellet进行权力下放,以纳入由计算和沟通延迟组成的整体长期培训。 关于趋同速度和DP水平的分析结果来自一个跳过计划(在超时后忽略了分流者 ) 和一个等待每个节点在继续培训之前完成的基线计划。 以跳过计划超时为参数的总体培训延缓性、准确性和隐私之间的权衡被确定并被经验验证为真实世界数据集的后勤倒退。