Federated learning (FL) enables distributed clients to collaboratively train a machine learning model without sharing raw data with each other. However, it suffers the leakage of private information from uploading models. In addition, as the model size grows, the training latency increases due to limited transmission bandwidth and the model performance degrades while using differential privacy (DP) protection. In this paper, we propose a gradient sparsification empowered FL framework over wireless channels, in order to improve training efficiency without sacrificing convergence performance. Specifically, we first design a random sparsification algorithm to retain a fraction of the gradient elements in each client's local training, thereby mitigating the performance degradation induced by DP and and reducing the number of transmission parameters over wireless channels. Then, we analyze the convergence bound of the proposed algorithm, by modeling a non-convex FL problem. Next, we formulate a time-sequential stochastic optimization problem for minimizing the developed convergence bound, under the constraints of transmit power, the average transmitting delay, as well as the client's DP requirement. Utilizing the Lyapunov drift-plus-penalty framework, we develop an analytical solution to the optimization problem. Extensive experiments have been implemented on three real life datasets to demonstrate the effectiveness of our proposed algorithm. We show that our proposed algorithms can fully exploit the interworking between communication and computation to outperform the baselines, i.e., random scheduling, round robin and delay-minimization algorithms.

翻译：联邦学习（FL）允许分布式客户端在互不共享原始数据的情况下协同训练机器学习模型。然而，由于上传模型会泄露私人信息，因此FL受到了保护隐私的限制。另外，随着模型大小的增加，由于传输带宽有限，训练延迟增加以及模型性能受差分隐私的保护影响而下降。在本文中，我们提出了一种在无线信道上使用梯度稀疏化的FL框架，以提高训练效率而不影响收敛性能。具体而言，我们首先设计了一种随机稀疏化算法，以保留每个客户端的局部训练梯度元素的一部分，从而减轻了DP引起的性能退化并减少了在无线信道上传输的参数数量。然后，我们通过建模非凸FL问题来分析所提出算法的收敛界限。接下来，我们将一个时间序列随机优化问题制定为最小化发展出的渐进界限，同时满足传输功率、平均传输延迟和客户端的差分隐私需求。利用Lyapunov漂移加惩罚框架，我们开发了一个分析解来解决优化问题。我们在三个真实数据集上进行了大量实验来证明我们所提出的算法的有效性。我们展示了我们的算法可以充分利用通信和计算之间的相互作用，以胜过基线，即随机调度、轮流和延迟最小化算法。