The recent success of neural networks for solving difficult decision tasks has incentivized incorporating smart decision making "at the edge." However, this work has traditionally focused on neural network inference, rather than training, due to memory and compute limitations, especially in emerging non-volatile memory systems, where writes are energetically costly and reduce lifespan. Yet, the ability to train at the edge is becoming increasingly important as it enables real-time adaptability to device drift and environmental variation, user customization, and federated learning across devices. In this work, we address two key challenges for training on edge devices with non-volatile memory: low write density and low auxiliary memory. We present a low-rank training scheme that addresses these challenges while maintaining computational efficiency. We then demonstrate the technique on a representative convolutional neural network across several adaptation problems, where it out-performs standard SGD both in accuracy and in number of weight writes.

翻译：解决困难决策任务的神经网络最近的成功激励了“边缘”智能决策。然而,这项工作传统上侧重于神经网络推导,而不是培训,因为记忆和计算限制,特别是在新兴的非挥发性记忆系统,因为书写费用高昂,寿命缩短。然而,在边缘培训的能力越来越重要,因为它能够实时适应性地安装漂移和环境变异设备、用户定制和跨设备封存学习。在这项工作中,我们解决了在非挥发性记忆边缘装置上培训的两大挑战:低写密度和低辅助记忆。我们提出了一个低级别培训计划,既能应对这些挑战,又能保持计算效率。然后,我们展示了在多个适应问题中具有代表性的革命性神经网络的技术,在准确性和重量写作数量上都比标准SGD标准要好。