Spiking neural networks (SNNs) are bio-inspired neural networks with asynchronous discrete and sparse characteristics, which have increasingly manifested their superiority in low energy consumption. Recent research is devoted to utilizing spatio-temporal information to directly train SNNs by backpropagation. However, the binary and non-differentiable properties of spike activities force directly trained SNNs to suffer from serious gradient vanishing and network degradation, which greatly limits the performance of directly trained SNNs and prevents them from going deeper. In this paper, we propose a multi-level firing (MLF) method based on the existing spatio-temporal back propagation (STBP) method, and spiking dormant-suppressed residual network (spiking DS-ResNet). MLF enables more efficient gradient propagation and the incremental expression ability of the neurons. Spiking DS-ResNet can efficiently perform identity mapping of discrete spikes, as well as provide a more suitable connection for gradient propagation in deep SNNs. With the proposed method, our model achieves superior performances on a non-neuromorphic dataset and two neuromorphic datasets with much fewer trainable parameters and demonstrates the great ability to combat the gradient vanishing and degradation problem in deep SNNs.

翻译：脉冲神经网络（SNN）是一种生物启发式的神经网络，具有异步离散和稀疏特性，这些特性已经在低能耗方面展现出卓越的优势。最近的研究致力于利用时空信息通过反向传播直接训练SNN。然而，尖峰活动的二值化和非可微性质会导致直接训练的SNN受到严重的梯度消失和网络退化的困扰，这极大地限制了直接训练的SNN的性能，并阻止它们更深地发展。在本文中，我们提出了一种基于现有时空背传（STBP）方法的多级射击（MLF）方法和脉冲休眠扼抑残差网络（脉冲DS-ResNet）。MLF增强了梯度传播的效率和神经元的递增表达能力。脉冲DS-ResNet可以高效地执行离散尖峰的身份映射，并为深度SNN的梯度传播提供更合适的连接。通过所提出的方法，我们的模型在非神经型数据集和两个神经型数据集上取得了卓越的性能，所需的可训练参数更少，并展现了在深度SNN的梯度消失和退化问题上具有很强的应对能力。