Brain inspired spiking neural networks (SNNs) have been successfully applied to many pattern recognition domains. The SNNs based deep structure have achieved considerable results in perceptual tasks, such as image classification, target detection. However, the application of deep SNNs in reinforcement learning (RL) tasks is still a problem to be explored. Although there have been previous studies on the combination of SNNs and RL, most of them focus on robotic control problems with shallow networks or using ANN-SNN conversion method to implement spiking deep Q Network (SDQN). In this work, we mathematically analyzed the problem of the disappearance of spiking signal features in SDQN and proposed a potential based layer normalization(pbLN) method to directly train spiking deep Q networks. Experiment shows that compared with state-of-art ANN-SNN conversion method and other SDQN works, the proposed pbLN spiking deep Q networks (PL-SDQN) achieved better performance on Atari game tasks.

翻译：大脑启发神经网络(SNN)已被成功地应用于许多模式识别域。基于SNN的深层结构在图像分类、目标检测等感知任务方面取得了相当大的成果。然而,深层SNN在强化学习(RL)任务中的应用仍然是一个有待探讨的问题。虽然以前曾对SNN和RL的结合进行了研究,但大多数研究的重点是浅层网络的机器人控制问题,或使用ANN-SNN的转换方法执行深Q网络(SDQN)。在这项工作中,我们从数学上分析了SDQN中弹射信号功能消失的问题,并提出了直接培训深Q网络的潜在基于层正常化(pbLN)的方法。实验显示,与状态的ANN-SNN的转换方法和其他SDQN工程相比,拟议的PbLN Spinking深Q网络(PL-SDQN)在阿塔里游戏任务上取得了更好的表现。