标题： 基于三值神经元的尖峰深度Q学习性能提升 显示英文标题

标题： Improving Performance of Spike-based Deep Q-Learning using Ternary Neurons

摘要： 我们提出了一种新的三值尖峰神经元模型，以提高二值尖峰神经元在深度Q学习中的表示能力。尽管最近引入了一种三值神经元模型来克服二值尖峰神经元提供的有限表示能力，但我们表明其性能在深度Q学习任务中不如二值模型。我们假设训练过程中的梯度估计偏差是潜在的根本原因，并通过数学和经验分析证明了这一点。我们提出了一种新颖的三值尖峰神经元模型，通过减少估计偏差来缓解此问题。我们将所提出的三值尖峰神经元作为深度尖峰Q学习网络（DSQN）的基本计算单元，并评估该网络在Gym环境中七个Atari游戏中的性能。结果显示，所提出的三值尖峰神经元缓解了三值神经元在Q学习任务中的性能急剧下降问题，并且与现有的二值神经元相比提高了网络性能，使DSQN成为车载自主决策任务的更实用解决方案。 显示英文摘要

摘要： We propose a new ternary spiking neuron model to improve the representation capacity of binary spiking neurons in deep Q-learning. Although a ternary neuron model has recently been introduced to overcome the limited representation capacity offered by the binary spiking neurons, we show that its performance is worse than that of binary models in deep Q-learning tasks. We hypothesize gradient estimation bias during the training process as the underlying potential cause through mathematical and empirical analysis. We propose a novel ternary spiking neuron model to mitigate this issue by reducing the estimation bias. We use the proposed ternary spiking neuron as the fundamental computing unit in a deep spiking Q-learning network (DSQN) and evaluate the network's performance in seven Atari games from the Gym environment. Results show that the proposed ternary spiking neuron mitigates the drastic performance degradation of ternary neurons in Q-learning tasks and improves the network performance compared to the existing binary neurons, making DSQN a more practical solution for on-board autonomous decision-making tasks.