标题： 通过学习记忆来寻找在线神经更新规则 显示英文标题

标题： Finding online neural update rules by learning to remember

摘要： 我们从头开始研究神经激活（身体）和权重（突触）的在线局部更新规则的学习。 我们用小向量表示每个权重和激活的状态，并使用（元）神经网络对它们的更新进行参数化。 不同的神经元类型通过不同的嵌入向量表示，这使得可以对所有神经元使用相同的两个函数。 我们提出并研究了一个不同的目标：即记住过去的体验片段，而不是像在类似系统中那样直接使用进化或长期反向传播进行训练。 我们解释了这个目标如何与标准的反向传播训练和其他形式的学习相关。 我们使用短期反向传播来训练这个目标，并分析性能作为不同网络类型和问题难度的函数。 我们发现，这种分析提供了关于构成学习规则的有趣见解。 我们还讨论了这样的系统如何形成解决诸如情景记忆、元学习和辅助目标等问题的自然基础。 显示英文摘要

摘要： We investigate learning of the online local update rules for neural activations (bodies) and weights (synapses) from scratch. We represent the states of each weight and activation by small vectors, and parameterize their updates using (meta-) neural networks. Different neuron types are represented by different embedding vectors which allows the same two functions to be used for all neurons. Instead of training directly for the objective using evolution or long term back-propagation, as is commonly done in similar systems, we motivate and study a different objective: That of remembering past snippets of experience. We explain how this objective relates to standard back-propagation training and other forms of learning. We train for this objective using short term back-propagation and analyze the performance as a function of both the different network types and the difficulty of the problem. We find that this analysis gives interesting insights onto what constitutes a learning rule. We also discuss how such system could form a natural substrate for addressing topics such as episodic memories, meta-learning and auxiliary objectives.