标题： 记忆到泛化：从联想记忆中出现的扩散模型 显示英文标题

标题： Memorization to Generalization: Emergence of Diffusion Models from Associative Memory

摘要： 霍普菲尔德网络是联想记忆（AM）系统，设计用于将模式存储和检索为能量景观的局部最小值。 在经典的霍普菲尔德模型中，当训练数据量达到其临界记忆负载$- spurious\,\,states$时，会发生一种有趣的现象，或者在检索动力学结束时出现非预期的稳定点，导致错误的回忆。 在本工作中，我们从AM的角度审视扩散模型，扩散模型通常用于生成建模。 扩散模型的训练阶段被概念化为记忆编码（训练数据存储在记忆中）。 生成阶段被视为尝试记忆检索。 在小数据情况下，扩散模型表现出强烈的记忆化阶段，其中网络在训练集中的每个样本周围创建不同的吸引盆地，类似于临界记忆负载以下的霍普菲尔德模型。 在大数据情况下，会出现不同的阶段，训练集大小的增加促进了对应于生成样本流形的新吸引子状态的创建。 虚假状态出现在这一过渡的边界处，并对应于出现的吸引子状态，这些状态在训练集中不存在，但同时在它们周围有明显的吸引盆地。 我们的发现提供了：通过AM的视角对扩散模型中的记忆-泛化现象的新观点，虚假状态存在的理论预测，以及在常用扩散模型中对该预测的经验验证。 显示英文摘要

摘要： Hopfield networks are associative memory (AM) systems, designed for storing and retrieving patterns as local minima of an energy landscape. In the classical Hopfield model, an interesting phenomenon occurs when the amount of training data reaches its critical memory load $- spurious\,\,states$, or unintended stable points, emerge at the end of the retrieval dynamics, leading to incorrect recall. In this work, we examine diffusion models, commonly used in generative modeling, from the perspective of AMs. The training phase of diffusion model is conceptualized as memory encoding (training data is stored in the memory). The generation phase is viewed as an attempt of memory retrieval. In the small data regime the diffusion model exhibits a strong memorization phase, where the network creates distinct basins of attraction around each sample in the training set, akin to the Hopfield model below the critical memory load. In the large data regime, a different phase appears where an increase in the size of the training set fosters the creation of new attractor states that correspond to manifolds of the generated samples. Spurious states appear at the boundary of this transition and correspond to emergent attractor states, which are absent in the training set, but, at the same time, have distinct basins of attraction around them. Our findings provide: a novel perspective on the memorization-generalization phenomenon in diffusion models via the lens of AMs, theoretical prediction of existence of spurious states, empirical validation of this prediction in commonly-used diffusion models.