标题： 探索RBMs的能量景观：关于玻色子、分层学习和对称性破缺的倒空间见解 显示英文标题

标题： Exploring the Energy Landscape of RBMs: Reciprocal Space Insights into Bosons, Hierarchical Learning and Symmetry Breaking

摘要： 深度生成模型由于其能够学习并从复杂分布中采样的能力而变得无处不在。 尽管存在各种框架，这些模型之间的关系仍然 largely 未被探索，这一差距阻碍了人工智能学习统一理论的发展。 我们解决了两个核心挑战：阐明不同深度生成模型之间的联系，并加深对它们学习机制的理解。 我们专注于受限玻尔兹曼机（RBMs），它以对离散分布的通用逼近能力而闻名。 通过引入倒数空间公式，我们揭示了 RBMs、扩散过程和耦合玻色子之间的联系。 我们表明，在初始化时，RBM 处于一个鞍点，其中局部曲率由奇异值决定，其分布遵循 Marcenko-Pastur 定律并表现出旋转对称性。 在训练过程中，由于分层学习，旋转对称性被打破，不同的自由度逐步捕捉多层次的抽象特征。 这导致能量景观中的对称性破缺，类似于朗道理论。 能量景观中的这种对称性破缺由奇异值和权重矩阵特征向量矩阵表征。 我们在平均场近似下推导了相应的自由能。 我们表明，在无限大小 RBM 的极限情况下，倒数变量是高斯分布的。 我们的发现表明，在这种情况下，某些模式的扩散过程将不会收敛到玻尔兹曼分布。 为了说明我们的结果，我们使用 MNIST 数据集训练了具有不同隐藏层大小的 RBM 副本。 我们的发现弥合了不同生成框架之间的差距，并且也揭示了生成模型中学习过程的机制。 显示英文摘要

摘要： Deep generative models have become ubiquitous due to their ability to learn and sample from complex distributions. Despite the proliferation of various frameworks, the relationships among these models remain largely unexplored, a gap that hinders the development of a unified theory of AI learning. We address two central challenges: clarifying the connections between different deep generative models and deepening our understanding of their learning mechanisms. We focus on Restricted Boltzmann Machines (RBMs), known for their universal approximation capabilities for discrete distributions. By introducing a reciprocal space formulation, we reveal a connection between RBMs, diffusion processes, and coupled Bosons. We show that at initialization, the RBM operates at a saddle point, where the local curvature is determined by the singular values, whose distribution follows the Marcenko-Pastur law and exhibits rotational symmetry. During training, this rotational symmetry is broken due to hierarchical learning, where different degrees of freedom progressively capture features at multiple levels of abstraction. This leads to a symmetry breaking in the energy landscape, reminiscent of Landau theory. This symmetry breaking in the energy landscape is characterized by the singular values and the weight matrix eigenvector matrix. We derive the corresponding free energy in a mean-field approximation. We show that in the limit of infinite size RBM, the reciprocal variables are Gaussian distributed. Our findings indicate that in this regime, there will be some modes for which the diffusion process will not converge to the Boltzmann distribution. To illustrate our results, we trained replicas of RBMs with different hidden layer sizes using the MNIST dataset. Our findings bridge the gap between disparate generative frameworks and also shed light on the processes underpinning learning in generative models.