Title: The Persian Rug: solving toy models of superposition using large-scale symmetries Show Chinese title

Title: 波斯地毯：使用大规模对称性解决叠加的玩具模型

Abstract: We present a complete mechanistic description of the algorithm learned by a minimal non-linear sparse data autoencoder in the limit of large input dimension. The model, originally presented in arXiv:2209.10652, compresses sparse data vectors through a linear layer and decompresses using another linear layer followed by a ReLU activation. We notice that when the data is permutation symmetric (no input feature is privileged) large models reliably learn an algorithm that is sensitive to individual weights only through their large-scale statistics. For these models, the loss function becomes analytically tractable. Using this understanding, we give the explicit scalings of the loss at high sparsity, and show that the model is near-optimal among recently proposed architectures. In particular, changing or adding to the activation function any elementwise or filtering operation can at best improve the model's performance by a constant factor. Finally, we forward-engineer a model with the requisite symmetries and show that its loss precisely matches that of the trained models. Unlike the trained model weights, the low randomness in the artificial weights results in miraculous fractal structures resembling a Persian rug, to which the algorithm is oblivious. Our work contributes to neural network interpretability by introducing techniques for understanding the structure of autoencoders. Code to reproduce our results can be found at https://github.com/KfirD/PersianRug . Show Chinese abstract

Abstract: 我们提供了一个最小非线性稀疏数据自编码器在输入维度大的极限下的算法的完整机制描述。 该模型最初在arXiv:2209.10652中提出，通过一个线性层压缩稀疏数据向量，并使用另一个线性层后接ReLU激活函数进行解压缩。 我们注意到，当数据是排列对称的（没有输入特征是优先的）时，大模型可靠地学习一种仅通过其大规模统计量对单个权重敏感的算法。 对于这些模型，损失函数变得解析可处理。 利用这种理解，我们给出了高稀疏性下损失的显式缩放关系，并表明该模型在最近提出的架构中接近最优。 特别是，对激活函数进行任何逐元素或过滤操作的更改或添加，最多只能以常数因子改善模型性能。 最后，我们正向工程了一个具有必要对称性的模型，并表明其损失精确匹配训练模型的损失。 与训练模型的权重不同，人工权重中的低随机性导致了类似于波斯地毯的奇迹般的分形结构，而算法对此是无意识的。 我们的工作通过引入理解自编码器结构的技术，为神经网络的可解释性做出了贡献。 可以找到重现我们结果的代码https://github.com/KfirD/PersianRug 。