Title: Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models Show Chinese title

Title: 深度学习的随机矩阵理论：超越线性模型的特征值

Abstract: Modern Machine Learning (ML) and Deep Neural Networks (DNNs) often operate on high-dimensional data and rely on overparameterized models, where classical low-dimensional intuitions break down. In particular, the proportional regime where the data dimension, sample size, and number of model parameters are all large and comparable, gives rise to novel and sometimes counterintuitive behaviors. This paper extends traditional Random Matrix Theory (RMT) beyond eigenvalue-based analysis of linear models to address the challenges posed by nonlinear ML models such as DNNs in this regime. We introduce the concept of High-dimensional Equivalent, which unifies and generalizes both Deterministic Equivalent and Linear Equivalent, to systematically address three technical challenges: high dimensionality, nonlinearity, and the need to analyze generic eigenspectral functionals. Leveraging this framework, we provide precise characterizations of the training and generalization performance of linear models, nonlinear shallow networks, and deep networks. Our results capture rich phenomena, including scaling laws, double descent, and nonlinear learning dynamics, offering a unified perspective on the theoretical understanding of deep learning in high dimensions. Show Chinese abstract

Abstract: 现代机器学习（ML）和深度神经网络（DNNs）通常处理高维数据，并依赖于过参数化模型，在这种情况下，经典的低维直觉失效。 特别是，在数据维度、样本量和模型参数数量都较大且可比较的比例域中，会出现新的、有时违反直觉的行为。 本文将传统的随机矩阵理论（RMT）从线性模型的基于特征值的分析扩展到非线性机器学习模型（如DNNs）所提出的挑战。 我们引入了高维等效性的概念，它统一并推广了确定性等效性和线性等效性，以系统地解决三个技术挑战：高维性、非线性和需要分析通用特征谱泛函的需求。 利用这一框架，我们对线性模型、非线性浅层网络和深层网络的训练和泛化性能提供了精确的描述。 我们的结果捕获了丰富的现象，包括缩放规律、双重下降和非线性学习动力学，为高维下深度学习的理论理解提供了统一的视角。