Title: Statistically guided deep learning Show Chinese title

Title: 统计引导的深度学习

Abstract: We present a theoretically well-founded deep learning algorithm for nonparametric regression. It uses over-parametrized deep neural networks with logistic activation function, which are fitted to the given data via gradient descent. We propose a special topology of these networks, a special random initialization of the weights, and a data-dependent choice of the learning rate and the number of gradient descent steps. We prove a theoretical bound on the expected $L_2$ error of this estimate, and illustrate its finite sample size performance by applying it to simulated data. Our results show that a theoretical analysis of deep learning which takes into account simultaneously optimization, generalization and approximation can result in a new deep learning estimate which has an improved finite sample performance. Show Chinese abstract

Abstract: 我们提出了一种理论上基础扎实的深度学习算法，用于非参数回归。 该算法使用带有逻辑激活函数的过参数化深度神经网络，并通过梯度下降法拟合给定数据。 我们提出了这些网络的一种特殊拓扑结构、权重的一种特殊随机初始化方法，以及一种基于数据的学习率和梯度下降步数的选择策略。 我们证明了该估计值预期$L_2$误差的一个理论界限，并通过将其应用于模拟数据来展示其有限样本规模下的性能。 我们的结果显示，同时考虑优化、泛化和逼近的深度学习理论分析可以产生一种改进有限样本性能的新深度学习估计方法。