Title: Normalized Convolutional Neural Network Show Chinese title

Title: 归一化卷积神经网络

Abstract: We introduce a Normalized Convolutional Neural Layer, a novel approach to normalization in convolutional networks. Unlike conventional methods, this layer normalizes the rows of the im2col matrix during convolution, making it inherently adaptive to sliced inputs and better aligned with kernel structures. This distinctive approach differentiates it from standard normalization techniques and prevents direct integration into existing deep learning frameworks optimized for traditional convolution operations. Our method has a universal property, making it applicable to any deep learning task involving convolutional layers. By inherently normalizing within the convolution process, it serves as a convolutional adaptation of Self-Normalizing Networks, maintaining their core principles without requiring additional normalization layers. Notably, in micro-batch training scenarios, it consistently outperforms other batch-independent normalization methods. This performance boost arises from standardizing the rows of the im2col matrix, which theoretically leads to a smoother loss gradient and improved training stability. Show Chinese abstract

Abstract: 我们引入了一种归一化的卷积神经层，这是一种在卷积网络中进行归一化的新方法。与传统方法不同，该层在卷积过程中对im2col矩阵的行进行归一化，使其本质上适应切片输入，并更好地与核结构对齐。这种独特的做法使其区别于标准的归一化技术，并且不能直接集成到现有的优化用于传统卷积操作的深度学习框架中。我们的方法具有普遍性，适用于任何涉及卷积层的深度学习任务。通过在卷积过程中内在地进行归一化，它作为自归一化网络的一种卷积适应方式，在不增加额外归一化层的情况下保持了其核心原则。值得注意的是，在微批次训练场景中，它始终优于其他批次独立的归一化方法。这种性能提升来源于对im2col矩阵行的标准处理，理论上导致更平滑的损失梯度和改进的训练稳定性。