标题： 对于随机特征模型的一比特量化 显示英文标题

标题： One-Bit Quantization for Random Features Models

摘要： 最近神经网络的进展导致了显著的计算和内存需求，促使人们关注一比特权重压缩，以在资源受限设备上实现高效的推理。 然而，这种压缩的理论基础仍然不甚明了。 我们通过分析随机特征模型中的一个比特量化来解决这一问题，这是一个简化的框架，对应于具有随机表示的神经网络。 我们证明，从渐近角度看，除了最后一层之外，对所有层的权重进行量化不会导致泛化误差的损失，与全精度随机特征模型相比。 我们的研究结果为神经网络压缩提供了理论见解。 我们还通过实验证明，即使在笔记本电脑的GPU上，一比特量化也能显著提高随机特征模型的推理速度，证实了我们工作的实际好处。 此外，我们提供了对具有任意层数的随机特征模型的泛化误差的渐近精确描述。 据我们所知，我们的分析比相关文献中所有先前的工作得出了更普遍的结果。 显示英文摘要

摘要： Recent advances in neural networks have led to significant computational and memory demands, spurring interest in one-bit weight compression to enable efficient inference on resource-constrained devices. However, the theoretical underpinnings of such compression remain poorly understood. We address this gap by analyzing one-bit quantization in the Random Features model, a simplified framework that corresponds to neural networks with random representations. We prove that, asymptotically, quantizing weights of all layers except the last incurs no loss in generalization error, compared to the full precision random features model. Our findings offer theoretical insights into neural network compression. We also demonstrate empirically that one-bit quantization leads to significant inference speed ups for the Random Features models even on a laptop GPU, confirming the practical benefits of our work. Additionally, we provide an asymptotically precise characterization of the generalization error for Random Features with an arbitrary number of layers. To the best of our knowledge, our analysis yields more general results than all previous works in the related literature.