标题： 量子关系知识蒸馏 显示英文标题

标题： Quantum Relational Knowledge Distillation

摘要： 知识蒸馏（KD）是一种广泛采用的技术，用于将大型模型压缩成更小、更高效的学生产模型，这些模型可以在计算资源有限的设备上部署。 在各种KD方法中，关系知识蒸馏（RKD）通过对齐特征空间中的关系结构（如成对距离和角度）来提高学生模型的性能。 在本工作中，我们提出了量子关系知识蒸馏（QRKD），通过引入量子关系信息来扩展RKD。 具体来说，我们将经典特征映射到一个希尔伯特空间，将其解释为量子态，并计算量子核值以捕捉更丰富的样本间关系。 这些量子信息的关系随后用于指导蒸馏过程。 我们在视觉和语言任务上评估了QRKD，包括MNIST和CIFAR-10上的CNN，以及WikiText-2、Penn Treebank和IMDB上的GPT-2。 在所有基准测试中，与经典RKD相比，QRKD始终提高了学生模型的性能。 重要的是，教师和学生模型仍然是经典的，并且可以在标准硬件上部署，仅在训练期间需要量子计算。 这项工作首次在完全经典的部署环境中展示了量子增强的知识蒸馏。 显示英文摘要

摘要： Knowledge distillation (KD) is a widely adopted technique for compressing large models into smaller, more efficient student models that can be deployed on devices with limited computational resources. Among various KD methods, Relational Knowledge Distillation (RKD) improves student performance by aligning relational structures in the feature space, such as pairwise distances and angles. In this work, we propose Quantum Relational Knowledge Distillation (QRKD), which extends RKD by incorporating quantum relational information. Specifically, we map classical features into a Hilbert space, interpret them as quantum states, and compute quantum kernel values to capture richer inter-sample relationships. These quantum-informed relations are then used to guide the distillation process. We evaluate QRKD on both vision and language tasks, including CNNs on MNIST and CIFAR-10, and GPT-2 on WikiText-2, Penn Treebank, and IMDB. Across all benchmarks, QRKD consistently improves student model performance compared to classical RKD. Importantly, both teacher and student models remain classical and deployable on standard hardware, with quantum computation required only during training. This work presents the first demonstration of quantum-enhanced knowledge distillation in a fully classical deployment setting.