标题： 知识蒸馏在设计计算成像系统中的应用 显示英文标题

标题： Distilling Knowledge for Designing Computational Imaging Systems

摘要： 设计物理编码器对于计算成像（CI）系统中的准确图像重建至关重要。 目前，这些系统是通过端到端（E2E）优化来设计的，其中编码器被建模为神经网络层，并与解码器联合优化。 然而，由于对编码器施加了物理约束，E2E优化的性能会显著降低。 此外，由于E2E通过反向传播重建误差来学习编码器的参数，它不会促进最优的中间输出，并且会受到梯度消失的影响。 为了解决这些限制，我们通过将预训练的、约束较少的CI系统的知识进行迁移，重新诠释了知识蒸馏（KD）的概念，以设计一个物理约束的CI系统。 我们的方法包括三个步骤： (1) 给定原始CI系统（学生），通过放松对学生编码器的约束来创建一个教师系统。 (2) 教师被优化以解决学生问题的一个约束较少的版本。 (3) 通过两个提出的知识转移函数，教师指导学生的训练，针对编码器和解码器特征空间。 所提出的方法可以应用于任何成像模式，因为放松方案和损失函数可以根据物理采集和使用的解码器进行调整。 该方法在三种典型的CI模式上进行了验证：磁共振、单像素和压缩光谱成像。 模拟结果显示，具有与学生编码器结构相似的编码器的教师系统能提供有效的指导。 我们的方法在重建性能和编码器设计方面都显著优于E2E优化和传统的非数据驱动的编码器设计。 显示英文摘要

摘要： Designing the physical encoder is crucial for accurate image reconstruction in computational imaging (CI) systems. Currently, these systems are designed via end-to-end (E2E) optimization, where the encoder is modeled as a neural network layer and is jointly optimized with the decoder. However, the performance of E2E optimization is significantly reduced by the physical constraints imposed on the encoder. Also, since the E2E learns the parameters of the encoder by backpropagating the reconstruction error, it does not promote optimal intermediate outputs and suffers from gradient vanishing. To address these limitations, we reinterpret the concept of knowledge distillation (KD) for designing a physically constrained CI system by transferring the knowledge of a pretrained, less-constrained CI system. Our approach involves three steps: (1) Given the original CI system (student), a teacher system is created by relaxing the constraints on the student's encoder. (2) The teacher is optimized to solve a less-constrained version of the student's problem. (3) The teacher guides the training of the student through two proposed knowledge transfer functions, targeting both the encoder and the decoder feature space. The proposed method can be employed to any imaging modality since the relaxation scheme and the loss functions can be adapted according to the physical acquisition and the employed decoder. This approach was validated on three representative CI modalities: magnetic resonance, single-pixel, and compressive spectral imaging. Simulations show that a teacher system with an encoder that has a structure similar to that of the student encoder provides effective guidance. Our approach achieves significantly improved reconstruction performance and encoder design, outperforming both E2E optimization and traditional non-data-driven encoder designs.