Title: In situ fine-tuning of in silico trained Optical Neural Networks Show Chinese title

Title: 原位微调在体训练的光学神经网络

Abstract: Optical Neural Networks (ONNs) promise significant advantages over traditional electronic neural networks, including ultrafast computation, high bandwidth, and low energy consumption, by leveraging the intrinsic capabilities of photonics. However, training ONNs poses unique challenges, notably the reliance on simplified in silico models whose trained parameters must subsequently be mapped to physical hardware. This process often introduces inaccuracies due to discrepancies between the idealized digital model and the physical ONN implementation, particularly stemming from noise and fabrication imperfections. In this paper, we analyze how noise misspecification during in silico training impacts ONN performance and we introduce Gradient-Informed Fine-Tuning (GIFT), a lightweight algorithm designed to mitigate this performance degradation. GIFT uses gradient information derived from the noise structure of the ONN to adapt pretrained parameters directly in situ, without requiring expensive retraining or complex experimental setups. GIFT comes with formal conditions under which it improves ONN performance. We also demonstrate the effectiveness of GIFT via simulation on a five-layer feed forward ONN trained on the MNIST digit classification task. GIFT achieves up to $28\%$ relative accuracy improvement compared to the baseline performance under noise misspecification, without resorting to costly retraining. Overall, GIFT provides a practical solution for bridging the gap between simplified digital models and real-world ONN implementations. Show Chinese abstract

Abstract: 光学神经网络（ONNs）通过利用光子学的固有特性，相较于传统的电子神经网络具有显著优势，包括超快计算、高带宽和低能耗。 然而，训练ONNs面临独特的挑战，特别是依赖于简化的仿真模型，这些模型训练后的参数随后必须映射到物理硬件上。 这一过程通常会因理想化的数字模型与物理ONN实现之间的差异而引入不准确性，尤其是由于噪声和制造缺陷引起的差异。 在本文中，我们分析了在仿真训练过程中噪声误设对ONN性能的影响，并引入了梯度感知微调（GIFT），这是一种轻量级算法，旨在缓解这种性能下降。 GIFT利用来自ONN噪声结构的梯度信息，在不需昂贵重新训练或复杂实验设置的情况下，直接在现场调整预训练参数。 GIFT附带有正式条件，表明在这些条件下可以提高ONN性能。 我们还通过在五层前馈ONN上的仿真展示了GIFT的有效性，该ONN在MNIST数字分类任务上进行了训练。 在噪声误设情况下，GIFT相对于基线性能实现了高达$28\%$的相对准确率提升，而无需 resorting 到昂贵的重新训练。 总体而言，GIFT为弥合简化数字模型与实际ONN实现之间的差距提供了一个实用的解决方案。