标题： 光子-ViT：一种基于硅光子学的近传感器感兴趣区域感知视觉变换器加速器 显示英文标题

标题： Opto-ViT: Architecting a Near-Sensor Region of Interest-Aware Vision Transformer Accelerator with Silicon Photonics

摘要： 视觉变压器（ViTs）由于其能够建模长距离依赖关系和全局上下文关系，已成为计算机视觉任务的强大架构。 然而，它们巨大的计算和内存需求阻碍了在具有严格能耗和带宽限制的场景中的高效部署。 在本工作中，我们提出了OptoViT，这是第一个近传感器、区域感知的ViT加速器，利用硅光子学（SiPh）实现实时且节能的视觉处理。 Opto-ViT具有混合电子-光子架构，其中光学核心使用垂直腔面发射激光器（VCSELs）和微环谐振器（MRs）执行计算密集型矩阵乘法，而非线性函数和归一化则通过电子方式执行。 为了减少冗余计算和块处理，我们引入了一个轻量级的掩码生成网络（MGNet），该网络识别当前帧中的感兴趣区域，并在ViT编码之前修剪无关的块。 我们进一步通过针对光子约束量化的训练和矩阵分解对ViT主干进行共同优化。 在器件制造、电路和架构协同设计，到分类、检测和视频任务的实验表明，OptoViT在准确率损失小于1.6%的情况下实现了高达84%的能耗节省，达到100.4 KFPS/W的性能，同时实现了边缘处可扩展且高效的ViT部署。 显示英文摘要

摘要： Vision Transformers (ViTs) have emerged as a powerful architecture for computer vision tasks due to their ability to model long-range dependencies and global contextual relationships. However, their substantial compute and memory demands hinder efficient deployment in scenarios with strict energy and bandwidth limitations. In this work, we propose OptoViT, the first near-sensor, region-aware ViT accelerator leveraging silicon photonics (SiPh) for real-time and energy-efficient vision processing. Opto-ViT features a hybrid electronic-photonic architecture, where the optical core handles compute-intensive matrix multiplications using Vertical-Cavity Surface-Emitting Lasers (VCSELs) and Microring Resonators (MRs), while nonlinear functions and normalization are executed electronically. To reduce redundant computation and patch processing, we introduce a lightweight Mask Generation Network (MGNet) that identifies regions of interest in the current frame and prunes irrelevant patches before ViT encoding. We further co-optimize the ViT backbone using quantization-aware training and matrix decomposition tailored for photonic constraints. Experiments across device fabrication, circuit and architecture co-design, to classification, detection, and video tasks demonstrate that OptoViT achieves 100.4 KFPS/W with up to 84% energy savings with less than 1.6% accuracy loss, while enabling scalable and efficient ViT deployment at the edge.