Title: SwiftTron: An Efficient Hardware Accelerator for Quantized Transformers Show Chinese title

Title: SwiftTron：一种用于量化Transformer的高效硬件加速器

Abstract: Transformers' compute-intensive operations pose enormous challenges for their deployment in resource-constrained EdgeAI / tinyML devices. As an established neural network compression technique, quantization reduces the hardware computational and memory resources. In particular, fixed-point quantization is desirable to ease the computations using lightweight blocks, like adders and multipliers, of the underlying hardware. However, deploying fully-quantized Transformers on existing general-purpose hardware, generic AI accelerators, or specialized architectures for Transformers with floating-point units might be infeasible and/or inefficient. Towards this, we propose SwiftTron, an efficient specialized hardware accelerator designed for Quantized Transformers. SwiftTron supports the execution of different types of Transformers' operations (like Attention, Softmax, GELU, and Layer Normalization) and accounts for diverse scaling factors to perform correct computations. We synthesize the complete SwiftTron architecture in a $65$ nm CMOS technology with the ASIC design flow. Our Accelerator executes the RoBERTa-base model in 1.83 ns, while consuming 33.64 mW power, and occupying an area of 273 mm^2. To ease the reproducibility, the RTL of our SwiftTron architecture is released at https://github.com/albertomarchisio/SwiftTron. Show Chinese abstract

Abstract: Transformer的计算密集型操作对其在资源受限的EdgeAI/ tinyML设备中的部署带来了巨大的挑战。 作为一种已建立的神经网络压缩技术，量化减少了硬件计算和内存资源。 特别是，定点量化有利于利用底层硬件的轻量级模块（如加法器和乘法器）来简化计算。 然而，在现有的通用硬件、通用AI加速器或具有浮点单元的专用Transformer架构上部署完全量化的Transformer可能是不可行和/或低效的。 为此，我们提出了SwiftTron，这是一种为量化Transformer设计的高效专用硬件加速器。 SwiftTron支持执行不同类型的Transformer操作（如注意力、Softmax、GELU和层归一化），并考虑了多种缩放因子以进行正确的计算。 我们在$65$nm CMOS技术中使用ASIC设计流程综合了完整的SwiftTron架构。 我们的加速器在1.83 ns内执行RoBERTa-base模型，同时消耗33.64 mW功率，并占用273 mm^2的面积。 为了便于复现，我们的SwiftTron架构的RTL已发布在https://github.com/albertomarchisio/SwiftTron。