标题： 基于心室阵列的的状态空间模型加速器 显示英文标题

标题： Systolic Array-based Accelerator for State-Space Models

摘要： 序列建模对于人工智能理解时间数据和检测复杂的时间依赖模式至关重要。 虽然循环神经网络（RNNs）、卷积神经网络（CNNs）和Transformer在捕捉长距离依赖关系方面取得了进展，但由于有限的记忆保持能力（固定上下文窗口），它们在处理非常长的序列时难以实现高精度。 状态空间模型（SSMs）利用指数衰减的记忆，使长时间的上下文窗口成为可能，因此它们比循环和基于Transformer的模型更高效地处理非常长的数据序列。 与传统的神经网络模型如CNNs和RNNs不同，基于SSM的模型需要通过连续积分求解微分方程，这使得训练和推理在传统CPU和GPU上既计算密集又内存密集。 在本文中，我们介绍了一种专用硬件加速器EpochCore，用于加速SSMs。 EpochCore基于脉动阵列（SAs），旨在提高基于SSM的模型在长距离序列任务中的能效和吞吐量。 在脉动阵列中，我们提出了一种称为LIMA-PE的多功能处理单元（PE），用于执行传统的和专门的乘加（MAC）操作，以支持传统的DNNs和SSMs。 为了补充EpochCore的微架构，我们提出了一种新的数据流ProDF，它能够高效执行基于SSM的模型。 通过利用LIMA-PE微架构和ProDF，EpochCore在性能上平均提高了250倍，在能效上提高了45倍，相对于传统的基于SA的加速器，面积成本增加了2倍，并且在LRA数据集上的延迟/推理方面相比GPU内核操作提高了约2000倍。 显示英文摘要

摘要： Sequence modeling is crucial for AI to understand temporal data and detect complex time-dependent patterns. While recurrent neural networks (RNNs), convolutional neural networks (CNNs), and Transformers have advanced in capturing long-range dependencies, they struggle with achieving high accuracy with very long sequences due to limited memory retention (fixed context window). State-Space Models (SSMs) leverage exponentially decaying memory enabling lengthy context window and so they process very long data sequences more efficiently than recurrent and Transformer-based models. Unlike traditional neural models like CNNs and RNNs, SSM-based models require solving differential equations through continuous integration, making training and inference both compute- and memory-intensive on conventional CPUs and GPUs. In this paper we introduce a specialized hardware accelerator, EpochCore, for accelerating SSMs. EpochCore is based on systolic arrays (SAs) and is designed to enhance the energy efficiency and throughput of inference of SSM-based models for long-range sequence tasks. Within the SA, we propose a versatile processing element (PE) called LIMA-PE to perform traditional and specialized MAC operations to support traditional DNNs and SSMs. To complement the EpochCore microarchitecture, we propose a novel dataflow, ProDF, which enables highly efficient execution of SSM-based models. By leveraging the LIMA-PE microarchitecture and ProDF, EpochCore achieves on average 250x gains in performance and 45x improvement in energy efficiency, at the expense of 2x increase in area cost over traditional SA-based accelerators, and around ~2,000x improvement in latency/inference on LRA datasets compared to GPU kernel operations.