标题： 蜂鸟：一种在嵌入式FPGA上的更小更快的大型语言模型加速器 显示英文标题

标题： Hummingbird: A Smaller and Faster Large Language Model Accelerator on Embedded FPGA

摘要： 在嵌入式设备上部署大型语言模型（LLMs）由于LLMs的高计算和内存需求以及此类环境中的有限硬件资源，仍然是一个重要的研究挑战。 尽管嵌入式FPGA在传统深度神经网络中表现出性能和能效，但其在LLM推理方面的潜力尚未得到充分探索。 最近将LLMs部署到FPGAs的努力主要依赖于大型且昂贵的云级硬件，并且仅在相对较小的LLMs上显示出有希望的结果，限制了它们的实际应用。 在本工作中，我们提出了Hummingbird，这是一种专为嵌入式FPGAs上的LLM推理设计的新型FPGA加速器。 Hummingbird更小，针对如KV260和ZCU104这样的嵌入式FPGA，相比现有研究分别实现了67%的LUT、39%的DSP和42%的功耗节省。 Hummingbird更强，针对LLaMA3-8B并支持更长的上下文，通过卸载策略克服了嵌入式FPGAs通常的4GB内存限制。 最后， Hummingbird更快，在KV260和ZCU104上分别实现了4.8个标记/秒和8.6个标记/秒的速度，模型带宽利用率达到93-94%，超过了先前LLaMA2-7B的4.9个标记/秒和84%带宽利用率基线。 我们进一步通过在成本优化的Spartan UltraScale FPGA上部署Hummingbird，证明了工业应用的可行性，为边缘端提供了负担得起的LLM解决方案。 显示英文摘要

摘要： Deploying large language models (LLMs) on embedded devices remains a significant research challenge due to the high computational and memory demands of LLMs and the limited hardware resources available in such environments. While embedded FPGAs have demonstrated performance and energy efficiency in traditional deep neural networks, their potential for LLM inference remains largely unexplored. Recent efforts to deploy LLMs on FPGAs have primarily relied on large, expensive cloud-grade hardware and have only shown promising results on relatively small LLMs, limiting their real-world applicability. In this work, we present Hummingbird, a novel FPGA accelerator designed specifically for LLM inference on embedded FPGAs. Hummingbird is smaller, targeting embedded FPGAs such as the KV260 and ZCU104 with 67% LUT, 39% DSP, and 42% power savings over existing research. Hummingbird is stronger, targeting LLaMA3-8B and supporting longer contexts, overcoming the typical 4GB memory constraint of embedded FPGAs through offloading strategies. Finally, Hummingbird is faste, achieving 4.8 tokens/s and 8.6 tokens/s for LLaMA3-8B on the KV260 and ZCU104 respectively, with 93-94% model bandwidth utilization, outperforming the prior 4.9 token/s for LLaMA2-7B with 84% bandwidth utilization baseline. We further demonstrate the viability of industrial applications by deploying Hummingbird on a cost-optimized Spartan UltraScale FPGA, paving the way for affordable LLM solutions at the edge.