标题： 面向云大语言模型推理中实体碳摊销的衰老感知CPU核心管理 显示英文标题

标题： Aging-aware CPU Core Management for Embodied Carbon Amortization in Cloud LLM Inference

摘要： 大规模语言模型（LLM）的广泛采用要求快速扩展云LLM推理集群，导致了嵌入碳排放$-$的积累，这些排放来自制造和供应IT资产$-$，主要集中在推理服务器CPU。 本文深入探讨了云LLM推理可持续增长的挑战，强调在延长使用寿命的情况下对CPU嵌入碳的摊销。 鉴于硅老化带来的可靠性风险，我们提出了一种考虑老化的CPU核心管理技术，以延迟CPU老化效应，使集群操作员能够安全地延长CPU寿命。 我们的技术利用了我们在云LLM推理中发现的CPU利用率不足模式，通过选择性深度闲置和考虑老化的推理任务分配，停止未使用核心的老化并平衡活跃核心的老化。 通过使用真实世界的Azure推理痕迹和微软扩展的LLM集群模拟器进行大量仿真，我们展示了我们的技术在现有方法上的优越性能，通过管理CPU老化效应的p99性能，预计每年嵌入碳排放减少37.67%，CPU利用率不足减少77%，对推理服务质量的影响不到10%。 显示英文摘要

摘要： Broad adoption of Large Language Models (LLM) demands rapid expansions of cloud LLM inference clusters, leading to accumulation of embodied carbon$-$the emissions from manufacturing and supplying IT assets$-$that mostly concentrate on inference server CPU. This paper delves into the challenges of sustainable growth of cloud LLM inference, emphasizing extended amortization of CPU embodied over an increased lifespan. Given the reliability risks of silicon aging, we propose an aging-aware CPU core management technique to delay CPU aging effects, allowing the cluster operator to safely increase CPU life. Our technique exploits CPU underutilization patterns that we uncover in cloud LLM inference by halting aging in unused cores and even-outing aging in active cores via selective deep idling and aging-aware inference task allocation. Through extensive simulations using real-world Azure inference traces and an extended LLM cluster simulator from Microsoft, we show superior performance of our technique over existing methods with an estimated 37.67\% reduction in yearly embodied carbon emissions through p99 performance of managing CPU aging effects, a 77\% reduction in CPU underutilization, and less than 10\% impact to the inference service quality.