标题： OLAF：异步分布式强化学习的可编程数据平面加速 显示英文标题

标题： OLAF: Programmable Data Plane Acceleration for Asynchronous Distributed Reinforcement Learning

摘要： 异步分布式强化学习（DRL）在模型更新变得过时时可能会出现收敛性能下降，这通常是大规模训练期间网络拥塞和数据包丢失的结果。 这项工作引入了一种网络数据平面加速架构，通过在加速器引擎中对DRL模型更新进行内联处理来缓解这种过时问题。 为此，我们设计并原型化了一种新的队列机制，机会性地结合共享同一网络元素的兼容更新，减少冗余流量并保留更新效用。 作为补充，我们在工作节点上提供了一个轻量级传输控制机制，该机制由网络内的加速器反馈引导。 为了以线速评估模型效用，我们引入了模型年龄（AoM）指标作为过时的代理，并使用形式化验证方法验证全局公平性和响应性特性。 我们的评估结果表明，这种架构显著减少了更新过时和拥塞，最终提高了异步DRL工作负载的收敛速度。 显示英文摘要

摘要： Asynchronous Distributed Reinforcement Learning (DRL) can suffer from degraded convergence when model updates become stale, often the result of network congestion and packet loss during large-scale training. This work introduces a network data-plane acceleration architecture that mitigates such staleness by enabling inline processing of DRL model updates as they traverse the accelerator engine. To this end, we design and prototype a novel queueing mechanism that opportunistically combines compatible updates sharing a network element, reducing redundant traffic and preserving update utility. Complementing this we provide a lightweight transmission control mechanism at the worker nodes that is guided by feedback from the in-network accelerator. To assess model utility at line rate, we introduce the Age-of-Model (AoM) metric as a proxy for staleness and verify global fairness and responsiveness properties using a formal verification method. Our evaluations demonstrate that this architecture significantly reduces update staleness and congestion, ultimately improving the convergence rate in asynchronous DRL workloads.