Title: Cyrus+: A DRL-based Puncturing Solution to URLLC/eMBB Multiplexing in O-RAN Show Chinese title

Title: Cyrus+：一种基于DRL的O-RAN中URLLC/eMBB复用穿孔解决方案

Abstract: Puncturing is a promising technique in 3GPP to multiplex Enhanced Mobile Broadband (eMBB) and Ultra-Reliable Low Latency Communications (URLLC) traffic on the same 5G New Radio (NR) air interface. The essence of puncturing is to transmit URLLC packets on demand upon their arrival, by preempting the radio resources (or subcarriers) that are already allocated to eMBB traffic. Although it is considered most bandwidth efficient, puncturing URLLC data on eMBB can lead to degradation of eMBB's performance. Most of the state-of-the-art research addressing this problem employ raw eMBB data throughput as performance metric. This is inadequate as, after puncturing, eMBB data may or may not be successfully decoded at its receiver. This paper presents Cyrus+, a deep reinforcement learning (DRL)-based puncturing solution that employs goodput (through feedback from a receiver's decoder), rather than estimated raw throughput, in its design of reward function. Further, Cyrus+ is tailored specifically for the Open RAN (O-RAN) architecture and fully leverages O-RAN's three control loops at different time scales in its design of DRL. In the Non-Real-Time (Non-RT) RAN Intelligent Controller (RIC), Cyrus+ initializes the policy network that will be used in the RT Open Distributed Unit (O-DU). In the Near-RT RIC, Cyrus+ refines the policy based on dynamic network conditions and feedback from the receivers. In the RT O-DU, Cyrus+ generates a puncturing codebook by considering all possible URLLC arrivals. We build a standard-compliant link-level 5G NR simulator to demonstrate the efficacy of Cyrus+. Experimental results show that Cyrus+ outperforms benchmark puncturing algorithms and meets the stringent timing requirement in 5G NR (numerology 3). Show Chinese abstract

Abstract: 在3GPP中，穿孔（puncturing）是一种很有前景的技术，可以在同一5G新无线（NR）空中接口上复用增强型移动宽带（eMBB）和超高可靠低时延通信（URLLC）业务。穿孔的本质是在URLLC数据包到达时，通过抢占已经分配给eMBB业务的无线电资源（或子载波），按需传输URLLC数据包。尽管它被认为是带宽效率最高的方法，但在eMBB上穿孔URLLC数据可能会导致eMBB性能下降。大多数现有的研究采用原始eMBB数据吞吐量作为性能指标来解决这个问题。然而，这种方法并不充分，因为在穿孔之后，eMBB数据可能成功解码也可能无法成功解码。本文提出了一种基于深度强化学习（DRL）的穿孔解决方案Cyrus+，其奖励函数的设计采用了有效吞吐量（通过接收器解码器的反馈获得），而不是估计的原始吞吐量。此外，Cyrus+专门针对开放无线接入网络（O-RAN）架构进行了优化，并充分利用了O-RAN在不同时间尺度上的三个控制环路来设计DRL。在非实时（Non-RT）RAN智能控制器（RIC）中，Cyrus+初始化将在实时（RT）开放分布式单元（O-DU）中使用的策略网络。在近实时（Near-RT）RIC中，Cyrus+根据动态网络条件和接收器反馈微调策略。在RT O-DU中，Cyrus+通过考虑所有可能的URLLC到达生成穿孔码本。我们构建了一个符合标准的5G NR链路级仿真器，以证明Cyrus+的有效性。实验结果显示，Cyrus+优于基准穿孔算法，并满足5G NR（第3组子载波间隔）的严格时序要求。