标题： 强化学习在自动驾驶中的动作空间缩减策略 显示英文标题

标题： Action Space Reduction Strategies for Reinforcement Learning in Autonomous Driving

摘要： 强化学习（RL）通过使代理通过与环境的交互学习控制策略，为自动驾驶提供了一个有前景的框架。 然而，用于支持精细控制的大而高维的动作空间可能会阻碍训练效率并增加探索成本。 在本研究中，我们引入并评估了两种新的结构化动作空间修改策略，用于自动驾驶中的强化学习：动态遮罩和相对动作空间缩减。 这些方法与固定缩减方案和完整动作空间基线进行系统比较，以评估它们对策略学习和性能的影响。 我们的框架利用了一个多模态近端策略优化代理，该代理处理语义图像序列和标量车辆状态。 所提出的动态和相对策略基于上下文和状态转换进行实时动作遮罩，在保持动作一致性的同时消除无效或次优选择。 通过在多样化驾驶路线上的全面实验，我们表明动作空间缩减显著提高了训练稳定性和策略性能。 特别是动态和相对方案在学习速度、控制精度和泛化能力之间实现了有利的平衡。 这些发现强调了上下文感知动作空间设计在自动驾驶任务中可扩展且可靠强化学习的重要性。 显示英文摘要

摘要： Reinforcement Learning (RL) offers a promising framework for autonomous driving by enabling agents to learn control policies through interaction with environments. However, large and high-dimensional action spaces often used to support fine-grained control can impede training efficiency and increase exploration costs. In this study, we introduce and evaluate two novel structured action space modification strategies for RL in autonomous driving: dynamic masking and relative action space reduction. These approaches are systematically compared against fixed reduction schemes and full action space baselines to assess their impact on policy learning and performance. Our framework leverages a multimodal Proximal Policy Optimization agent that processes both semantic image sequences and scalar vehicle states. The proposed dynamic and relative strategies incorporate real-time action masking based on context and state transitions, preserving action consistency while eliminating invalid or suboptimal choices. Through comprehensive experiments across diverse driving routes, we show that action space reduction significantly improves training stability and policy performance. The dynamic and relative schemes, in particular, achieve a favorable balance between learning speed, control precision, and generalization. These findings highlight the importance of context-aware action space design for scalable and reliable RL in autonomous driving tasks.