标题： VLM-UDMC：用于城市自动驾驶的VLM增强型统一决策与运动控制 显示英文标题

标题： VLM-UDMC: VLM-Enhanced Unified Decision-Making and Motion Control for Urban Autonomous Driving

摘要： 场景理解和风险感知注意力对于人类驾驶员做出安全有效的驾驶决策至关重要。 为了在城市自动驾驶中模仿这种认知能力并确保透明性和可解释性，我们提出了一种视觉-语言模型（VLM）增强的统一决策和运动控制框架，名为VLM-UDMC。 该框架将场景推理和风险感知洞察整合到一个上层慢速系统中，该系统动态重新配置下游快速系统的最佳运动规划。 重新配置基于实时环境变化，这些变化通过上下文感知势函数进行编码。 更具体地说，上层慢速系统采用两步推理策略，结合检索增强生成（RAG），利用基础模型处理多模态输入并检索上下文知识，从而生成风险感知洞察。 同时，一个轻量级多核分解LSTM通过提取短期轨迹预测的更平滑趋势表示，为异构交通参与者提供实时轨迹预测。 通过全尺寸自动驾驶车辆的仿真和真实实验验证了所提出的VLM-UDMC框架的有效性。 结果表明，所提出的VLM-UDMC能够有效利用场景理解和注意力分解进行合理的驾驶决策，从而提高整体城市驾驶性能。 我们的开源项目可在https://github.com/henryhcliu/vlmudmc.git获取。 显示英文摘要

摘要： Scene understanding and risk-aware attentions are crucial for human drivers to make safe and effective driving decisions. To imitate this cognitive ability in urban autonomous driving while ensuring the transparency and interpretability, we propose a vision-language model (VLM)-enhanced unified decision-making and motion control framework, named VLM-UDMC. This framework incorporates scene reasoning and risk-aware insights into an upper-level slow system, which dynamically reconfigures the optimal motion planning for the downstream fast system. The reconfiguration is based on real-time environmental changes, which are encoded through context-aware potential functions. More specifically, the upper-level slow system employs a two-step reasoning policy with Retrieval-Augmented Generation (RAG), leveraging foundation models to process multimodal inputs and retrieve contextual knowledge, thereby generating risk-aware insights. Meanwhile, a lightweight multi-kernel decomposed LSTM provides real-time trajectory predictions for heterogeneous traffic participants by extracting smoother trend representations for short-horizon trajectory prediction. The effectiveness of the proposed VLM-UDMC framework is verified via both simulations and real-world experiments with a full-size autonomous vehicle. It is demonstrated that the presented VLM-UDMC effectively leverages scene understanding and attention decomposition for rational driving decisions, thus improving the overall urban driving performance. Our open-source project is available at https://github.com/henryhcliu/vlmudmc.git.