标题： 基于径向粘结注意机制的多模态多智能体协同感知 显示英文标题

标题： RG-Attn: Radian Glue Attention for Multi-modality Multi-agent Cooperative Perception

摘要： 协同感知通过利用车到一切（V2X）通信实现多个代理之间的数据共享和融合，提供了一种克服单代理系统感知限制的最优解决方案。 然而，大多数现有的方法集中于单一模态的数据交换，限制了同构和异构融合的潜力。 这忽略了每个代理利用多模态数据的机会，从而限制了系统的性能。 在汽车行业，制造商采用不同的传感器配置，导致不同代理之间传感器模态的异构组合。 为了充分利用每一个可能的数据源以获得最佳性能，我们设计了一个鲁棒的激光雷达和相机跨模态融合模块，即Radian-Glue-Attention (RG-Attn)，由于可以通过变换矩阵方便地进行坐标转换以及统一的采样/反演机制，该模块适用于代理内部跨模态融合和代理间跨模态融合场景。 我们还提出了两种不同的架构，名为Paint-To-Puzzle (PTP) 和 Co-Sketching-Co-Coloring (CoS-CoCo)，用于执行协同感知。 PTP旨在实现最大精度性能，并通过将跨代理融合限制为单个实例来减小数据包大小，但要求所有参与者都配备激光雷达。 相比之下，CoS-CoCo支持任何配置的代理——仅激光雷达、仅摄像头或同时具有激光雷达和摄像头，表现出更强的泛化能力。 我们的方法在真实和模拟的协同感知数据集上均达到了最先进的（SOTA）性能。 代码现已在GitHub上发布。 显示英文摘要

摘要： Cooperative perception offers an optimal solution to overcome the perception limitations of single-agent systems by leveraging Vehicle-to-Everything (V2X) communication for data sharing and fusion across multiple agents. However, most existing approaches focus on single-modality data exchange, limiting the potential of both homogeneous and heterogeneous fusion across agents. This overlooks the opportunity to utilize multi-modality data per agent, restricting the system's performance. In the automotive industry, manufacturers adopt diverse sensor configurations, resulting in heterogeneous combinations of sensor modalities across agents. To harness the potential of every possible data source for optimal performance, we design a robust LiDAR and camera cross-modality fusion module, Radian-Glue-Attention (RG-Attn), applicable to both intra-agent cross-modality fusion and inter-agent cross-modality fusion scenarios, owing to the convenient coordinate conversion by transformation matrix and the unified sampling/inversion mechanism. We also propose two different architectures, named Paint-To-Puzzle (PTP) and Co-Sketching-Co-Coloring (CoS-CoCo), for conducting cooperative perception. PTP aims for maximum precision performance and achieves smaller data packet size by limiting cross-agent fusion to a single instance, but requiring all participants to be equipped with LiDAR. In contrast, CoS-CoCo supports agents with any configuration-LiDAR-only, camera-only, or LiDAR-camera-both, presenting more generalization ability. Our approach achieves state-of-the-art (SOTA) performance on both real and simulated cooperative perception datasets. The code is now available at GitHub.