标题： RG-Attn：多模态多智能体协同感知的弧度粘合注意力 显示英文标题

标题： RG-Attn: Radian Glue Attention for Multi-modality Multi-agent Cooperative Perception

摘要： 协作感知通过利用车对一切（V2X）通信在多个代理之间进行数据共享和融合，为克服单智能体系统的感知限制提供了最佳解决方案。 然而，大多数现有方法专注于单模态数据交换，限制了跨代理的同构和异构融合的潜力。 这忽略了利用每个代理的多模态数据的机会，限制了系统性能。 在汽车工业中，制造商采用不同的传感器配置，导致代理之间传感器模态的异构组合。 为了充分利用每一个可能的数据源以实现最佳性能，我们设计了一个鲁棒的激光雷达和相机跨模态融合模块，称为Radian-Glue-Attention（RG-Attn），由于变换矩阵带来的方便坐标转换以及统一的采样/反演机制，该模块适用于内部智能体跨模态融合和跨智能体跨模态融合场景。 我们还提出了两种不同的架构，分别命名为Paint-To-Puzzle（PTP）和Co-Sketching-Co-Coloring（CoS-CoCo），用于进行协作感知。 PTP旨在实现最大精度性能，并通过将跨智能体融合限制为一个实例来实现更小的数据包大小，但要求所有参与者都配备激光雷达。 相比之下，CoS-CoCo支持任何配置的智能体——仅激光雷达、仅相机或同时具备激光雷达和相机，表现出更强的泛化能力。 我们的方法在真实和模拟的协作感知数据集上均达到了最先进的（SOTA）性能。 代码将于2025年初在GitHub上发布。 显示英文摘要

摘要： Cooperative perception offers an optimal solution to overcome the perception limitations of single-agent systems by leveraging Vehicle-to-Everything (V2X) communication for data sharing and fusion across multiple agents. However, most existing approaches focus on single-modality data exchange, limiting the potential of both homogeneous and heterogeneous fusion across agents. This overlooks the opportunity to utilize multi-modality data per agent, restricting the system's performance. In the automotive industry, manufacturers adopt diverse sensor configurations, resulting in heterogeneous combinations of sensor modalities across agents. To harness the potential of every possible data source for optimal performance, we design a robust LiDAR and camera cross-modality fusion module, Radian-Glue-Attention (RG-Attn), applicable to both intra-agent cross-modality fusion and inter-agent cross-modality fusion scenarios, owing to the convenient coordinate conversion by transformation matrix and the unified sampling/inversion mechanism. We also propose two different architectures, named Paint-To-Puzzle (PTP) and Co-Sketching-Co-Coloring (CoS-CoCo), for conducting cooperative perception. PTP aims for maximum precision performance and achieves smaller data packet size by limiting cross-agent fusion to a single instance, but requiring all participants to be equipped with LiDAR. In contrast, CoS-CoCo supports agents with any configuration-LiDAR-only, camera-only, or LiDAR-camera-both, presenting more generalization ability. Our approach achieves state-of-the-art (SOTA) performance on both real and simulated cooperative perception datasets. The code will be released at GitHub in early 2025.