标题： VP-SelDoA：通过语义-空间匹配的视觉提示选择性到达方向估计 显示英文标题

标题： VP-SelDoA: Visual-prompted Selective DoA Estimation of Target Sound via Semantic-Spatial Matching

摘要： 音视频声源定位（AV-SSL）通过利用听觉和视觉信号的互补优势来确定声源的位置。 然而，现有的AV-SSL方法面临三个主要挑战：1）在多源场景中无法选择性地隔离目标声源，2）语义视觉特征与空间声学特征之间的错位，3）对配对的音视频数据过度依赖。 为克服这些限制，我们引入了跨实例音视频定位（CI-AVL），这是一个新任务，利用同一声音事件类别的不同实例的图像来定位目标声源，从而减少对配对数据的依赖，同时增强泛化能力。 我们提出的VP-SelDoA通过语义级模态融合来解决这一具有挑战性的任务，并采用频率-时间ConMamba架构生成目标选择性掩码以实现声音隔离。 我们进一步开发了一个语义-空间匹配机制，通过集成的交叉注意力和自注意力机制来对齐异构的语义和空间特征。 为了促进CI-AVL的研究，我们构建了一个名为VGG-SSL的大规模数据集，包含296个声音事件类别中的13,981个空间音频片段。 大量实验表明，我们提出的方法优于最先进的音视频定位方法，实现了12.04的平均绝对误差（MAE）和78.23%的准确率（ACC）。 显示英文摘要

摘要： Audio-visual sound source localization (AV-SSL) identifies the position of a sound source by exploiting the complementary strengths of auditory and visual signals. However, existing AV-SSL methods encounter three major challenges: 1) inability to selectively isolate the target sound source in multi-source scenarios, 2) misalignment between semantic visual features and spatial acoustic features, and 3) overreliance on paired audio-visual data. To overcome these limitations, we introduce Cross-Instance Audio-Visual Localization (CI-AVL), a novel task that leverages images from different instances of the same sound event category to localize target sound sources, thereby reducing dependence on paired data while enhancing generalization capabilities. Our proposed VP-SelDoA tackles this challenging task through a semantic-level modality fusion and employs a Frequency-Temporal ConMamba architecture to generate target-selective masks for sound isolation. We further develop a Semantic-Spatial Matching mechanism that aligns the heterogeneous semantic and spatial features via integrated cross- and self-attention mechanisms. To facilitate the CI-AVL research, we construct a large-scale dataset named VGG-SSL, comprising 13,981 spatial audio clips across 296 sound event categories. Extensive experiments show that our proposed method outperforms state-of-the-art audio-visual localization methods, achieving a mean absolute error (MAE) of 12.04 and an accuracy (ACC) of 78.23%.