标题： MEAN-RIR：用于鲁棒房间脉冲响应估计的多模态环境感知网络 显示英文标题

标题： MEAN-RIR: Multi-Modal Environment-Aware Network for Robust Room Impulse Response Estimation

摘要： 本文提出了一种多模态环境感知网络（MEAN-RIR），该网络使用编码器-解码器框架，基于来自音频、视觉和文本源的多级环境信息预测房间脉冲响应（RIR）。 具体而言，混响语音捕获房间声学特性作为主要输入，与全景图像和文本描述作为补充输入相结合。 每个输入由其各自的编码器处理，并将输出馈入交叉注意力模块，以实现不同模态之间的有效交互。 MEAN-RIR解码器生成两个不同的组件：第一个组件捕捉直达声和早期反射，而第二个组件产生调制可学习滤波噪声的掩码以合成晚期混响。 这两个组件混合以重建最终的RIR。 结果表明，MEAN-RIR显著提高了RIR估计，特别是在声学参数方面有明显提升。 显示英文摘要

摘要： This paper presents a Multi-Modal Environment-Aware Network (MEAN-RIR), which uses an encoder-decoder framework to predict room impulse response (RIR) based on multi-level environmental information from audio, visual, and textual sources. Specifically, reverberant speech capturing room acoustic properties serves as the primary input, which is combined with panoramic images and text descriptions as supplementary inputs. Each input is processed by its respective encoder, and the outputs are fed into cross-attention modules to enable effective interaction between different modalities. The MEAN-RIR decoder generates two distinct components: the first component captures the direct sound and early reflections, while the second produces masks that modulate learnable filtered noise to synthesize the late reverberation. These two components are mixed to reconstruct the final RIR. The results show that MEAN-RIR significantly improves RIR estimation, with notable gains in acoustic parameters.