标题： PromptSep：通过多模态提示的生成音频分离 显示英文标题

标题： PromptSep: Generative Audio Separation via Multimodal Prompting

摘要： 语言查询音频源分离（LASS）的最新突破表明，生成模型可以实现比传统基于掩码的方法更高的分离音频质量。 然而，两个关键限制阻碍了它们的实际应用：（1）用户通常需要超出分离的操作，例如声音移除；（2）仅依赖文本提示在指定声音源时可能不够直观。 在本文中，我们提出了PromptSep，以将LASS扩展为一个更广泛的通用声音分离框架。 PromptSep利用了一个增强的数据模拟条件扩散模型，以实现音频提取和声音移除。 为了超越仅文本查询，我们通过引入Sketch2Sound作为数据增强策略，将语音模仿作为一种额外且更直观的条件模态融入到我们的模型中。 在多个基准上的客观和主观评估表明，PromptSep在声音移除和语音模仿引导的源分离方面达到了最先进的性能，同时在语言查询源分离上保持了有竞争力的结果。 显示英文摘要

摘要： Recent breakthroughs in language-queried audio source separation (LASS) have shown that generative models can achieve higher separation audio quality than traditional masking-based approaches. However, two key limitations restrict their practical use: (1) users often require operations beyond separation, such as sound removal; and (2) relying solely on text prompts can be unintuitive for specifying sound sources. In this paper, we propose PromptSep to extend LASS into a broader framework for general-purpose sound separation. PromptSep leverages a conditional diffusion model enhanced with elaborated data simulation to enable both audio extraction and sound removal. To move beyond text-only queries, we incorporate vocal imitation as an additional and more intuitive conditioning modality for our model, by incorporating Sketch2Sound as a data augmentation strategy. Both objective and subjective evaluations on multiple benchmarks demonstrate that PromptSep achieves state-of-the-art performance in sound removal and vocal-imitation-guided source separation, while maintaining competitive results on language-queried source separation.