标题： PicoAudio2：基于自然语言描述的时序可控文本到音频生成 显示英文标题

标题： PicoAudio2: Temporal Controllable Text-to-Audio Generation with Natural Language Description

摘要： 可控制的文本到音频生成（TTA）最近引起了广泛关注。 尽管现有工作可以基于时间戳信息实现细粒度的控制，但声音事件类别仅限于固定集合。 此外，由于仅使用模拟数据进行训练，生成的音频质量和在真实数据上的泛化性能受到限制。 为了解决这个问题，我们提出了PicoAudio2，通过新的数据处理流程和模型架构来改进时间可控的TTA。 具体来说，我们使用一个定位模型对真实音频-文本数据集中的事件时间戳进行注释，以整理时间性强的真实数据，除了现有工作中使用的模拟数据。 该模型在真实数据和模拟数据的组合上进行训练。 此外，遵循PicoAudio，我们将时间戳信息编码成时间戳矩阵，以在粗粒度文本描述的基础上，为模型提供额外的细粒度时间对齐信息。 实验表明， PicoAudio2在时间可控性和音频质量方面表现出优越的性能。 显示英文摘要

摘要： Controllable text-to-audio generation (TTA) has attracted much attention recently. Although existing works can achieve fine-grained controllability based on timestamp information, sound event categories are limited to a fixed set. Moreover, since only simulated data is used for training, the generated audio quality and generalization performance on real data are limited. To tackle this issue, we propose PicoAudio2, improving temporal-controllable TTA via a new data processing pipeline and model architecture. Specifically, we use a grounding model to annotate event timestamps of real audio-text datasets to curate temporally-strong real data, in addition to simulation data from existing works. The model is trained on the combination of real and simulation data. Moreover, following PicoAudio, we encode timestamp information into a timestamp matrix to provide extra fine-grained time-aligned information to the model, on top of the coarse-grained textual description. Experiments show that PicoAudio2 exhibits superior performance in terms of temporal controllability and audio quality.