标题： 使用合成模式的预训练用于音频 显示英文标题

标题： Pre-training with Synthetic Patterns for Audio

摘要： 本文中，我们提议使用合成模式而非真实音频数据来预训练音频编码器。 我们提出的框架包含两个关键元素。 第一个是掩码自动编码器（MAE），这是一种自监督学习框架，通过从随机屏蔽的对应部分重建数据来学习。 MAEs 倾向于关注低级信息，如数据中的视觉模式和规律性。 因此，输入内容是什么并不重要，无论是图像、音频梅尔频谱图，甚至是合成模式。 这引出了第二个关键元素，即合成数据。 与真实音频不同，合成数据不存在隐私和许可侵权问题。 通过结合 MAEs 和合成模式，我们的框架使模型能够在没有真实数据的情况下学习通用特征表示，同时解决了与真实音频相关的问题。 为了评估我们框架的有效性，我们在总共 13 个音频任务和 17 个合成数据集上进行了广泛的实验。 这些实验提供了关于哪种类型的合成模式对音频有效的见解。 我们的结果显示，我们的框架在性能上可以媲美在 AudioSet-2M 上预训练的模型，并且在某些方面超过了基于图像的预训练方法。 显示英文摘要

摘要： In this paper, we propose to pre-train audio encoders using synthetic patterns instead of real audio data. Our proposed framework consists of two key elements. The first one is Masked Autoencoder (MAE), a self-supervised learning framework that learns from reconstructing data from randomly masked counterparts. MAEs tend to focus on low-level information such as visual patterns and regularities within data. Therefore, it is unimportant what is portrayed in the input, whether it be images, audio mel-spectrograms, or even synthetic patterns. This leads to the second key element, which is synthetic data. Synthetic data, unlike real audio, is free from privacy and licensing infringement issues. By combining MAEs and synthetic patterns, our framework enables the model to learn generalized feature representations without real data, while addressing the issues related to real audio. To evaluate the efficacy of our framework, we conduct extensive experiments across a total of 13 audio tasks and 17 synthetic datasets. The experiments provide insights into which types of synthetic patterns are effective for audio. Our results demonstrate that our framework achieves performance comparable to models pre-trained on AudioSet-2M and partially outperforms image-based pre-training methods.