标题： 用于语音增强的Schrödinger桥一致性轨迹模型 显示英文标题

标题： Schrödinger Bridge Consistency Trajectory Models for Speech Enhancement

摘要： 语音增强（SE）利用扩散模型是一种有前景的技术，可改善嘈杂语音数据中的语音质量。 此外，薛定谔桥（SB）最近被用于基于扩散的SE，通过解决前向过程的终点和反向过程的起点之间的不匹配来提高语音质量。 然而，由于需要大量的函数评估（NFE）来进行推理以获得高质量的结果，SB仍然表现出推理速度慢的问题。 虽然一致性模型（CMs）通过使用来自图像生成领域的预训练模型进行蒸馏的一致性训练来解决这个问题，但当步骤数增加时，它不会提高生成质量。 为了解决这个问题，一致性轨迹模型（CTMs）不仅加速了推理速度，还保持了质量和速度之间的良好权衡。 此外，SoundCTM展示了CTM技术在声音生成领域的适用性。 在本文中，我们通过将CTM的技术应用于SE的薛定谔桥，提出了薛定谔桥一致性轨迹模型（SBCTM）。 此外，我们在原始CTM的训练框架中引入了一种新的辅助损失，包括感知损失。 结果，与传统的SE薛定谔桥相比，SBCTM的实时因子（RTF）大约提高了16倍。 此外，SBCTM在质量和速度之间的良好权衡使得通过将多步细化限制在1步推理不足的情况下，实现了时间高效的推理。 我们的代码、预训练模型和音频样本可在 https://github.com/sony/sbctm/ 获取。 显示英文摘要

摘要： Speech enhancement (SE) utilizing diffusion models is a promising technology that improves speech quality in noisy speech data. Furthermore, the Schr\"odinger bridge (SB) has recently been used in diffusion-based SE to improve speech quality by resolving a mismatch between the endpoint of the forward process and the starting point of the reverse process. However, the SB still exhibits slow inference owing to the necessity of a large number of function evaluations (NFE) for inference to obtain high-quality results. While Consistency Models (CMs) address this issue by employing consistency training that uses distillation from pretrained models in the field of image generation, it does not improve generation quality when the number of steps increases. As a solution to this problem, Consistency Trajectory Models (CTMs) not only accelerate inference speed but also maintain a favorable trade-off between quality and speed. Furthermore, SoundCTM demonstrates the applicability of CTM techniques to the field of sound generation. In this paper, we present Schr\"odinger bridge Consistency Trajectory Models (SBCTM) by applying the CTM's technique to the Schr\"odinger bridge for SE. Additionally, we introduce a novel auxiliary loss, including a perceptual loss, into the original CTM's training framework. As a result, SBCTM achieves an approximately 16x improvement in the real-time factor (RTF) compared to the conventional Schr\"odinger bridge for SE. Furthermore, the favorable trade-off between quality and speed in SBCTM allows for time-efficient inference by limiting multi-step refinement to cases where 1-step inference is insufficient. Our code, pretrained models, and audio samples are available at https://github.com/sony/sbctm/.