标题： 一种非自回归条件序列生成的EM方法 显示英文标题

标题： An EM Approach to Non-autoregressive Conditional Sequence Generation

摘要： 自回归（AR）模型一直是条件序列生成的主导方法，但存在推理延迟高的问题。 非自回归（NAR）模型最近被提出以通过并行生成所有输出标记来减少延迟，但与自回归模型相比，其准确性较差，主要是因为在处理序列生成中的多模态性时存在困难。 本文提出了一种新方法，在统一的期望最大化（EM）框架中联合优化AR和NAR模型。 在E步中，AR模型学习近似NAR模型的正则化后验分布。 在M步中，NAR模型在新的后验分布上进行更新，并为下一个AR模型选择训练样本。 这种迭代过程可以有效地引导系统去除输出序列中的多模态性。 据我们所知，这是第一个用于NAR序列生成的EM方法。 我们在机器翻译任务上评估了我们的方法。 在基准数据集上的实验结果表明，所提出的方法在现有NAR模型中表现具有竞争力，甚至更好，并显著降低了推理延迟。 显示英文摘要

摘要： Autoregressive (AR) models have been the dominating approach to conditional sequence generation, but are suffering from the issue of high inference latency. Non-autoregressive (NAR) models have been recently proposed to reduce the latency by generating all output tokens in parallel but could only achieve inferior accuracy compared to their autoregressive counterparts, primarily due to a difficulty in dealing with the multi-modality in sequence generation. This paper proposes a new approach that jointly optimizes both AR and NAR models in a unified Expectation-Maximization (EM) framework. In the E-step, an AR model learns to approximate the regularized posterior of the NAR model. In the M-step, the NAR model is updated on the new posterior and selects the training examples for the next AR model. This iterative process can effectively guide the system to remove the multi-modality in the output sequences. To our knowledge, this is the first EM approach to NAR sequence generation. We evaluate our method on the task of machine translation. Experimental results on benchmark data sets show that the proposed approach achieves competitive, if not better, performance with existing NAR models and significantly reduces the inference latency.