标题： 流式同时语音翻译增强记忆变压器 显示英文标题

标题： Streaming Simultaneous Speech Translation with Augmented Memory Transformer

摘要： 基于Transformer的模型在语音翻译任务中取得了最先进的性能。 然而，由于自注意力机制是针对整个输入序列计算的，模型架构在流式场景中效率不够高，计算成本随着输入序列长度的增加而呈二次增长。 然而，大多数关于同时语音翻译的先前工作，在分析延迟时忽略了生成翻译所花费的时间。 在这种假设下，一个系统可能具有良好的延迟质量权衡，但在实时场景中可能无法应用。 在本文中，我们专注于流式同时语音翻译任务，其中系统不仅能够使用部分输入进行翻译，还能够处理非常长或连续的输入。 我们提出了一种端到端的基于Transformer的序列到序列模型，配备了一个增强记忆的Transformer编码器，该模型在使用混合或转换器基础模型的流式自动语音识别任务中已表现出巨大的成功。 我们在段、上下文和记忆大小上对所提出的模型进行了经验评估，并将我们的方法与具有单向掩码的Transformer进行了比较。 显示英文摘要

摘要： Transformer-based models have achieved state-of-the-art performance on speech translation tasks. However, the model architecture is not efficient enough for streaming scenarios since self-attention is computed over an entire input sequence and the computational cost grows quadratically with the length of the input sequence. Nevertheless, most of the previous work on simultaneous speech translation, the task of generating translations from partial audio input, ignores the time spent in generating the translation when analyzing the latency. With this assumption, a system may have good latency quality trade-offs but be inapplicable in real-time scenarios. In this paper, we focus on the task of streaming simultaneous speech translation, where the systems are not only capable of translating with partial input but are also able to handle very long or continuous input. We propose an end-to-end transformer-based sequence-to-sequence model, equipped with an augmented memory transformer encoder, which has shown great success on the streaming automatic speech recognition task with hybrid or transducer-based models. We conduct an empirical evaluation of the proposed model on segment, context and memory sizes and we compare our approach to a transformer with a unidirectional mask.