标题： 语义帧插值 显示英文标题

标题： Semantic Frame Interpolation

摘要： 基于给定的第一帧和最后一帧以及文本提示信息，生成不同长度的中间视频内容，具有重要的研究和应用潜力。 然而，传统的帧插值任务主要集中在帧数较少、无文本控制且第一帧和最后一帧差异较小的场景中。 最近的社区开发者利用了以万为代表的大型视频模型，赋予帧到帧的能力。 然而，这些模型只能生成固定数量的帧，并且在某些帧长度上常常无法产生满意的结果，而这种设置缺乏明确的官方定义和成熟的标准基准。 在本文中，我们首先从学术定义的角度提出了一种新的实用语义帧插值（SFI）任务，该任务涵盖了上述两种设置，并支持在多种帧率下进行推理。 为了实现这一目标，我们提出了一种基于万2.1的新SemFi模型，该模型结合了Mixture-of-LoRA模块，以确保在各种帧长限制下生成与控制条件一致的高一致性内容。 此外，我们提出了SFI-300K，这是首个专为SFI设计的通用数据集和基准。 为了支持这一点，我们从SFI的角度收集和处理数据，精心设计评估指标和方法，以多维方式评估模型性能，涵盖图像和视频，以及一致性、多样性等多个方面。 通过在SFI-300K上的大量实验，我们证明了我们的方法特别适合满足SFI任务的要求。 显示英文摘要

摘要： Generating intermediate video content of varying lengths based on given first and last frames, along with text prompt information, offers significant research and application potential. However, traditional frame interpolation tasks primarily focus on scenarios with a small number of frames, no text control, and minimal differences between the first and last frames. Recent community developers have utilized large video models represented by Wan to endow frame-to-frame capabilities. However, these models can only generate a fixed number of frames and often fail to produce satisfactory results for certain frame lengths, while this setting lacks a clear official definition and a well-established benchmark. In this paper, we first propose a new practical Semantic Frame Interpolation (SFI) task from the perspective of academic definition, which covers the above two settings and supports inference at multiple frame rates. To achieve this goal, we propose a novel SemFi model building upon Wan2.1, which incorporates a Mixture-of-LoRA module to ensure the generation of high-consistency content that aligns with control conditions across various frame length limitations. Furthermore, we propose SFI-300K, the first general-purpose dataset and benchmark specifically designed for SFI. To support this, we collect and process data from the perspective of SFI, carefully designing evaluation metrics and methods to assess the model's performance across multiple dimensions, encompassing image and video, and various aspects, including consistency and diversity. Through extensive experiments on SFI-300K, we demonstrate that our method is particularly well-suited to meet the requirements of the SFI task.