标题： Nexus-Gen：通过共享嵌入空间中的预填充自回归实现统一的图像理解、生成和编辑 显示英文标题

标题： Nexus-Gen: Unified Image Understanding, Generation, and Editing via Prefilled Autoregression in Shared Embedding Space

摘要： 统一的多模态生成模型旨在整合图像理解和生成能力，在利用多模态语料库，特别是交错的文本-图像数据方面具有显著优势。然而，现有的统一模型在图像合成质量、自回归误差累积和图像编辑能力方面存在局限性。在本工作中，我们提出了Nexus-Gen，一种新颖的架构，它在一个共享的图像嵌入空间中统一了图像理解、生成和编辑任务。这个共享空间作为自回归和扩散模型之间的桥梁，无缝整合了它们在跨模态建模中的互补优势。为了减轻自回归嵌入预测过程中的严重误差累积，我们提出了一种新的预填充自回归策略，通过用可学习嵌入预填充输入序列来对齐训练-推理动态。在我们构建的包含2630万样本的大规模数据集上进行多阶段和多任务训练后，Nexus-Gen在涵盖图像理解、生成和编辑任务的评估基准上达到了最先进性能。所有模型、数据集和源代码都发布在https://github.com/modelscope/Nexus-Gen，以促进该领域的进一步发展。 显示英文摘要

摘要： Unified multimodal generative models aim to integrate image understanding and generation abilities, offering significant advantages in harnessing multimodal corpora, particularly interleaved text-image data. However, existing unified models exhibit limitations in image synthesis quality, autoregressive error accumulation, and image editing capability. In this work, we propose Nexus-Gen, a novel architecture that unifies image understanding, generation, and editing tasks in a shared image embedding space. This shared space serves as a bridge for the autoregressive and diffusion models, which seamlessly integrates their complementary strengths in cross-modal modeling. To mitigate the severe error accumulation during autoregressive embedding prediction, we propose a novel prefilled autoregression strategy that aligns training-inference dynamics by prefilling input sequences with learnable embeddings. After multi-stage and multi-task training on our constructed large-scale dataset with 26.3 million samples, Nexus-Gen achieves state-of-the-art performance on the evaluation benchmarks spanning image understanding, generation and editing tasks. All models, datasets, and source codes are released in https://github.com/modelscope/Nexus-Gen to facilitate further advancements across the field.