标题： 通过高质量数据整理实现可扩展的视觉语言模型训练 显示英文标题

标题： Scalable Vision Language Model Training via High Quality Data Curation

摘要： 本文介绍了一个名为SAIL-VL（通过高质量数据整理实现可扩展视觉语言模型训练）的开源视觉语言模型（VLM）系列，该系列在2B和8B参数规模上达到了最先进的性能（SOTA）。 以下三个关键改进促成了SAIL-VL的领先性能：（1）可扩展的高质量视觉理解数据构建：我们实施了一条数据构建管道，以实现亿级规模的高质量重新标注数据注释。所得到的数据集SAIL-Caption被验证与开源数据集相比具有最高的数据质量。（2）使用高质量视觉理解数据进行可扩展预训练：我们将SAIL-VL的预训练预算扩展到655B个标记，并表明即使是2B的VLM也能从扩大的训练数据量中受益，在基准测试性能上表现出对数数据规模缩放定律。（3）通过数据量和复杂度缩放实现可扩展的SFT：我们策划了一组高质量的SFT数据集收集，具有领先的数量缩放效果，并证明使用逐步增加复杂性的数据训练大大超过了基线的一阶段训练。 在我们的评估中，SAIL-VL系列模型在18个广泛使用的VLM基准测试中取得了最高的平均分，其中2B模型在OpenCompass 2024（https://rank.opencompass.org.cn/leaderboard-multimodal）上的同类大小的VLM中排名第一，展示了强大的视觉理解能力。SAIL-VL系列模型已在HuggingFace（https://huggingface.co/BytedanceDouyinContent）发布。 显示英文摘要

摘要： In this paper, we introduce SAIL-VL (ScAlable Vision Language Model TraIning via High QuaLity Data Curation), an open-source vision language model (VLM) series achieving state-of-the-art (SOTA) performance in 2B and 8B parameters. The following three key improvements contribute to SAIL-VL's leading performance: (1) Scalable high-quality visual understanding data construction: We implement a data construction pipeline to enable hundred-million-scale high-quality recaption data annotation. The resulted dataset SAIL-Caption is validated to be of the highest data quality compared with opensource datasets. (2) Scalable Pretraining with High-Quality Visual Understanding Data: We scale SAIL-VL's pretraining budget up to 655B tokens and show that even a 2B VLM benefits from scaled up training data sizes, exhibiting logarithmic data size scaling laws in benchmark performance. (3) Scalable SFT via data quantity and complexity scaling: We curate a high-quality SFT dataset collection with leading data quantity scaling effectiveness and demonstrate that training with progressively higher-complexity data surpasses baseline one-stage training by a large margin. SAIL-VL series models achieve the highest average score in 18 widely used VLM benchmarks in our evaluation, with the 2B model takes the top position over VLMs of comparable sizes on OpenCompass 2024 (https://rank.opencompass.org.cn/leaderboard-multimodal), demonstrating robust visual comprehension abilities. SAIL-VL series models are released at HuggingFace (https://huggingface.co/BytedanceDouyinContent).