标题： SimVecVis：用于增强可视化理解的多模态大语言模型数据集 显示英文标题

标题： SimVecVis: A Dataset for Enhancing MLLMs in Visualization Understanding

摘要： 当前多模态大语言模型（MLLMs），虽然在自然图像理解方面有效，但由于无法解码数据到视觉的映射并提取结构化信息，因此在可视化理解方面存在困难。 为解决这些挑战，我们提出了SimVec，一种新的简化向量格式，用于编码图表元素，如标记类型、位置和大小。 通过使用MLLMs从SimVec格式中重建图表信息，证明了SimVec的有效性。 然后，我们构建了一个新的可视化数据集SimVecVis，以提高MLLMs在可视化理解方面的性能，该数据集包含三个关键维度：图表的位图图像、它们的SimVec表示以及带有解释性思维链（CoT）描述的数据中心问题回答（QA）对。 我们使用不同数据集维度的SimVecVis对最先进的MLLMs（例如MiniCPM和Qwen-VL）进行微调。 实验结果表明，这显著提高了具有良好空间感知能力的MLLMs（例如MiniCPM）在数据中心QA任务中的性能。 我们的数据集和源代码可在以下地址获取：https://github.com/VIDA-Lab/SimVecVis。 显示英文摘要

摘要： Current multimodal large language models (MLLMs), while effective in natural image understanding, struggle with visualization understanding due to their inability to decode the data-to-visual mapping and extract structured information. To address these challenges, we propose SimVec, a novel simplified vector format that encodes chart elements such as mark type, position, and size. The effectiveness of SimVec is demonstrated by using MLLMs to reconstruct chart information from SimVec formats. Then, we build a new visualization dataset, SimVecVis, to enhance the performance of MLLMs in visualization understanding, which consists of three key dimensions: bitmap images of charts, their SimVec representations, and corresponding data-centric question-answering (QA) pairs with explanatory chain-of-thought (CoT) descriptions. We finetune state-of-the-art MLLMs (e.g., MiniCPM and Qwen-VL), using SimVecVis with different dataset dimensions. The experimental results show that it leads to substantial performance improvements of MLLMs with good spatial perception capabilities (e.g., MiniCPM) in data-centric QA tasks. Our dataset and source code are available at: https://github.com/VIDA-Lab/SimVecVis.