标题： LOCOFY 大型设计模型 -- 设计到代码转换解决方案 显示英文标题

标题： LOCOFY Large Design Models -- Design to code conversion solution

摘要： 尽管大型语言模型和多模态大型语言模型（LLMs）取得了快速进展，但在可解释性、可扩展性、资源需求和可重复性方面仍存在许多与它们在设计到代码领域应用相关的问题。 为了解决这些问题，我们引入了大型设计模型（LDMs）范式，专门针对设计和网页进行训练，以实现从设计到代码的无缝转换。 我们通过结合数据工程和适当的模型架构修改，开发了一个训练和推理管道。 训练管道包括以下内容：1）设计优化器：使用专有的真实数据集开发，用于解决次优设计问题；2）标记和特征检测：使用预训练和微调模型，这使得能够准确检测和分类用户界面元素；3）自动组件：将重复的用户界面结构提取为可重用组件，从而实现模块化代码的创建，减少冗余同时提高代码可重用性。 以这种方式，每个模型解决了设计到代码转换中的不同但关键的问题。 此外，我们的推理管道处理现实世界的设计，生成精确且可解释的代码生成指令，并确保可靠性。 此外，我们的模型使用一种新颖的预览匹配评分指标，在端到端的设计到代码转换准确性方面表现出色。 比较实验表明，LDMs在节点定位准确性、响应性和可重复性方面优于LLMs。 此外，我们的自定义训练的标记和特征检测模型在识别广泛测试设计中的用户界面元素方面表现出高精度和一致性。 因此，我们提出的LDMs是一种可靠且优越的解决方案，用于理解设计，从而实现高效且可靠的生产就绪代码的生成。 显示英文摘要

摘要： Despite rapid advances in Large Language Models and Multimodal Large Language Models (LLMs), numerous challenges related to interpretability, scalability, resource requirements and repeatability remain, related to their application in the design-to-code space. To address this, we introduce the Large Design Models (LDMs) paradigm specifically trained on designs and webpages to enable seamless conversion from design-to-code. We have developed a training and inference pipeline by incorporating data engineering and appropriate model architecture modification. The training pipeline consists of the following: 1)Design Optimiser: developed using a proprietary ground truth dataset and addresses sub-optimal designs; 2)Tagging and feature detection: using pre-trained and fine-tuned models, this enables the accurate detection and classification of UI elements; and 3)Auto Components: extracts repeated UI structures into reusable components to enable creation of modular code, thus reducing redundancy while enhancing code reusability. In this manner, each model addresses distinct but key issues for design-to-code conversion. Separately, our inference pipeline processes real-world designs to produce precise and interpretable instructions for code generation and ensures reliability. Additionally, our models illustrated exceptional end-to-end design-to-code conversion accuracy using a novel preview match score metric. Comparative experiments indicated superior performance of LDMs against LLMs on accuracy of node positioning, responsiveness and reproducibility. Moreover, our custom-trained tagging and feature detection model demonstrated high precision and consistency in identifying UI elements across a wide sample of test designs. Thus, our proposed LDMs are a reliable and superior solution to understanding designs that subsequently enable the generation of efficient and reliable production-ready code.