Title: Intelligent Manufacturing Support: Specialized LLMs for Composite Material Processing and Equipment Operation Show Chinese title

Title: 智能制造支持：用于复合材料加工和设备操作的专业化大语言模型

Abstract: Engineering educational curriculum and standards cover many material and manufacturing options. However, engineers and designers are often unfamiliar with certain composite materials or manufacturing techniques. Large language models (LLMs) could potentially bridge the gap. Their capacity to store and retrieve data from large databases provides them with a breadth of knowledge across disciplines. However, their generalized knowledge base can lack targeted, industry-specific knowledge. To this end, we present two LLM-based applications based on the GPT-4 architecture: (1) The Composites Guide: a system that provides expert knowledge on composites material and connects users with research and industry professionals who can provide additional support and (2) The Equipment Assistant: a system that provides guidance for manufacturing tool operation and material characterization. By combining the knowledge of general AI models with industry-specific knowledge, both applications are intended to provide more meaningful information for engineers. In this paper, we discuss the development of the applications and evaluate it through a benchmark and two informal user studies. The benchmark analysis uses the Rouge and Bertscore metrics to evaluate our model performance against GPT-4o. The results show that GPT-4o and the proposed models perform similarly or better on the ROUGE and BERTScore metrics. The two user studies supplement this quantitative evaluation by asking experts to provide qualitative and open-ended feedback about our model performance on a set of domain-specific questions. The results of both studies highlight a potential for more detailed and specific responses with the Composites Guide and the Equipment Assistant. Show Chinese abstract

Abstract: 工程教育课程和标准涵盖了许多材料和制造选项。 然而，工程师和设计师通常对某些复合材料或制造技术不熟悉。 大型语言模型（LLMs）有可能弥合这一差距。 它们能够从大型数据库中存储和检索数据，使其在不同学科领域具备广泛的知识。 然而，它们的通用知识库可能缺乏针对性的行业特定知识。 为此，我们基于GPT-4架构提出了两个基于LLM的应用程序：（1）复合材料指南：一个提供复合材料专业知识的系统，并将用户与能够提供额外支持的研究和行业专业人士联系起来； （2）设备助手：一个为制造工具操作和材料表征提供建议的系统。 通过结合通用AI模型的知识与行业特定知识，这两个应用程序旨在为工程师提供更有意义的信息。 在本文中，我们讨论了这些应用程序的开发，并通过基准测试和两项非正式用户研究进行评估。 基准分析使用Rouge和Bertscore指标来评估我们的模型性能与GPT-4o的对比。 结果表明，在ROUGE和BERTScore指标上，GPT-4o和所提出的模型表现相似或更好。 两项用户研究通过让专家对我们在一组领域特定问题上的模型性能提供定性和开放式的反馈，补充了这种定量评估。 两项研究的结果都突显了复合材料指南和设备助手在提供更详细和具体回答方面的潜力。