标题： 大规模机器学习系统中的来源跟踪 显示英文标题

标题： Provenance Tracking in Large-Scale Machine Learning Systems

摘要： 随着对大规模人工智能模型需求的持续增长，优化其训练以平衡计算效率、执行时间、准确性和能耗代表了一个关键的多维挑战。 实现这种平衡不仅需要创新的算法技术和硬件架构，还需要全面的工具来监控、分析和理解模型训练和部署过程中涉及的底层过程。 关于数据和过程的起源、上下文和转换的溯源数据信息已成为这一追求中的关键组成部分。 通过利用溯源信息，研究人员和工程师可以深入了解资源使用模式，识别低效之处，并确保人工智能开发工作流中的可再现性和问责性。 因此，如何最优地利用分布式资源以节能的方式扩展大规模人工智能模型是一个基本问题。 为了支持这一努力，我们引入了yProv4ML库，这是一个设计用于以JSON格式收集符合W3C PROV和ProvML标准的溯源数据的工具。 yProv4ML注重灵活性和可扩展性，并允许用户通过插件集成其他数据收集工具。 该库完全集成于yProv框架中，使得在通过工作流管理系统运行的任务中能够进行更高层次的配对。 显示英文摘要

摘要： As the demand for large scale AI models continues to grow, the optimization of their training to balance computational efficiency, execution time, accuracy and energy consumption represents a critical multidimensional challenge. Achieving this balance requires not only innovative algorithmic techniques and hardware architectures but also comprehensive tools for monitoring, analyzing, and understanding the underlying processes involved in model training and deployment. Provenance data information about the origins, context, and transformations of data and processes has become a key component in this pursuit. By leveraging provenance, researchers and engineers can gain insights into resource usage patterns, identify inefficiencies, and ensure reproducibility and accountability in AI development workflows. For this reason, the question of how distributed resources can be optimally utilized to scale large AI models in an energy efficient manner is a fundamental one. To support this effort, we introduce the yProv4ML library, a tool designed to collect provenance data in JSON format, compliant with the W3C PROV and ProvML standards. yProv4ML focuses on flexibility and extensibility, and enables users to integrate additional data collection tools via plugins. The library is fully integrated with the yProv framework, allowing for higher level pairing in tasks run also through workflow management systems.