标题： 面向基于大语言模型的因果业务流程推理的基准 显示英文标题

标题： Towards a Benchmark for Causal Business Process Reasoning with LLMs

摘要： 大型语言模型（LLMs）被越来越多地用于提高组织效率和自动化任务。 虽然最初并未设计用于复杂的认知过程，但最近的努力进一步扩展到将LLMs应用于推理、规划和决策等活动中。 在业务流程中，这种能力可以利用LLMs训练所使用的大量语料库，从而深入了解这些流程。 在这项工作中，我们为开发一个基准测试奠定了基础，以评估LLMs对业务操作的因果和流程视角进行推理的能力。 我们将这一观点称为因果增强型业务流程（BP^C）。 基准测试的核心包括一组与BP^C相关的场景、一组关于这些场景的问题，以及一组用于系统地解决这些问题的地面真实答案的演绎规则。 同时，借助LLMs的强大功能，种子数据随后被实例化为更大规模的特定领域场景和问题。 对BP^C进行推理对于流程干预和流程改进至关重要。 我们的基准测试可在https://huggingface.co/datasets/ibm/BPC获取，可以以两种可能的方式使用：测试任何目标LLM的性能，或训练LLM以提升其对BP^C进行推理的能力。 显示英文摘要

摘要： Large Language Models (LLMs) are increasingly used for boosting organizational efficiency and automating tasks. While not originally designed for complex cognitive processes, recent efforts have further extended to employ LLMs in activities such as reasoning, planning, and decision-making. In business processes, such abilities could be invaluable for leveraging on the massive corpora LLMs have been trained on for gaining deep understanding of such processes. In this work, we plant the seeds for the development of a benchmark to assess the ability of LLMs to reason about causal and process perspectives of business operations. We refer to this view as Causally-augmented Business Processes (BP^C). The core of the benchmark comprises a set of BP^C related situations, a set of questions about these situations, and a set of deductive rules employed to systematically resolve the ground truth answers to these questions. Also with the power of LLMs, the seed is then instantiated into a larger-scale set of domain-specific situations and questions. Reasoning on BP^C is of crucial importance for process interventions and process improvement. Our benchmark, accessible at https://huggingface.co/datasets/ibm/BPC, can be used in one of two possible modalities: testing the performance of any target LLM and training an LLM to advance its capability to reason about BP^C.