标题： 面向具有LLMs的因果业务流程推理的基准 显示英文标题

标题： Towards a Benchmark for Causal Business Process Reasoning with LLMs

摘要： 大型语言模型（LLMs）被越来越多地用于提升组织效率和自动化任务。 尽管最初并非为复杂的认知过程而设计，但最近的努力进一步扩展了LLMs在推理、规划和决策等活动中应用。 在业务流程中，这些能力可以利用LLMs训练所使用的大量语料库，从而深入了解这些流程，具有极大的价值。 在这项工作中，我们为开发一个基准测试奠定了基础，以评估LLMs在因果关系和流程视角方面进行推理的能力。 我们将这种观点称为因果增强型业务流程（BP^C）。 该基准的核心包括一组与BP^C相关的场景、一组关于这些场景的问题，以及一组用于系统解决这些问题真实答案的演绎规则。 同时，借助LLMs的强大能力，种子数据随后被实例化为更大规模的特定领域场景和问题。 对BP^C进行推理对于流程干预和流程改进至关重要。 我们的基准测试可以以两种可能的方式使用：测试任何目标LLM的性能，或训练LLM以提高其对BP^C进行推理的能力。 显示英文摘要

摘要： Large Language Models (LLMs) are increasingly used for boosting organizational efficiency and automating tasks. While not originally designed for complex cognitive processes, recent efforts have further extended to employ LLMs in activities such as reasoning, planning, and decision-making. In business processes, such abilities could be invaluable for leveraging on the massive corpora LLMs have been trained on for gaining deep understanding of such processes. In this work, we plant the seeds for the development of a benchmark to assess the ability of LLMs to reason about causal and process perspectives of business operations. We refer to this view as Causally-augmented Business Processes (BP^C). The core of the benchmark comprises a set of BP^C related situations, a set of questions about these situations, and a set of deductive rules employed to systematically resolve the ground truth answers to these questions. Also with the power of LLMs, the seed is then instantiated into a larger-scale set of domain-specific situations and questions. Reasoning on BP^C is of crucial importance for process interventions and process improvement. Our benchmark could be used in one of two possible modalities: testing the performance of any target LLM and training an LLM to advance its capability to reason about BP^C.