标题： OCDB：在全面基准和评估框架下重新审视因果发现 显示英文标题

标题： OCDB: Revisiting Causal Discovery with a Comprehensive Benchmark and Evaluation Framework

摘要： 大型语言模型（LLMs）在各种自然语言处理任务中表现出色，但可解释性和可信度方面的挑战仍然存在，限制了它们在高风险领域中的应用。因果发现为提高透明度和可靠性提供了一种有前景的方法。然而，当前的评估往往片面，缺乏针对可解释性性能的评估。此外，这些评估依赖于合成数据，缺乏对现实数据集的全面评估。这导致一些有潜力的方法可能被忽视。为了解决这些问题，我们提出了一种灵活的评估框架，包含用于评估因果结构和因果效应差异的指标，这些是有助于提高LLMs可解释性的关键属性。我们引入了基于真实数据的开放因果发现基准（OCDB），以促进公平比较并推动算法优化。此外，我们的新指标考虑了无向边，使得有向无环图（DAGs）和完整部分有向无环图（CPDAGs）之间的公平比较成为可能。实验结果表明现有算法在真实数据上的泛化能力存在显著不足，突显了性能改进的潜力以及我们的框架在推进因果发现技术中的重要性。 显示英文摘要

摘要： Large language models (LLMs) have excelled in various natural language processing tasks, but challenges in interpretability and trustworthiness persist, limiting their use in high-stakes fields. Causal discovery offers a promising approach to improve transparency and reliability. However, current evaluations are often one-sided and lack assessments focused on interpretability performance. Additionally, these evaluations rely on synthetic data and lack comprehensive assessments of real-world datasets. These lead to promising methods potentially being overlooked. To address these issues, we propose a flexible evaluation framework with metrics for evaluating differences in causal structures and causal effects, which are crucial attributes that help improve the interpretability of LLMs. We introduce the Open Causal Discovery Benchmark (OCDB), based on real data, to promote fair comparisons and drive optimization of algorithms. Additionally, our new metrics account for undirected edges, enabling fair comparisons between Directed Acyclic Graphs (DAGs) and Completed Partially Directed Acyclic Graphs (CPDAGs). Experimental results show significant shortcomings in existing algorithms' generalization capabilities on real data, highlighting the potential for performance improvement and the importance of our framework in advancing causal discovery techniques.