标题： 使用大语言模型自动发现数据库管理系统测试规范 显示英文标题

标题： Automated Discovery of Test Oracles for Database Management Systems Using LLMs

摘要： 自2020年以来，数据库管理系统（DBMSs）的自动化测试蓬勃发展，在广泛使用的系统中发现了数百个错误。 这些技术的核心是测试预言，通常实现一种生成等价查询对的机制，从而通过检查结果的一致性来识别错误。 然而，尽管应用这些预言可以自动化，但其设计仍然是一个根本上手动的过程。 本文探讨了使用大型语言模型（LLMs）来自动化测试预言的发现和实例化，解决了完全自动化DBMS测试的一个长期瓶颈。 尽管LLMs表现出令人印象深刻的新颖性，但它们容易产生幻觉，可能导致大量错误的错误报告。 此外，它们显著的经济成本和延迟意味着应限制LLM调用，以确保错误检测既高效又经济。 为此，我们引入了Argus，这是一个新的框架，基于约束抽象查询的核心概念——一个包含占位符及其相关实例化条件（例如，要求占位符由布尔列填充）的SQL骨架。 Argus使用LLMs生成被断言为语义等价的这些骨架对。 然后使用SQL等价求解器对这种等价性进行形式化证明，以确保正确性。 最后，将经过验证的骨架中的占位符用具体的、可重用的SQL片段进行实例化，这些片段也由LLMs合成，以高效生成复杂的测试用例。 我们实现了Argus，并在五个广泛测试的DBMSs上进行了评估，发现了40个之前未知的错误，其中35个是逻辑错误，36个已被确认，26个已由开发人员修复。 显示英文摘要

摘要： Since 2020, automated testing for Database Management Systems (DBMSs) has flourished, uncovering hundreds of bugs in widely-used systems. A cornerstone of these techniques is test oracle, which typically implements a mechanism to generate equivalent query pairs, thereby identifying bugs by checking the consistency between their results. However, while applying these oracles can be automated, their design remains a fundamentally manual endeavor. This paper explores the use of large language models (LLMs) to automate the discovery and instantiation of test oracles, addressing a long-standing bottleneck towards fully automated DBMS testing. Although LLMs demonstrate impressive creativity, they are prone to hallucinations that can produce numerous false positive bug reports. Furthermore, their significant monetary cost and latency mean that LLM invocations should be limited to ensure that bug detection is efficient and economical. To this end, we introduce Argus, a novel framework built upon the core concept of the Constrained Abstract Query - a SQL skeleton containing placeholders and their associated instantiation conditions (e.g., requiring a placeholder to be filled by a boolean column). Argus uses LLMs to generate pairs of these skeletons that are asserted to be semantically equivalent. This equivalence is then formally proven using a SQL equivalence solver to ensure soundness. Finally, the placeholders within the verified skeletons are instantiated with concrete, reusable SQL snippets that are also synthesized by LLMs to efficiently produce complex test cases. We implemented Argus and evaluated it on five extensively tested DBMSs, discovering 40 previously unknown bugs, 35 of which are logic bugs, with 36 confirmed and 26 already fixed by the developers.