标题： CySecBench：基于生成式AI的面向网络安全的提示数据集，用于基准测试大型语言模型 显示英文标题

标题： CySecBench: Generative AI-based CyberSecurity-focused Prompt Dataset for Benchmarking Large Language Models

摘要： 大量研究已经调查了使大型语言模型（LLMs）越狱以生成有害内容的方法。通常，这些方法使用由恶意提示组成的数据集进行评估，这些提示旨在绕过LLM提供商建立的安全策略。然而，现有数据集通常范围广泛且开放性较强，这可能会使越狱效果的评估变得复杂，尤其是在特定领域，尤其是网络安全领域。为了解决这个问题，我们提出了并公开发布了CySecBench，这是一个包含12662个专门设计用于评估网络安全领域越狱技术的全面数据集。该数据集分为10个不同的攻击类型类别，包含封闭式提示，以实现对越狱尝试更一致和准确的评估。此外，我们详细描述了数据集生成和过滤的方法，该方法可以调整以在其他领域创建类似的数据集。为了展示CySecBench的实用性，我们提出并评估了一种基于提示混淆的越狱方法。我们的实验结果表明，这种方法成功地从商业黑盒LLMs中引出了有害内容，在ChatGPT上实现了65%的成功率（SRs），在Gemini上实现了88%；相比之下，Claude表现出更大的韧性，越狱成功率为17%。与现有基准方法相比，我们的方法表现更优，突显了领域特定评估数据集在评估LLM安全措施中的价值。此外，当使用一个广泛使用的数据集（即AdvBench）中的提示进行评估时，它实现了78.5%的成功率，高于最先进方法。 显示英文摘要

摘要： Numerous studies have investigated methods for jailbreaking Large Language Models (LLMs) to generate harmful content. Typically, these methods are evaluated using datasets of malicious prompts designed to bypass security policies established by LLM providers. However, the generally broad scope and open-ended nature of existing datasets can complicate the assessment of jailbreaking effectiveness, particularly in specific domains, notably cybersecurity. To address this issue, we present and publicly release CySecBench, a comprehensive dataset containing 12662 prompts specifically designed to evaluate jailbreaking techniques in the cybersecurity domain. The dataset is organized into 10 distinct attack-type categories, featuring close-ended prompts to enable a more consistent and accurate assessment of jailbreaking attempts. Furthermore, we detail our methodology for dataset generation and filtration, which can be adapted to create similar datasets in other domains. To demonstrate the utility of CySecBench, we propose and evaluate a jailbreaking approach based on prompt obfuscation. Our experimental results show that this method successfully elicits harmful content from commercial black-box LLMs, achieving Success Rates (SRs) of 65% with ChatGPT and 88% with Gemini; in contrast, Claude demonstrated greater resilience with a jailbreaking SR of 17%. Compared to existing benchmark approaches, our method shows superior performance, highlighting the value of domain-specific evaluation datasets for assessing LLM security measures. Moreover, when evaluated using prompts from a widely used dataset (i.e., AdvBench), it achieved an SR of 78.5%, higher than the state-of-the-art methods.