标题： DREAM：通过分布建模实现文本到图像生成系统的可扩展红队测试 显示英文标题

标题： DREAM: Scalable Red Teaming for Text-to-Image Generative Systems via Distribution Modeling

摘要： 尽管整合了安全对齐和外部过滤器，文本到图像（T2I）生成模型仍然容易产生有害内容，例如色情或暴力图像。 这引发了关于无意暴露和潜在滥用的严重担忧。 红队测试旨在主动识别能够引发T2I系统不安全输出的各种提示（包括核心生成模型以及可能的外部安全过滤器和其他处理组件），目前被广泛认为是在实际部署前评估和提高安全性的必要方法。 然而，现有的自动化红队测试方法通常将提示发现视为一个孤立的、基于提示的优化任务，这限制了它们的可扩展性、多样性以及整体效果。 为了弥补这一差距，本文提出DREAM，一个可扩展的红队测试框架，用于从给定的T2I系统中自动发现多样化的有问题的提示。 与大多数之前的工作不同，DREAM直接建模目标系统有问题提示的概率分布，这使得可以对有效性和多样性进行显式优化，并在训练后实现高效的大规模采样。 为了在没有直接访问代表性训练样本的情况下实现这一点，我们受到基于能量模型的启发，并将目标重新表述为简单且易于处理的目标。 我们进一步引入GC-SPSA，一种高效的优化算法，通过长且可能不可微的T2I流水线提供稳定的梯度估计。 通过广泛的实验验证了DREAM的有效性，结果表明，在提示成功率和多样性方面，DREAM在各种T2I模型和安全过滤器上显著超越了9个最先进的基线方法。 显示英文摘要

摘要： Despite the integration of safety alignment and external filters, text-to-image (T2I) generative models are still susceptible to producing harmful content, such as sexual or violent imagery. This raises serious concerns about unintended exposure and potential misuse. Red teaming, which aims to proactively identify diverse prompts that can elicit unsafe outputs from the T2I system (including the core generative model as well as potential external safety filters and other processing components), is increasingly recognized as an essential method for assessing and improving safety before real-world deployment. Yet, existing automated red teaming approaches often treat prompt discovery as an isolated, prompt-level optimization task, which limits their scalability, diversity, and overall effectiveness. To bridge this gap, in this paper, we propose DREAM, a scalable red teaming framework to automatically uncover diverse problematic prompts from a given T2I system. Unlike most prior works that optimize prompts individually, DREAM directly models the probabilistic distribution of the target system's problematic prompts, which enables explicit optimization over both effectiveness and diversity, and allows efficient large-scale sampling after training. To achieve this without direct access to representative training samples, we draw inspiration from energy-based models and reformulate the objective into simple and tractable objectives. We further introduce GC-SPSA, an efficient optimization algorithm that provide stable gradient estimates through the long and potentially non-differentiable T2I pipeline. The effectiveness of DREAM is validated through extensive experiments, demonstrating that it surpasses 9 state-of-the-art baselines by a notable margin across a broad range of T2I models and safety filters in terms of prompt success rate and diversity.