Title: DRBench: A Realistic Benchmark for Enterprise Deep Research Show Chinese title

Title: DRBench：企业深度研究的现实基准

Abstract: We introduce DRBench, a benchmark for evaluating AI agents on complex, open-ended deep research tasks in enterprise settings. Unlike prior benchmarks that focus on simple questions or web-only queries, DRBench evaluates agents on multi-step queries (for example, ``What changes should we make to our product roadmap to ensure compliance with this standard?") that require identifying supporting facts from both the public web and private company knowledge base. Each task is grounded in realistic user personas and enterprise context, spanning a heterogeneous search space that includes productivity software, cloud file systems, emails, chat conversations, and the open web. Tasks are generated through a carefully designed synthesis pipeline with human-in-the-loop verification, and agents are evaluated on their ability to recall relevant insights, maintain factual accuracy, and produce coherent, well-structured reports. We release 15 deep research tasks across 10 domains, such as Sales, Cybersecurity, and Compliance. We demonstrate the effectiveness of DRBench by evaluating diverse DR agents across open- and closed-source models (such as GPT, Llama, and Qwen) and DR strategies, highlighting their strengths, weaknesses, and the critical path for advancing enterprise deep research. Code is available at https://github.com/ServiceNow/drbench. Show Chinese abstract

Abstract: 我们引入了DRBench，这是一个用于在企业环境中评估人工智能代理在复杂、开放性的深度研究任务上的基准测试。 与以往专注于简单问题或仅限网络查询的基准不同，DRBench通过多步骤查询（例如，“为了确保符合此标准，我们应该对产品路线图做出哪些更改？”）来评估代理，这些查询需要从公共网络和私人公司知识库中识别支持事实。 每个任务都基于现实的用户角色和企业背景，涵盖包括生产力软件、云文件系统、电子邮件、聊天对话和开放网络在内的异构搜索空间。 任务通过一个精心设计的合成流程生成，并经过人工验证，代理在回忆相关见解、保持事实准确性以及生成连贯、结构良好的报告方面的能力得到评估。 我们在10个领域（如销售、网络安全和合规性）发布了15个深度研究任务。 我们通过在开源和闭源模型（如GPT、Llama和通义千问）以及深度研究策略上评估多种DR代理来展示DRBench的有效性，突出了它们的优势、劣势以及推动企业深度研究的关键路径。 代码可在https://github.com/ServiceNow/drbench获取。