Title: From Grounding to Planning: Benchmarking Bottlenecks in Web Agents Show Chinese title

Title: 从接地到规划：网络代理中的基准瓶颈

Abstract: General web-based agents are increasingly essential for interacting with complex web environments, yet their performance in real-world web applications remains poor, yielding extremely low accuracy even with state-of-the-art frontier models. We observe that these agents can be decomposed into two primary components: Planning and Grounding. Yet, most existing research treats these agents as black boxes, focusing on end-to-end evaluations which hinder meaningful improvements. We sharpen the distinction between the planning and grounding components and conduct a novel analysis by refining experiments on the Mind2Web dataset. Our work proposes a new benchmark for each of the components separately, identifying the bottlenecks and pain points that limit agent performance. Contrary to prevalent assumptions, our findings suggest that grounding is not a significant bottleneck and can be effectively addressed with current techniques. Instead, the primary challenge lies in the planning component, which is the main source of performance degradation. Through this analysis, we offer new insights and demonstrate practical suggestions for improving the capabilities of web agents, paving the way for more reliable agents. Show Chinese abstract

Abstract: 基于网络的通用代理在与复杂的网络环境交互中变得越来越重要，但在现实世界中的网络应用程序中的表现仍然很差，即使使用最先进的前沿模型，其准确性也非常低。 我们观察到，这些代理可以被分解为两个主要部分：规划和基础操作。 然而，大多数现有的研究将这些代理视为黑箱，专注于端到端评估，这阻碍了有意义的改进。 我们明确了规划和基础操作组件之间的区别，并通过改进Mind2Web数据集上的实验进行了新的分析。 我们的工作分别提出了每个组件的新基准，识别出限制代理性能的瓶颈和痛点。 与普遍假设相反，我们的研究结果表明，基础操作并不是一个重要的瓶颈，可以通过当前的技术有效解决。 相反，主要挑战在于规划组件，这是性能下降的主要来源。 通过这项分析，我们提供了新的见解，并展示了提高网络代理能力的实际建议，为更可靠的代理铺平了道路。