标题： 工具代理：强化学习中分层决策的研究 显示英文标题

标题： Agent-as-Tool: A Study on the Hierarchical Decision Making with Reinforcement Learning

摘要： 大型语言模型（LLMs）近年来已成为人工智能领域最重要的技术进步之一。 它们理解、生成和推理自然语言的能力改变了我们与AI系统互动的方式。 随着基于LLM的智能体和基于强化学习的推理模型的发展，将强化学习应用于智能体框架的研究已成为新的研究重点。 然而，所有先前的研究都面临同时决定工具调用过程和推理过程的挑战，推理链仅依赖于工具提供的未经处理的原始结果，其中包含与任务无关的冗余信息和符号，这给模型的推理能力带来了沉重负担。 因此，在我们的研究中，我们提出了一种分层框架Agent-as-tool，将工具调用过程和推理过程分离，使模型能够专注于口头推理过程，而工具调用过程由另一个智能体处理。 我们的工作仅在180个样本上进行了轻微的强化微调，就取得了可比的结果，并在Bamboogle上取得了优异的表现，精确匹配率达到63.2%，覆盖精确匹配率达到75.2%，在精确匹配上超过Search-R1 4.8%，在覆盖精确匹配上超过3.2%。 显示英文摘要

摘要： Large Language Models (LLMs) have emerged as one of the most significant technological advancements in artificial intelligence in recent years. Their ability to understand, generate, and reason with natural language has transformed how we interact with AI systems. With the development of LLM-based agents and reinforcement-learning-based reasoning models, the study of applying reinforcement learning in agent frameworks has become a new research focus. However, all previous studies face the challenge of deciding the tool calling process and the reasoning process simultaneously, and the chain of reasoning was solely relied on the unprocessed raw result with redundant information and symbols unrelated to the task from the tool, which impose a heavy burden on the model's capability to reason. Therefore, in our research, we proposed a hierarchical framework Agent-as-tool that detach the tool calling process and the reasoning process, which enables the model to focus on the verbally reasoning process while the tool calling process is handled by another agent. Our work had achieved comparable results with only a slight reinforcement fine-tuning on 180 samples, and had achieved exceptionally well performance in Bamboogle with 63.2% of exact match and 75.2% in cover exact match, exceeding Search-R1 by 4.8% in exact match and 3.2% in cover exact match.