ProTIP: Progressive Tool Retrieval Improves Planning
Raviteja Anantha*, Bortik Bandyopadhyay*, Anirudh Kashi, Sayantan Mahinder, Andrew W Hill, Srinivas Chappidi Apple
Abstract
Large language models (LLMs) are increasingly employed for complex multi-step planning tasks, where the tool retrieval (TR) step is crucial for achieving successful outcomes. Two prevalent approaches for TR are single-step retrieval, which utilizes the complete query, and sequential retrieval using task decomposition (TD), where a full query is segmented into discrete atomic subtasks. While single-step retrieval lacks the flexibility to handle “inter-tool dependency," the TD approach necessitates maintaining “subtask-tool atomicity alignment," as the toolbox can evolve dynamically. To address these limitations, we introduce the Progressive Tool retrieval to Improve Planning (ProTIP) framework. ProTIP is a lightweight, contrastive learning-based framework that implicitly performs TD without the explicit requirement of subtask labels, while simultaneously maintaining subtask-tool atomicity. On the ToolBench dataset, ProTIP outperforms the ChatGPT task decomposition-based approach by a remarkable margin, achieving a 24% improvement in Recall@K=10 for TR and a 41% enhancement in tool accuracy for plan generation.
中文速览
大规模语言模型在处理需要调用外部工具的多步规划任务时，如何精准检索出合适的工具是关键难题：单步检索无法处理工具间的依赖关系，而先把任务拆解再逐步检索的方法则需要昂贵的大模型或大量人工标注来保证"子任务与工具的原子对齐"。为此，论文提出了 ProTIP（Progressive Tool retrieval to Improve Planning）框架，它用对比学习对 BERT 编码器进行轻量微调，让模型在向量空间中通过迭代地从查询嵌入里"减去"已检索工具的嵌入来隐式完成任务拆解，无需任何子任务标注，同时自然地融入执行历史。在 ToolBench 数据集上，ProTIP 相比基于 ChatGPT 任务拆解的方法，工具检索的 Recall@K=10 提升了 24%，规划生成中的工具准确率提升了 41%，首次证明轻量级非大模型微调方案可以超越依赖大模型的检索方法，对降低实际部署成本具有重要意义。
原文 arXiv:2312.10332;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2312.10332v1