GeckOpt: LLM System Efficiency via Intent-Based Tool SelectionConference: Great Lakes Symposium on VLSI 2024; June 12–14, 2024; Clearwater, FL, USAGreat Lakes Symposium on VLSI 2024 (GLSVLSI ’24), June 12–14, 2024, Clearwater, FL, USADOI: 10.1145/3649476.3658784ISBN: 979-8-4007-0605-9/24/06CCS: Computing methodologies Artificial intelligenceCCS: Computing methodologies Computer visionCCS: Computing methodologies Natural language processing
Michael Fore Affiliation: Microsoft Corporation , Reston , VA , USA email: , Simranjit Singh Affiliation: Microsoft Corporation , Silicon Valley Campus , CA , USA email: and Dimitrios Stamoulis Affiliation: Microsoft Corporation , Redmond , WA , USA email:
Abstract
In this preliminary study, we investigate a GPT-driven intent-based reasoning approach to streamline tool selection for large language models (LLMs) aimed at system efficiency. By identifying the intent behind user prompts at runtime, we narrow down the API toolset required for task execution, reducing token consumption by up to 24.6%. Early results on a real-world, massively parallel Copilot platform with over 100 GPT-4-Turbo nodes show cost reductions and potential towards improving LLM-based system efficiency.
原文 arXiv:2404.15804;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2404.15804v1