AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts
Tongshuang Wu University of WashingtonUSA , Michael Terry Google ResearchUSA and Carrie J. Cai Google ResearchUSA
Abstract
Although large language models (LLMs) have demonstrated impressive potential on simple tasks, their breadth of scope, lack of transparency, and insufficient controllability can make them less effective when assisting humans on more complex tasks. In response, we introduce the concept of Chaining LLM steps together, where the output of one step becomes the input for the next, thus aggregating the gains per step. We first define a set of LLM primitive operations useful for Chain construction, then present an interactive system where users can modify these Chains, along with their intermediate results, in a modular way. In a 20-person user study, we found that Chaining not only improved the quality of task outcomes, but also significantly enhanced system transparency, controllability, and sense of collaboration. Additionally, we saw that users developed new ways of interacting with LLMs through Chains: they leveraged sub-tasks to calibrate model expectations, compared and contrasted alternative strategies by observing parallel downstream effects, and debugged unexpected model outputs by “unit-testing” sub-components of a Chain. In two case studies, we further explore how LLM Chains ma
中文速览
大型语言模型（LLM）在简单任务上表现出色，但面对复杂任务时往往因透明度低、难以控制而让用户沮丧。研究者提出了"链式调用"（LLM Chaining）的思路：把一个复杂任务拆成若干小步骤，每一步用独立的提示词驱动模型完成一个子任务，前一步的输出自动成为下一步的输入，最终将各步的收益叠加起来。他们还梳理了73个真实LLM应用场景，归纳出一套"基础操作"原语，并在此基础上构建了一个可交互界面，让用户能逐步运行链条、编辑中间结果、调整步骤结构。在20人的用户实验中，链式方法不仅让约82%的任务取得了更高质量的输出，还显著提升了用户对系统透明度、可控性和协作感的评价，用户甚至自发摸索出"单元测试"子步骤、对比并行策略等新的交互方式。这项工作表明，无需修改底层模型，仅靠合理的任务分解与中间结果可视化，就能大幅改善人与黑箱LLM协作时的体验，为未来人机协同系统的设计提供了重要参考。
原文 arXiv:2110.01691;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2110.01691v3