LEXI: Large Language Models Experimentation Interface
Guy Laban∗ Department of Computer Science and Technology University of Cambridge Cambridge, UK、Tomer Laban、Hatice Gunes Department of Computer Science and Technology University of Cambridge Cambridge, UK
Abstract
The recent developments in Large Language Models (LLM), mark a significant moment in the research and development of social interactions with artificial agents. These agents are widely deployed in a variety of settings, with potential impact on users. However, the study of social interactions with agents powered by LLM is still emerging, limited by access to the technology and to data, the absence of standardised interfaces, and challenges to establishing controlled experimental setups using the currently available business-oriented platforms. To answer these gaps, we developed LEXI, LLMs Experimentation Interface, an open-source tool enabling the deployment of artificial agents powered by LLM in social interaction behavioural experiments. Using a graphical interface, LEXI allows researchers to build agents, and deploy them in experimental setups along with forms and questionnaires while collecting interaction logs and self-reported data. The outcomes of usability testing indicate LEXI’s broad utility, high usability and minimum mental workload requirement, with distinctive benefits observed across disciplines. A proof-of-concept study exploring the tool’s efficacy in evaluating so
中文速览
大语言模型（LLM）驱动的聊天代理正被广泛应用于各类社交场景，但研究者想要在受控实验中部署这类代理却面临重重障碍：缺乏统一接口、商业平台难以支持实验设计、交互日志和问卷数据也难以同步采集。为此，研究团队开发了开源工具 LEXI（LLMs Experimentation Interface），让研究者无需编程基础，只需通过图形界面就能配置 LLM 代理、设置实验条件、嵌入问卷并自动收集对话日志与自我报告数据。可用性测试显示，LEXI 易用性高、认知负荷低，跨学科用户均给出正面评价；概念验证实验进一步表明，该工具能产出高质量数据——对比"共情型"与"中性型"代理后发现，人们认为共情代理更具社交性，并向其发送更长、更积极的消息。LEXI 的意义在于填补了社会与行为科学研究者和 LLM 技术之间的鸿沟，为人机交互（HAI）领域的大规模、可重复实验研究提供了低门槛的基础设施。
原文 arXiv:2407.01488;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2407.01488v2