arXiv:2609.12394 · 中英对照阅读
BlueLM-GUI 技术报告:面向自我改进移动端 GUI 智能体的以真实设备为核心的飞轮
BlueLM-GUI Technical Report: A Real-Device-Centric Flywheel for Self-Improving Mobile GUI Agents
中文速览
手机操作智能体在真实应用里常会遇到模拟环境没有的弹窗、网络波动和账号状态,训练数据用不上失败案例,固定测试也会随着模型变强而失去区分度。BlueLM-GUI把数据收集、训练和评测连成一个真实设备闭环:在数百台手机上运行任务,把成功轨迹用于训练,并修复失败或异常轨迹,从中提取新的训练信号;同时按明确指标设计并持续更新测试任务。模型在MobileGUI-VBench上取得87.4分,超过当时最佳闭源模型5.1分,在AndroidWorld上取得84.9分,是开源模型中的最佳成绩。研究说明,让模型在真实环境中学习、善用失败经验,并用会进化的测试持续找出短板,能提升手机智能体的实际表现和部署可靠性。
摘要
摘要。移动端 GUI 智能体正从多模块框架转向经端到端训练的原生模型,但工业部署仍面临三个长期存在的差距。沙箱训练会导致模型与生产环境之间出现分布失配;代价高昂的真实设备失败案例未得到充分利用;固定基准测试则会趋于饱和,失去指导迭代的能力。我们提出 BlueLM-GUI,这是一个 35B-A3B 移动端 GUI 智能体,构建为以真实设备为核心的飞轮,通过三项原则弥合这些差距。每个样本都至关重要:采用异构三系统共识评估和错误校正与推导模块的双轨流水线,将每条轨迹都转化为可用的监督信号。每次轨迹采样都基于真实环境:通过三个阶段的方案——持续预训练、监督微调和在数百部真实手机上进行智能体强化学习——使每次轨迹采样都扎根于真实生产环境,从而让模型学到的能力能够直接迁移至部署场景。每个查询都在演进:采用具有三个正交维度的配额驱动基准测试方法,实现精确归因,并使基准测试能够随着模型能力提升而系统性升级。BlueLM-GUI 在 MobileGUI-VBench 上取得 87.4 分,超过最佳闭源模型 5.1 分;在 AndroidWorld 上取得 84.9 分,创下开源模型最佳成绩,并可与闭源模型相媲美。这些结果表明,将模型训练和迭代改进扎根于真实设备及“每个样本都至关重要”“每次轨迹采样都基于真实环境”“每个查询都在演进”这三项原则,能够带来强大、稳健且可迁移的移动端 GUI 能力。
Abstract. Mobile GUI agents are shifting from multi-module frameworks to native models trained end-to-end, yet industrial deployment faces three persistent gaps. Sandbox training produces a distribution mismatch with production environments; expensive real-device failures remain underutilized; and fixed benchmarks saturate, losing the power to guide iteration. We present BlueLM-GUI, a 35B-A3B mobile GUI agent built as a real-device-centric flywheel that closes these gaps through three principles. Every Sample Matters: a dual-track pipeline with Heterogeneous Triple-System Consensus evaluation and an Error Correction & Derivation Module salvages every trajectory into usable supervision. Every Rollout Is Real: a three-stage recipe—continual pre-training, supervised fine-tuning, and agentic reinforcement learning on hundreds of real phones—grounds every rollout in real production environments, so the capability the model learns transfers directly to deployment. Every Query Evolves: a quota-driven benchmark methodology with three orthogonal axes enables precise attribution and allows the benchmark to be systematically upgraded as the model improves. BlueLM-GUI achieves 87.4 on MobileGUI-VBench, surpassing the best closed-source model by 5.1 points, and 84.9 on AndroidWorld, the best result among open-source models and competitive with closed-source models. These results demonstrate that grounding model training and iterative improvement in both real devices and the three Every principles yields strong, robust, and transferable mobile GUI capability.
术语表
- Mobile GUI agent
- 移动端 GUI 智能体
- BlueLM-GUI
- BlueLM-GUI
- 35B-A3B
- 35B-A3B
- real-device-centric flywheel
- 以真实设备为核心的飞轮
- end-to-end model
- 端到端模型
- multi-module framework
- 多模块框架
- sandbox training
- 沙箱训练
- distribution mismatch
- 分布失配
- production environment
- 生产环境
- trajectory
- 轨迹
- Heterogeneous Triple-System Consensus
- 异构三系统共识
- Error Correction & Derivation Module
- 错误校正与推导模块
- continual pre-training
- 持续预训练
- supervised fine-tuning (SFT)
- 监督微调(SFT)
- agentic reinforcement learning (RL)
- 智能体强化学习(RL)
- rollout
- 轨迹采样
- quota-driven benchmark methodology
- 配额驱动的基准测试方法
- orthogonal axes
- 正交维度
- MobileGUI-VBench
- MobileGUI-VBench
- AndroidWorld
- AndroidWorld
- Large Language Model (LLM)
- 大语言模型(LLM)
- vision-language-action (VLA)
- 视觉-语言-动作(VLA)
- Vision-Language Model (VLM)
- 视觉语言模型(VLM)
- perception
- 感知
- grounding
- 定位
- planning
- 规划
- execution
- 执行
- closed-loop learning
- 闭环学习
- self-evolution
- 自我演进
- counterfactual query derivation
- 反事实查询推导