Aha.
正在载入中英对照阅读…

arXiv:2609.12394 · 中英对照阅读

BlueLM-GUI 技术报告:面向自我改进移动端 GUI 智能体的以真实设备为核心的飞轮

BlueLM-GUI Technical Report: A Real-Device-Centric Flywheel for Self-Improving Mobile GUI Agents

vivo AI Lab https://github.com/vivo-ai-lab/BlueLM-GUI

中文速览

手机操作智能体在真实应用里常会遇到模拟环境没有的弹窗、网络波动和账号状态,训练数据用不上失败案例,固定测试也会随着模型变强而失去区分度。BlueLM-GUI把数据收集、训练和评测连成一个真实设备闭环:在数百台手机上运行任务,把成功轨迹用于训练,并修复失败或异常轨迹,从中提取新的训练信号;同时按明确指标设计并持续更新测试任务。模型在MobileGUI-VBench上取得87.4分,超过当时最佳闭源模型5.1分,在AndroidWorld上取得84.9分,是开源模型中的最佳成绩。研究说明,让模型在真实环境中学习、善用失败经验,并用会进化的测试持续找出短板,能提升手机智能体的实际表现和部署可靠性。

摘要

摘要。移动端 GUI 智能体正从多模块框架转向经端到端训练的原生模型,但工业部署仍面临三个长期存在的差距。沙箱训练会导致模型与生产环境之间出现分布失配;代价高昂的真实设备失败案例未得到充分利用;固定基准测试则会趋于饱和,失去指导迭代的能力。我们提出 BlueLM-GUI,这是一个 35B-A3B 移动端 GUI 智能体,构建为以真实设备为核心的飞轮,通过三项原则弥合这些差距。每个样本都至关重要:采用异构三系统共识评估和错误校正与推导模块的双轨流水线,将每条轨迹都转化为可用的监督信号。每次轨迹采样都基于真实环境:通过三个阶段的方案——持续预训练、监督微调和在数百部真实手机上进行智能体强化学习——使每次轨迹采样都扎根于真实生产环境,从而让模型学到的能力能够直接迁移至部署场景。每个查询都在演进:采用具有三个正交维度的配额驱动基准测试方法,实现精确归因,并使基准测试能够随着模型能力提升而系统性升级。BlueLM-GUI 在 MobileGUI-VBench 上取得 87.4 分,超过最佳闭源模型 5.1 分;在 AndroidWorld 上取得 84.9 分,创下开源模型最佳成绩,并可与闭源模型相媲美。这些结果表明,将模型训练和迭代改进扎根于真实设备及“每个样本都至关重要”“每次轨迹采样都基于真实环境”“每个查询都在演进”这三项原则,能够带来强大、稳健且可迁移的移动端 GUI 能力。

Abstract. Mobile GUI agents are shifting from multi-module frameworks to native models trained end-to-end, yet industrial deployment faces three persistent gaps. Sandbox training produces a distribution mismatch with production environments; expensive real-device failures remain underutilized; and fixed benchmarks saturate, losing the power to guide iteration. We present BlueLM-GUI, a 35B-A3B mobile GUI agent built as a real-device-centric flywheel that closes these gaps through three principles. Every Sample Matters: a dual-track pipeline with Heterogeneous Triple-System Consensus evaluation and an Error Correction & Derivation Module salvages every trajectory into usable supervision. Every Rollout Is Real: a three-stage recipe—continual pre-training, supervised fine-tuning, and agentic reinforcement learning on hundreds of real phones—grounds every rollout in real production environments, so the capability the model learns transfers directly to deployment. Every Query Evolves: a quota-driven benchmark methodology with three orthogonal axes enables precise attribution and allows the benchmark to be systematically upgraded as the model improves. BlueLM-GUI achieves 87.4 on MobileGUI-VBench, surpassing the best closed-source model by 5.1 points, and 84.9 on AndroidWorld, the best result among open-source models and competitive with closed-source models. These results demonstrate that grounding model training and iterative improvement in both real devices and the three Every principles yields strong, robust, and transferable mobile GUI capability.

术语表

Mobile GUI agent
移动端 GUI 智能体
BlueLM-GUI
BlueLM-GUI
35B-A3B
35B-A3B
real-device-centric flywheel
以真实设备为核心的飞轮
end-to-end model
端到端模型
multi-module framework
多模块框架
sandbox training
沙箱训练
distribution mismatch
分布失配
production environment
生产环境
trajectory
轨迹
Heterogeneous Triple-System Consensus
异构三系统共识
Error Correction & Derivation Module
错误校正与推导模块
continual pre-training
持续预训练
supervised fine-tuning (SFT)
监督微调(SFT)
agentic reinforcement learning (RL)
智能体强化学习(RL)
rollout
轨迹采样
quota-driven benchmark methodology
配额驱动的基准测试方法
orthogonal axes
正交维度
MobileGUI-VBench
MobileGUI-VBench
AndroidWorld
AndroidWorld
Large Language Model (LLM)
大语言模型(LLM)
vision-language-action (VLA)
视觉-语言-动作(VLA)
Vision-Language Model (VLM)
视觉语言模型(VLM)
perception
感知
grounding
定位
planning
规划
execution
执行
closed-loop learning
闭环学习
self-evolution
自我演进
counterfactual query derivation
反事实查询推导