Portfolio Allocation for Bayesian Optimization
Eric Brochu, Matthew Hoffman, Nando de Freitas Department of Computer Science University of British Columbia Vancouver, Canada
Abstract
Bayesian optimization with Gaussian processes has become an increasingly popular tool in the machine learning community. It is efficient and can be used when very little is known about the objective function, making it popular in expensive black-box optimization scenarios. It uses Bayesian methods to sample the objective efficiently using an acquisition function which incorporates the model’s estimate of the objective and the uncertainty at any given point. However, there are several different parameterized acquisition functions in the literature, and it is often unclear which one to use. Instead of using a single acquisition function, we adopt a portfolio of acquisition functions governed by an online multi-armed bandit strategy. We propose several portfolio strategies, the best of which we call GP-Hedge, and show that this method outperforms the best individual acquisition function. We also provide a theoretical bound on the algorithm’s performance.
中文速览
贝叶斯优化（Bayesian Optimization）在实际使用中面临一个棘手的问题：文献中存在多种采集函数（acquisition function），如期望提升（EI）、改进概率（PI）和置信上界（UCB），但没有哪一种对所有目标函数都表现最好，用户往往不知道该选哪个。为了解决这一选择难题，作者借鉴在线学习中的多臂老虎机（multi-armed bandit）框架，提出了一种名为 GP-Hedge 的组合策略——在每次迭代中，根据各采集函数过去推荐点的高斯过程预测值作为奖励信号，动态调整选用每种采集函数的概率，让表现好的函数获得更多"话语权"。在多个标准测试函数上的实验表明，GP-Hedge 的优化效果稳定优于任意单一采集函数，且作者还给出了算法累积遗憾的理论上界，证明其具有良好的收敛保证。这项工作的意义在于：它把"选哪个采集函数"这一繁琐的人工决策自动化了，让贝叶斯优化在无需领域先验的情况下更加稳健，对机器学习超参数调优、机器人控制等昂贵黑箱优化场景具有直接的实用价值。
原文 arXiv:1009.5419;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1009.5419v2