On the Sample Complexity of the Linear Quadratic Regulator
Sarah Dean♯, Horia Mania♯, Nikolai Matni†, Benjamin Recht♯, and Stephen Tu♯ ♯ University of California, Berkeley ††\dagger California Institute of Technology
Abstract
This paper addresses the optimal control problem known as the Linear Quadratic Regulator in the case when the dynamics are unknown. We propose a multi-stage procedure, called Coarse-ID control, that estimates a model from a few experimental trials, estimates the error in that model with respect to the truth, and then designs a controller using both the model and uncertainty estimate. Our technique uses contemporary tools from random matrix theory to bound the error in the estimation procedure. We also employ a recently developed approach to control synthesis called System Level Synthesis that enables robust control design by solving a quasiconvex optimization problem. We provide end-to-end bounds on the relative error in control cost that are optimal in the number of parameters and that highlight salient properties of the system to be controlled such as closed-loop sensitivity and optimal control magnitude. We show experimentally that the Coarse-ID approach enables efficient computation of a stabilizing controller in regimes where simple control schemes that do not take the model uncertainty into account fail to stabilize the true system.
中文速览
当系统动力学完全未知时，如何设计一个既能保证稳定性、又能提供性能保障的最优控制器？论文提出了一套名为"粗粒度辨识控制"（Coarse-ID control）的三步框架：先用最小二乘法从少量实验数据中估计出一个粗略的线性动力学模型，再借助随机矩阵理论给出模型误差的有限样本概率上界，最后通过"系统层综合"（System Level Synthesis, SLS）将鲁棒控制器设计转化为一个拟凸优化问题来求解。理论分析证明，所需样本量在参数个数上达到最优阶，且最终控制代价与已知动力学时的最优代价之间的相对误差可以被显式界定。实验结果表明，在数据量相同的情况下，忽略模型不确定性的简单方法频繁导致闭环系统不稳定，而Coarse-ID方案却能可靠地找到稳定控制器——这为理解"数据驱动控制到底需要多少数据才够用"这一根本性问题奠定了严格的理论基础。
原文 arXiv:1710.01688;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1710.01688v3