Active Learning for Nonlinear System Identification with Guarantees
Horia Mania Michael I. Jordan Benjamin Recht Department of Electrical Engineering and Computer Science University of California, Berkeley
Abstract
While the identification of nonlinear dynamical systems is a fundamental building block of model-based reinforcement learning and feedback control, its sample complexity is only understood for systems that either have discrete states and actions or for systems that can be identified from data generated by i.i.d. random inputs. Nonetheless, many interesting dynamical systems have continuous states and actions and can only be identified through a judicious choice of inputs. Motivated by practical settings, we study a class of nonlinear dynamical systems whose state transitions depend linearly on a known feature embedding of state-action pairs. To estimate such systems in finite time identification methods must explore all directions in feature space. We propose an active learning approach that achieves this by repeating three steps: trajectory planning, trajectory tracking, and re-estimation of the system from all available data. We show that our method estimates nonlinear dynamical systems at a parametric rate, similar to the statistical rate of standard linear regression.
中文速览
用主动学习来解决非线性动力系统辨识（system identification）这一难题：对于状态转移依赖于已知特征嵌入（feature embedding）的非线性系统，随机输入往往无法覆盖特征空间的所有方向，导致系统参数根本无法被可靠估计。论文提出一种主动学习方法，循环执行三个步骤——基于当前估计规划参考轨迹、跟踪该轨迹以探索特征空间的高不确定性区域、再用所有历史数据重新估计系统参数——从而保证设计矩阵的最小奇异值以所需速率增长。理论分析证明，该方法获得的普通最小二乘（OLS）估计误差以参数化速率（parametric rate）收敛，与标准线性回归的统计率相近，仅额外依赖系统的可控性（controllability）参数和规划时域长度。这一结果填补了连续状态与输入的非线性系统辨识样本复杂度理论的空白，为基于模型的强化学习和反馈控制提供了有保证的数据采集方案。
原文 arXiv:2006.10277;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2006.10277v1