A Gaussian Process perspective on Convolutional Neural Networks
Anastasia Borovykh Dipartimento di Matematica, Università di Bologna, Bologna, Italy.e-mail:
Abstract
In this paper we cast the well-known convolutional neural network in a Gaussian process perspective. In this way we hope to gain additional insights into the performance of convolutional networks, in particular understand under what circumstances they tend to perform well and what assumptions are implicitly made in the network. While for fully-connected networks the properties of convergence to Gaussian processes have been studied extensively, little is known about situations in which the output from a convolutional network approaches a multivariate normal distribution.
中文速览
卷积神经网络（CNN）在实践中表现出色，但人们对它为何有效的理论理解仍然有限；这项研究试图从高斯过程（Gaussian Process, GP）的角度为CNN提供新的理论解释。作者将CNN的逐层卷积运算重新表述为随机变量之和，并借助一种广义中心极限定理（Lyapunov型界）证明：即使卷积层中各项并不同分布，只要滤波器宽度足够大，第一层输出就会趋近于多元正态分布，从而对应一个GP先验，并推导出相应的协方差核函数。数值实验进一步表明，即便在深层网络中CLT不能严格成立，相对较小的滤波器宽度就已足以让输出迅速逼近GP行为。这一结论将全连接网络收敛到GP的经典理论推广到了CNN，为理解卷积网络的归纳偏置与泛化性能提供了坚实的理论基础。
原文 arXiv:1810.10798;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1810.10798v2