A Biologically Plausible Learning Rule for Deep Learning in the Brain
Isabella Pozzi Vision、Cognition Group Netherlands Institute for Neuroscience Amsterdam, The Netherlands、Sander M. Bohté Machine Learning Group Centrum Wiskunde、Informatica Amsterdam, The Netherlands、Pieter R. Roelfsema Vision、Cognition Group Netherlands Institute for Neuroscience Amsterdam, The Netherlands
Abstract
Intelligence is our ability to learn appropriate responses to new stimuli and situations, and significant progress has been made in understanding how animals learn tasks by trial-and-error learning. The success of deep learning in end-to-end learning on a wide range of complex tasks is now fuelling the search for similar deep learning principles in the brain. While most work has focused on biologically plausible variants of error-backpropagation, learning in the brain seems to mostly adhere to a reinforcement learning paradigm, and while biologically plausible neural reinforcement learning has been proposed, these studies focused on shallow networks learning from compact and abstract sensory representations. Here, we demonstrate how these learning schemes generalize to deep networks with an arbitrary number of layers. The resulting reinforcement learning rule is equivalent to a particular form of error-backpropagation that trains one output unit at any time. We demonstrate the learning scheme on classical and hard image-classification benchmarks, namely MNIST, CIFAR10 and CIFAR100, cast as direct reward tasks, both for fully connected, convolutional and locally connected architectu
中文速览
动物和人类通过试错来学习，大脑中这一过程被认为遵循强化学习范式，但此前生物合理的神经强化学习模型只能处理单隐层、低维输入的浅层网络，无法应对真实视觉任务的复杂性。研究者将注意力门控强化学习规则（AGREL）推广到任意深度的多层网络，提出了 Q-AGREL 算法：前向传播计算 Q 值并选择动作，随后由获胜输出单元发出类似注意力反馈的信号逐层向下传播，结合全局奖励预测误差（reward prediction error）对每个突触进行仅依赖局部信息的四因子更新，并在理论上证明该规则等价于每次只训练一个输出单元的特殊误差反向传播。在 MNIST、CIFAR10 和 CIFAR100 三个图像分类基准测试中，Q-AGREL 的最终精度与监督式反向传播相当，仅需多消耗约 1.5–2.5 倍的训练轮次。这项工作表明大脑凭借多层结构同样有能力以生物合理的方式解决深度信用分配问题，为理解皮层如何实现类深度学习的试错学习提供了具体的计算框架。
原文 arXiv:1811.01768;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1811.01768v3