Abstract

Intelligence is our ability to learn appropriate responses to new stimuli and situations, and significant progress has been made in understanding how animals learn tasks by trial-and-error learning. The success of deep learning in end-to-end learning on a wide range of complex tasks is now fuelling the search for similar deep learning principles in the brain. While most work has focused on biologically plausible variants of error-backpropagation, learning in the brain seems to mostly adhere to a reinforcement learning paradigm, and while biologically plausible neural reinforcement learning has been proposed, these studies focused on shallow networks learning from compact and abstract sensory representations. Here, we demonstrate how these learning schemes generalize to deep networks with an arbitrary number of layers. The resulting reinforcement learning rule is equivalent to a particular form of error-backpropagation that trains one output unit at any time. We demonstrate the learning scheme on classical and hard image-classification benchmarks, namely MNIST, CIFAR10 and CIFAR100, cast as direct reward tasks, both for fully connected, convolutional and locally connected architectu