DISCO Nets: DISsimilarity COefficient Networks
Diane Bouchacourt University of Oxford、M. Pawan Kumar University of Oxford、Sebastian Nowozin Microsoft Research Cambridge
Abstract
We present a new type of probabilistic model which we call DISsimilarity COefficient Networks (DISCO Nets). DISCO Nets allow us to efficiently sample from a posterior distribution parametrised by a neural network. During training, DISCO Nets are learned by minimising the dissimilarity coefficient between the true distribution and the estimated distribution. This allows us to tailor the training to the loss related to the task at hand. We empirically show that (i) by modeling uncertainty on the output value, DISCO Nets outperform equivalent non-probabilistic predictive networks and (ii) DISCO Nets accurately model the uncertainty of the output, outperforming existing probabilistic models based on deep neural networks.
中文速览
训练神经网络预测结构化输出时，模型往往只给出单一点估计，无法捕捉输出的真实不确定性，而现有的概率深度模型又普遍采用与任务评估损失无关的通用目标函数，导致训练目标和评估目标脱节。DISCO Nets（不相似系数网络）通过在神经网络输入端注入随机噪声，让网络直接从后验分布中采样多个候选输出，并以Rao提出的"不相似系数"作为训练目标——该系数可接入任意非负对称损失函数，从而让训练目标与具体任务的评估损失保持一致。在手部姿态估计等结构化预测任务上的实验表明，DISCO Nets既比不建模不确定性的确定性网络表现更好，也比现有概率深度模型（如VAE、GAN）更准确地刻画输出不确定性。这一工作的价值在于提供了一个架构灵活、训练简单、且能按需定制损失函数的概率预测框架，为追求高精度结构化输出预测的实际应用打开了新思路。
原文 arXiv:1606.02556;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1606.02556v5