A Causal View on Robustness of Neural Networks
Cheng Zhang Microsoft Research、Kun Zhang Carnegie Mellon University、Yingzhen Li Microsoft Research Equal contribution
Abstract
We present a causal view on the robustness of neural networks against input manipulations, which applies not only to traditional classification tasks but also to general measurement data. Based on this view, we design a deep causal manipulation augmented model (deep CAMA) which explicitly models possible manipulations on certain causes leading to changes in the observed effect. We further develop data augmentation and test-time fine-tuning methods to improve deep CAMA’s robustness. When compared with discriminative deep neural networks, our proposed model shows superior robustness against unseen manipulations. As a by-product, our model achieves disentangled representation which separates the representation of manipulations from those of other latent causes.
中文速览
神经网络在面对旋转、平移、噪声等"未见过的"输入干扰时往往一败涂地，根本原因在于它们只学习了输入与标签之间的统计相关性，而没有理解数据背后的因果生成机制。本文从因果推断的角度重新审视这一问题，把数据生成过程拆分为目标变量、不可干预的潜在原因、以及可人工干预的操纵变量三类，据此设计了一种深度因果操纵增强模型（deep CAMA），用生成模型显式地将"操纵"与"其他潜在原因"分离建模，并配套了训练时数据增强和测试时微调两种策略，让模型在测试阶段无需标签就能自适应地学习未知的操纵方式。在MNIST手写数字及测量数据集上的实验表明，deep CAMA对未见过的各类扰动的鲁棒性显著优于判别式深度神经网络，同时还自然地学到了解耦的潜在表示。这项工作的意义在于，它为提升神经网络鲁棒性提供了一条有理论根基的因果建模路径，而不是简单地堆砌更多种类的数据增强。
原文 arXiv:2005.01095;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2005.01095v3