The Variational Fair Autoencoder
Christos Louizos∗, Kevin Swersky×, Yujia Li×, Max Welling∗†‡, Richard Zemel׆ ∗ Machine Learning Group, University of Amsterdam ×Department of Computer Science, University of Toronto † Canadian Institute for Advanced Research (CIFAR) ‡ University of California, Irvine {kswersky,
Abstract
We investigate the problem of learning representations that are invariant to certain nuisance or sensitive factors of variation in the data while retaining as much of the remaining information as possible. Our model is based on a variational autoencoding architecture (Kingma & Welling, 2014; Rezende et al., 2014) with priors that encourage independence between sensitive and latent factors of variation. Any subsequent processing, such as classification, can then be performed on this purged latent representation. To remove any remaining dependencies we incorporate an additional penalty term based on the “Maximum Mean Discrepancy” (MMD) (Gretton et al., 2006) measure. We discuss how these architectures can be efficiently trained on data and show in experiments that this method is more effective than previous work in removing unwanted sources of variation while maintaining informative latent representations.
中文速览
想让机器学习模型"公平"或"跨域"——即从数据表示中剔除性别、年龄、领域等敏感/干扰因素，同时尽量保留其他有用信息——是一个核心难题。作者提出了"变分公平自编码器"（Variational Fair Autoencoder，VFAE），在变分自编码器（VAE）框架下，用因子化先验来鼓励隐变量与敏感变量相互独立，再额外引入最大均值差异（Maximum Mean Discrepancy，MMD）惩罚项，强制对齐不同敏感变量取值下隐变量的边缘分布，从而彻底压制残余依赖。在公平分类（德国信用、美国收入、医疗健康数据集）和跨域情感分析（亚马逊评论）等多项实验中，VFAE在去除敏感信息的同时保持了更强的预测性能，整体表现优于此前方法。这项工作为同时兼顾"公平性"与"信息保留"提供了一个原则性的概率框架，对算法公平和领域自适应研究均有直接参考价值。
原文 arXiv:1511.00830;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1511.00830v6