Aha.
正在载入中英对照阅读…

arXiv:1711.08536 · 中英对照阅读

No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World

Shreya Shankar、Yoni Halpern、Eric Breck、James Atwood、Jimbo Wilson、D. Sculley

中文速览

用于发展中国家的图像识别模型,可能会因为训练图片主要来自少数国家,而认不准当地的人和事。研究者检查了 ImageNet 和 Open Images 两个常用数据集的图片来源,并用众包图片和按国家收集的网络图片测试它们训练出的模型。结果发现,两个数据集都明显偏向美国和欧洲,而来自中国、印度等地的图片很少;模型对不同国家的图片表现也有明显差异,例如更容易认出美国的新郎,却常把其他地区的新郎认成盔甲或衣物。研究提醒我们,不能只看数据集规模或通用测试成绩,还要检查图片是否覆盖模型将要服务的人群,并在建库时把地域代表性当作重要目标。

摘要

Modern machine learning systems such as image classifiers rely heavily on large scale data sets for training. Such data sets are costly to create, thus in practice a small number of freely available, open source data sets are widely used. We suggest that examining the geo-diversity of open data sets is critical before adopting a data set for use cases in the developing world. We analyze two large, publicly available image data sets to assess geo-diversity and find that these data sets appear to exhibit an observable amerocentric and eurocentric representation bias. Further, we analyze classifiers trained on these data sets to assess the impact of these training distributions and find strong differences in the relative performance on images from different locales. These results emphasize the need to ensure geo-representation when constructing data sets for use in the developing world.

术语表

machine learning (ML)
机器学习(ML)
image classifier
图像分类器
large-scale data set
大规模数据集
open-source data set
开源数据集
geo-diversity
地理多样性
geo-representation
地理代表性
representation bias
代表性偏差
amerocentric bias
美洲中心偏差
eurocentric bias
欧洲中心偏差
ImageNet
ImageNet
Open Images
Open Images
Inception V3
Inception V3
pretrained model
预训练模型
image classification
图像分类
geo-location
地理位置
country-level geo-location
国家级地理位置信息
proxy information
代理信息
human-verified label
人工核验标签
crowdsourced data
众包数据
stress-test data
压力测试数据
geo-located web images
带地理位置标注的网络图像
geo-dependent mis-classification
地理位置相关的误分类
SmoothGrad
SmoothGrad
saliency map
显著性图
standard evaluation data split
标准评测数据划分
log likelihood
对数似然
training distribution
训练数据分布
developing world
发展中地区