ML Privacy Meter: Aiding Regulatory Compliance by Quantifying the Privacy Risks of Machine Learning
Sasi Kumar Murakonda Reza Shokri Data Privacy and Trustworthy ML Research Lab National University of Singapore
Abstract
When building machine learning models using sensitive data, organizations should ensure that the data processed in such systems is adequately protected. For projects involving machine learning on personal data, Article 35 of the GDPR mandates it to perform a Data Protection Impact Assessment (DPIA). In addition to the threats of illegitimate access to data through security breaches, machine learning models pose an additional privacy risk to the data by indirectly revealing about it through the model predictions and parameters. Guidances released by the Information Commissioner’s Office (UK) and the National Institute of Standards and Technology (US) emphasize on the threats to data from models and recommend organizations to account for and estimate these risks to comply with data protection regulations. Hence, there is an immediate need for a tool that can quantify the privacy risks to data from models.
中文速览
机器学习模型在训练后会通过预测结果和模型参数间接泄露训练数据的隐私，这种风险在现有的数据保护合规流程（如GDPR要求的数据保护影响评估，DPIA）中往往被忽视。为此，研究团队开发了 ML Privacy Meter 这一工具，通过模拟成员推断攻击（membership inference attack）来量化模型对训练数据的隐私泄露风险，既支持只能观察模型预测的黑盒场景，也支持可访问模型参数的白盒场景。工具会为每条训练记录生成风险评分，并以ROC曲线和曲线下面积（AUC）等指标呈现整体泄露程度，帮助从业者定位高风险数据点。这一工具的重要性在于，它能直接对接合规需求——不仅帮助组织识别和评估隐私威胁，还能辅助选择正则化、差分隐私（differential privacy）等缓解措施，并为差分隐私的隐私参数 ε 提供实际风险参照，从而在隐私保护与模型精度之间找到更合理的平衡点。
原文 arXiv:2007.09339;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2007.09339v1