The Risks of Machine Learning Systems
Samson Tan Salesforce Research Asia, and School of Computing, National University of Singapore , Araz Taeihagh https://orcid.org/0000-0002-4812-4745 Lee Kuan Yew School of Public Policy and CTIC, National University of Singapore and Kathy Baxter Office of Ethical、Humane Use, Salesforce
Abstract
The speed and scale at which machine learning (ML) systems are deployed are accelerating even as an increasing number of studies highlight their potential for negative impact. There is a clear need for companies and regulators to manage the risk from proposed ML systems before they harm people. To achieve this, private and public sector actors first need to identify the risks posed by a proposed ML system. A system’s overall risk is influenced by its direct and indirect effects. However, existing frameworks for ML risk/impact assessment often address an abstract notion of risk or do not concretize this dependence. Keeping discussions of risk at an abstract level puts the onus of defining them on the assessor, which may result in incomplete and inconsistent assessments.
中文速览
机器学习（ML）系统在给生活带来便利的同时，也可能造成歧视、隐私泄露、系统失效等各种危害，但现有的风险评估框架要么过于抽象，要么只关注组织内部风险，缺乏一套专门针对ML系统、能把直接风险和社会影响串联起来的系统性分类。作者提出了"机器学习系统风险框架"（MLSR），将ML风险分为"一阶风险"（源自系统自身设计与实现缺陷，如性能不可靠、安全漏洞）和"二阶风险"（一阶风险与真实世界交互后产生的后果，如侵犯人权、加剧社会不平等），并详细梳理了影响每类风险的具体因素。框架用大量真实事件和已有研究加以验证，将技术层面的系统风险与伦理AI社区关注的人权风险统一在同一体系下。这一分类法填补了现有立法（如欧盟AI法案）和影响评估工具在风险定义上的空白，为企业和监管者开展全面、可操作的ML风险评估提供了实质性指引。
原文 arXiv:2204.09852;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2204.09852v1