Reducing malicious use of synthetic media research: Considerations and potential release practices for machine learning
Aviv Ovadya The Thoughtful Technology Project \AndJess Whittlestone Leverhulme Centre for the Future of Intelligence University of Cambridge
Abstract
The aim of this paper is to facilitate nuanced discussion around research norms and practices to mitigate the harmful impacts of advances in machine learning (ML). We focus particularly on the use of ML to create “synthetic media” (e.g. to generate or manipulate audio, video, images, and text), and the question of what publication and release processes around such research might look like, though many of the considerations discussed will apply to ML research more broadly. We are not arguing for any specific approach on when or how research should be distributed, but instead try to lay out some useful tools, analogies, and options for thinking about these issues.
中文速览
机器学习（ML）生成的"合成媒体"（synthetic media）——包括换脸视频、克隆声音、AI生成文本等——已被用于金融诈骗、骚扰记者和政治操纵，研究者迫切需要一套框架来评估和管控这类研究的潜在危害。本文梳理了ML研究导致伤害的三类信息风险（产品风险、数据风险、注意力风险）以及从技术能力到现实伤害的完整路径，同时借鉴生物安全、核技术等领域的风险管控经验，探讨哪些做法值得ML社区学习。研究发现，危害是否会发生并非非此即彼，而是取决于恶意行为者的意识、部署成本和持续收益等多重因素，且一旦技术被广泛普及、"棘轮"效应形成便难以逆转。为此，作者建议ML社区与领域专家合作评估风险图景，在重要会议上设立专门研讨机制，并建立支持"分级发布"的机构和规范——这对于在开放共享与防范滥用之间找到平衡具有重要的现实意义。
原文 arXiv:1907.11274;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1907.11274v2