Can Intelligence Explode?
Marcus Hutter Research School of Computer Science Australian National University、Department of Computer Science ETH Zürich, Switzerland
Abstract
The technological singularity refers to a hypothetical scenario in which technological advances virtually explode. The most popular scenario is the creation of super-intelligent algorithms that recursively create ever higher intelligences. It took many decades for these ideas to spread from science fiction to popular science magazines and finally to attract the attention of serious philosophers. David Chalmers’ (JCS 2010) article is the first comprehensive philosophical analysis of the singularity in a respected philosophy journal. The motivation of my article is to augment Chalmers’ and to discuss some issues not addressed by him, in particular what it could mean for intelligence to explode. In this course, I will (have to) provide a more careful treatment of what intelligence actually is, separate speed from intelligence explosion, compare what super-intelligent participants and classical human observers might experience and do, discuss immediate implications for the diversity and value of life, consider possible bounds on intelligence, and contemplate intelligences right at the singularity. Contents 1 Introduction 2 Will there be a Singularity 3 The Singularity from the Outside
中文速览
人工智能超过人类水平后会不断自我改进、引发"智能爆炸"，这一被称为"技术奇点"的假想场景究竟意味着什么，目前哲学层面的严肃讨论还很有限。作者在Chalmers已有分析的基础上，着重追问"智能爆炸"这个概念本身是否站得住脚——为此他系统区分了速度爆炸与真正的智能增长，深入探讨了智能的本质与数学定义，并分别从"局外观察者"和"内部参与者"两个视角审视奇点体验的差异。研究发现，若一切仅是计算速度均匀提速，内部主体未必能感受到智能的跃升；而对外部人类观察者而言，超级智能体的行为可能如同随机噪声，根本无从辨认；此外，智能在理论上可能存在上界，这将从根本上限制所谓的"智能奇点"。这项工作的重要性在于，它为我们提供了一套更严谨的概念框架，避免将速度加速与真实的智能提升混为一谈，同时也提醒人们：虚拟世界中的复制与多样性扩张，在奇点到来之前便会深刻改变生命与价值的意义。
原文 arXiv:1202.6177;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1202.6177v1