Aha.
正在载入中英对照阅读…

arXiv:2605.14831 · 中英对照阅读

有趣性作为未来压缩进展的归纳式启发法

Interestingness as an Inductive Heuristic for Future Compression Progress

Vincent Herrmann & Jürgen Schmidhuber

中文速览

要让能自我改进的系统持续进步,关键不只是从数据里学习,还得提前判断哪些任务和材料值得投入计算。作者把“有趣”定义成对未来压缩进步的预测,并用不同的概率先验分析复杂度随运行时间变化的轨迹。理论和三种通用计算系统上的实验都显示,过去的压缩突破确实能提示未来进展;但距离上次突破越久,近期再有突破的预期会按指数下降,而算法先验比长度先验对未来更乐观,同样的历史轨迹下预期发现量可高出约一倍。这个结果为自主生成学习任务提供了可检验的筛选思路,说明系统或许能依据自身的进步历史,优先探索仍可能带来新发现的方向。

摘要

迈向递归自我改进系统的瓶颈之一,是有趣性这一挑战:前瞻性地识别哪些任务或数据蕴含未来进展潜力的能力。我们将有趣性形式化为一种用于未来压缩进展的归纳式启发法,并借助 Kolmogorov复杂度(Kolmogorov Complexity)和算法统计学(Algorithmic Statistics)中的工具,研究其可预测性。通过分析长度先验(Length Prior)、算法先验(Algorithmic Prior)和速度先验(Speed Prior)下的复杂度—运行时间特征(complexity-runtime profile),我们证明,有趣性的归纳属性——即过去的进展能够预示未来发现的能力——在理论上可行,并得到经验支持。我们证明,未来预期进展取决于距上次观察到突破的时间远近,且呈指数关系。此外,我们表明,对于相同的观测特征,算法先验比长度先验乐观得多,预期发现量增加至原来的平方。这些发现通过三种多样化的通用计算范式上的实验得到了证实。

One of the bottlenecks on the way towards recursively self-improving systems is the challenge of interestingness: the ability to prospectively identify which tasks or data hold the potential for future progress. We formalize interestingness as an inductive heuristic for future compression progress and investigate its predictability using tools from Kolmogorov Complexity and Algorithmic Statistics. By analyzing complexity-runtime profiles under Length, Algorithmic, and Speed priors, we demonstrate that the inductive property of interestingness—the capacity for past progress to signal future discovery—is theoretically viable and empirically supported. We prove that expected future progress depends exponentially on the recency of the last observed breakthrough. Furthermore, we show that the Algorithmic Prior is significantly more optimistic than the Length Prior, yielding a quadratic increase in expected discovery for the same observed profile. These findings are experimentally confirmed across three diverse universal computational paradigms.

术语表

interestingness
有趣性
recursive self-improvement
递归自我改进
open-ended intelligence
开放式智能
inductive heuristic
归纳式启发法
compression progress
压缩进展
Kolmogorov Complexity
Kolmogorov复杂度
Algorithmic Statistics
算法统计学
complexity-runtime profile
复杂度—运行时间特征
Length Prior
长度先验
Algorithmic Prior
算法先验
Speed Prior
速度先验
stagnation length
停滞时长
compression breakthrough
压缩突破
Large Language Model (LLM)
大语言模型(LLM)
Reinforcement Learning (RL)
强化学习(RL)
model collapse
模型崩溃
intrinsic motivation
内在动机
count-based exploration
基于计数的探索
space coverage
空间覆盖
maximum entropy
最大熵
prediction error
预测误差
epistemic uncertainty
认知不确定性
aleatoric uncertainty
偶然不确定性
Noisy TV problem
噪声电视问题
information gain
信息增益
Bayesian surprise
贝叶斯惊奇度
Kullback-Leibler divergence
Kullback–Leibler散度
learning progress
学习进展
competence progress
能力进展
negative log-likelihood (NLL)
负对数似然(NLL)