Scaling Scaling Laws with Board Games
Andy L. Jones London, United Kingdom
Abstract
The largest experiments in machine learning now require resources far beyond the budget of all but a few institutions. Fortunately, it has recently been shown that the results of these huge experiments can often be extrapolated from the results of a sequence of far smaller, cheaper experiments. In this work, we show that not only can the extrapolation be done based on the size of the model, but on the size of the problem as well. By conducting a sequence of experiments using AlphaZero and Hex, we show that the performance achievable with a fixed amount of compute degrades predictably as the game gets larger and harder. Along with our main result, we further show that the test-time and train-time compute available to an agent can be traded off while maintaining performance.
中文速览
训练超大规模AI模型所需的算力已超出大多数研究机构的承受范围，而"规模缩放定律（scaling laws）"提供了一种出路——用小模型的实验结果来预测大模型的行为。本文将这一思路推进一步：不仅对模型规模进行缩放，还对问题规模本身进行缩放。研究者以棋盘游戏Hex为实验平台，用AlphaZero训练了约200个不同大小的模型，发现在固定算力预算下，随着棋盘尺寸增大，模型可达到的性能会以可预测的规律下降——具体而言，达到完美走法所需的训练算力随棋盘每增大一格就提高约7倍。更关键的是，在小棋盘上拟合出的缩放曲线能够准确外推到更大棋盘的性能表现，且随着纳入拟合的小棋盘数量增加，预测误差呈指数级下降。此外，研究还发现训练时算力与推理时算力可以相互替换：训练算力每多投入10倍，推理时的搜索算力就能减少约15倍而不损失性能。这项工作首次将缩放定律引入多智能体强化学习领域，并证明"问题难度"本身也是缩放定律的一个维度，为资源有限的研究者提供了更廉价地预测大规模实验结果的新途径。
原文 arXiv:2104.03113;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2104.03113v2