The Red Queen G\"odel Machine: Co-Evolving Agents and Their Evaluators

摘要

Self-improving agents aim to turn gains in coding and reasoning into a recursive loop, where each stronger agent can produce better variants of itself. Search methods such as the Darwin and Huxley Gödel Machines (DGM and HGM) (Zhang et al., 2026a; Wang et al., 2026a) set the open-source state-of-the-art on agentic coding tasks by editing their own code and retaining variants that improve an external utility signal. HyperAgents (Zhang et al., 2026b) then extended this self-improvement beyond coding to task agents in arbitrary domains. Despite this progress, current search methods still depend on stationary evaluation fixed outside the improvement loop. This is unlike biological evolution, where, under Van Valen’s Red Queen hypothesis (Van Valen, 1973), each species adapts to competitors that evolve in turn. This dependence constrains self-improvement when a target task has no direct benchmark, as for paper and proof writing, unlike paper reviewing and proof grading (Lu et al., 2026; Yamada et al., 2025; Luong et al., 2025), when evaluation is slow or weakly informative (Levine et al., 2018; Burger et al., 2020; Andrychowicz et al., 2017), and when static benchmarks saturate or become vulnerable to reward hacking (Yang et al., 2026; Ott et al., 2022).