Co-Evolving LLM Coder and Unit Tester via Reinforcement Learning
Yinjie Wang1∗ Ling Yang2,4∗ Ye Tian3 Ke Shen4 Mengdi Wang2 1University of Chicago 2Princeton University 3Peking University 4ByteDance Seed
Abstract
We propose CURE, a novel reinforcement learning framework with a dedicated reward design that co-evolves coding and unit test generation capabilities based on their interaction outcomes, without any ground-truth code as supervision. This approach enables flexible and scalable training and allows the unit tester to learn directly from the coder’s mistakes. Our derived ReasonFlux-Coder 7B and 14B models improve code generation accuracy by $5.3\%$ and Best of N accuracy by $9.0\%$ after optimization on Qwen2.5-Instruct models, outperforming similarly sized Qwen-Coder, DeepSeek-Coder, and Seed-Coder. They naturally extend to downstream tasks such as test-time scaling and agentic coding—achieving a 8.1% improvement over the base model. For the long-CoT model, our ReasonFlux-Coder-4B consistently outperforms Qwen3-4B while achieving 64.8% inference efficiency in unit test generation. Notably, we also find that our model can serve as an effective reward model for reinforcement learning on base models.
中文速览
让AI既能写出高质量代码、又能自动生成用于验证代码正确性的单元测试（unit test），一直是个难题，而现有方法都依赖昂贵的人工标注"标准答案代码"。CURE框架提出了一种强化学习方案，让同一个模型同时扮演"代码生成器"和"单元测试生成器"两个角色，通过让生成的代码和生成的测试相互执行、构建奖励矩阵来互相监督、协同进化，完全不需要任何标准答案代码作为监督信号。基于此训练的ReasonFlux-Coder系列模型（4B/7B/14B）在代码生成准确率上最高提升5.3%、Best-of-N选优准确率提升9.0%，超越同等规模的Qwen-Coder、DeepSeek-Coder和Seed-Coder，还能直接应用于推理时扩展（test-time scaling）和智能体编程（agentic coding）等下游场景。这项工作的意义在于：它打破了单元测试训练对昂贵人工标注数据的依赖，为低成本、大规模地提升LLM编程能力提供了一条可行路径。
原文 arXiv:2506.03136;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2506.03136v2