Unit Test Case Generation with Transformers and Focal Context
Michele Tufano, Dawn Drain, Alexey Svyatkovskiy, Shao Kun Deng, Neel Sundaresan M. Tufano, D. Drain, A. Svyatkovskiy, S. K. Deng, N. Sundaresan are with Microsoft, Redmond, WA, USA. E-mail: {mitufano, dadrain, alsvyatk, shade,
Abstract
Software testing is a critical part of software development life cycle which helps identify potential regressions and reduce maintenance costs, yet it is often neglected by developers. Automated unit test case generation tools facilitate test-driven development and support developers by suggesting tests intended to identify flaws in their code. Existing approaches are usually guided by the test coverage criteria, generating synthetic test cases that are often difficult for developers to read or understand. In this paper we propose AthenaTest, an approach that aims to generate unit test cases by learning from real-world focal methods and developer-written test cases. We formulate unit test case generation as a sequence-to-sequence learning task, adopting a two-step training procedure consisting of denoising pretraining on a large unsupervised Java corpus, and supervised finetuning for a downstream translation task of generating unit tests. We investigate the impact of natural language and source code pretraining, as well as the focal context information surrounding the focal method. We found that both techniques provide improvements in terms of validation loss, with pretraining yiel
中文速览
开发者普遍不爱写单元测试，现有自动化工具（如EvoSuite）虽能生成测试用例，却靠覆盖率指标驱动，产出的代码可读性差、像机器写的。AthenaTest转换思路，把单元测试生成看作一个"翻译"任务——给定一个待测方法，让模型学会像真实开发者那样写出对应的测试用例：先在1.6亿词的英文语料和大规模Java代码上做去噪预训练，再在研究团队构建的78万对"测试方法↔被测方法"平行语料库Methods2Test上做监督微调，并将被测方法周边的类信息（focal context）一并输入模型。在Defects4J五个真实项目上，AthenaTest一次最多尝试30次便能为43.7%的被测方法生成可通过的测试用例，代码覆盖率与EvoSuite相当，而在可读性、可理解性和测试有效性三项指标上，专业开发者的偏好压倒性地倒向AthenaTest，表明这一方向有望切实提升开发者的测试效率。
原文 arXiv:2009.05617;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2009.05617v2