Abstract

Generative models of code, pretrained on large corpora of programs, have shown great success in translating natural language to code (Chen et al. 2021; Austin et al. 2021; Li et al. 2022, inter alia). While these models do not explicitly incorporate program semantics (i.e., execution results) during training, they are able to generate correct solutions for many problems. However, choosing a single correct program from a generated set for each problem remains challenging.