Code Repair on DeepFix
82.1Pass@1PaLM-Coder
Evaluation results
| Method | Links | |
|---|---|---|
| PaLM-CoderTraining stage=Code Finetuning, Model scale=540B, Shots=2-shot2022.04 | 82.1 | |
| Davinci CodexTraining stage=Code Finetuning, Shots=2-shot, Note=Calculation via OpenAI Codex API2022.04 | 81.1 | |
| PaLMTraining stage=Pretraining only, Model scale=540B, Shots=2-shot2022.04 | 73.7 | |
| Yasunaga & Liang (2021)Shots=2-shot2022.04 | 71.7 | |
| LaMDATraining stage=Pretraining only, Model scale=137B, Shots=2-shot2022.04 | 4.3 |