Code Translation on TransCoder
55.1Pass@1PaLM-Coder
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PaLM-CoderTraining stage=Code Finetuning, Model scale=540B, Shots=3-shot2022.04 | 55.1 | 82.5 | |
| Davinci CodexTraining stage=Code Finetuning, Shots=3-shot, Note=Calculation via OpenAI Codex API2022.04 | 54.4 | 71.7 | |
| PaLMTraining stage=Pretraining only, Model scale=540B, Shots=3-shot2022.04 | 51.8 | 79.8 | |
| Lachaux et al. (2020)Shots=3-shot2022.04 | 44.5 | 67.2 | |
| LaMDATraining stage=Pretraining only, Model scale=137B, Shots=3-shot2022.04 | 30.2 | — |