Program Synthesis on openai_humaneval (test)
15.52Pass@1BLOOM-176B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BLOOM-176BPrompt=EN2022.11 | 15.52 | 32.2 | 55.45 | |
| BLOOMZ-MTPrompt=EN2022.11 | 13.55 | 26.26 | 47.01 | |
| BLOOMZPrompt=EN2022.11 | 12.06 | 26.53 | 48.44 | |
| mT0-13BPrompt=EN2022.11 | 0 | 0 | 0 |