Coding on Codex-Eval
94.1Pass@10GPT-4
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-4Shots=0-shot2023.06 | 94.1 | |
| ChatGPTShots=0-shot2023.06 | 88.4 | |
| ShareGPT 65BShots=0-shot2023.06 | 56.2 | |
| TÜLU 65BShots=0-shot2023.06 | 49.4 | |
| TÜLU 30BShots=0-shot2023.06 | 48 | |
| LLaMa 65BShots=0-shot2023.06 | 46.9 | |
| Human mix. 65BShots=0-shot2023.06 | 44.6 | |
| LLaMa 30BShots=0-shot2023.06 | 42.8 | |
| TÜLU-1.1 13BShots=0-shot2023.06 | 38.9 | |
| TÜLU 13BShots=0-shot2023.06 | 35.9 | |
| TÜLU-1.1 7BShots=0-shot2023.06 | 33.9 | |
| LLaMa-2 13BShots=0-shot2023.06 | 32.5 | |
| TÜLU 7BShots=0-shot2023.06 | 29.1 | |
| LLaMa 13BShots=0-shot2023.06 | 28.6 | |
| LLaMa-2 7BShots=0-shot2023.06 | 26.8 | |
| LLaMa 7BShots=0-shot2023.06 | 20.5 |