Code Generation on MBPP EvalPlus base
90.5Pass@1Claude 3.5 Sonnet
Evaluation results
| Method | Links | |
|---|---|---|
| Claude 3.5 Sonnet2024.07 | 90.5 | |
| Llama 3 405BParameters=405B2024.07 | 88.6 | |
| GPT-4o2024.07 | 87.8 | |
| Llama 3 70BParameters=70B2024.07 | 86 | |
| GPT-42024.07 | 83.6 | |
| GPT-3.5 Turbo2024.07 | 82 | |
| Mixtral 8x22BParameters=8x22B2024.07 | 78.6 | |
| Llama 3 8BParameters=8B2024.07 | 72.8 | |
| Nemotron 4 340BParameters=340B2024.07 | 72.8 | |
| Gemma 2 9BParameters=9B2024.07 | 71.7 | |
| Mistral 7BParameters=7B2024.07 | 49.5 |