Code Generation on MBPP hand-verified
60.7Pass@1Mixtral 8x7B
Evaluation Results
| Method | Links | |
|---|---|---|
| Mixtral 8x7Bhand-verified subset=true2024.01 | 60.7 | |
| GPT-3.5hand-verified subset=true2024.01 | 52.2 | |
| LLaMA 2 70Bhand-verified subset=true2024.01 | 49.8 |
Loading the SOTA2 catalog…