Code Generation on MultiPL-E (Accuracy)
59.6AccuracyQwen2-72B
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen2-72B2024.07 | 59.6 | |
| Qwen1.5-110B2024.07 | 52.7 | |
| Mixtral-8x22B2024.07 | 46.7 | |
| Llama-3-70B2024.07 | 46.3 | |
| Qwen1.5-72B2024.07 | 41.8 |
Loading the SOTA2 catalog…
| Method | Links | |
|---|---|---|
| Qwen2-72B2024.07 | 59.6 | |
| Qwen1.5-110B2024.07 | 52.7 | |
| Mixtral-8x22B2024.07 | 46.7 | |
| Llama-3-70B2024.07 | 46.3 | |
| Qwen1.5-72B2024.07 | 41.8 |