Python Coding on HumanEval-X v1 (test)
48.2Pass@1InternLM2-20B
Evaluation results
| Method | Links | |
|---|---|---|
| InternLM2-20Bshot count=5-shot, Parameter size group=13~20B Models2024.03 | 48.2 | |
| InternLM2-7Bshot count=5-shot, Parameter size group=< 7B Models2024.03 | 39.6 | |
| ChatGLM3-6B-Baseshot count=5-shot, Parameter size group=< 7B Models2024.03 | 38.3 | |
| Mixtral-8x7B-v0.1shot count=5-shot, Parameter size group=13~20B Models2024.03 | 38.3 | |
| InternLM2-20B-Baseshot count=5-shot, Parameter size group=13~20B Models2024.03 | 31.5 | |
| Qwen-14Bshot count=5-shot, Parameter size group=13~20B Models2024.03 | 31 | |
| InternLM2-7B-Baseshot count=5-shot, Parameter size group=< 7B Models2024.03 | 28.8 | |
| Mistral-7B-v0.1shot count=5-shot, Parameter size group=< 7B Models2024.03 | 28.5 | |
| Qwen-7B-Chatshot count=5-shot, Parameter size group=< 7B Models2024.03 | 24.4 | |
| Baichuan2-13B-Baseshot count=5-shot, Parameter size group=13~20B Models2024.03 | 19.5 | |
| Llama2-13Bshot count=5-shot, Parameter size group=13~20B Models2024.03 | 17.2 | |
| Baichuan2-7B-Baseshot count=5-shot, Parameter size group=< 7B Models2024.03 | 16.1 | |
| Llama2-7Bshot count=5-shot, Parameter size group=< 7B Models2024.03 | 11.2 |