Python Coding on MBPP-CN v1 (test)
53.2Pass@1InternLM2-20B
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| InternLM2-20Bshot count=0-shot, Parameter size group=13~20B Models2024.03 | 53.2 | — | |
| ChatGLM3-6B-Baseshot count=0-shot, Parameter size group=< 7B Models2024.03 | 50 | — | |
| InternLM2-7Bshot count=0-shot, Parameter size group=< 7B Models2024.03 | 45.4 | — | |
| InternLM2-20B-Baseshot count=0-shot, Parameter size group=13~20B Models2024.03 | 44.4 | — | |
| Mixtral-8x7B-v0.1shot count=0-shot, Parameter size group=13~20B Models2024.03 | 43.8 | — | |
| Qwen-14Bshot count=0-shot, Parameter size group=13~20B Models2024.03 | 41 | — | |
| InternLM2-7B-Baseshot count=0-shot, Parameter size group=< 7B Models2024.03 | 40.6 | — | |
| Mistral-7B-v0.1shot count=0-shot, Parameter size group=< 7B Models2024.03 | 35 | — | |
| Baichuan2-13B-Baseshot count=0-shot, Parameter size group=13~20B Models2024.03 | 30.6 | — | |
| Qwen-7B-Chatshot count=0-shot, Parameter size group=< 7B Models2024.03 | 27 | — | |
| Llama2-13Bshot count=0-shot, Parameter size group=13~20B Models2024.03 | 25.2 | — | |
| Baichuan2-7B-Baseshot count=0-shot, Parameter size group=< 7B Models2024.03 | 23.2 | — | |
| Llama2-7Bshot count=0-shot, Parameter size group=< 7B Models2024.03 | 16 | — | |
| Baichuan2-13B-ChatShots=0-shot, Model Size Group=13~20B Models2024.03 | — | 27.8 | |
| Baichuan2-7B-ChatShots=0-shot, Model Size Group=< 7B Models2024.03 | — | 20.8 | |
| ChatGLM3-6BShots=0-shot, Model Size Group=< 7B Models2024.03 | — | 38.2 | |
| InternLM2-Chat-20BShots=0-shot, Model Size Group=13~20B Models2024.03 | — | 54 | |
| InternLM2-Chat-20B-SFTShots=0-shot, Model Size Group=13~20B Models2024.03 | — | 52.2 | |
| InternLM2-Chat-7BShots=0-shot, Model Size Group=< 7B Models2024.03 | — | 46.4 | |
| InternLM2-Chat-7B-SFTShots=0-shot, Model Size Group=< 7B Models2024.03 | — | 44.4 | |
| Llama2-13B-ChatShots=0-shot, Model Size Group=13~20B Models2024.03 | — | 22.2 | |
| Llama2-7B-ChatShots=0-shot, Model Size Group=< 7B Models2024.03 | — | 17.8 | |
| Mistral-7B-Instruct-v0.2Shots=0-shot, Model Size Group=< 7B Models2024.03 | — | 17.6 | |
| Mixtral-8x7B-Instruct-v0.1Shots=0-shot, Model Size Group=13~20B Models2024.03 | — | 43.8 | |
| Qwen-14B-ChatShots=0-shot, Model Size Group=13~20B Models2024.03 | — | 27.6 | |
| Qwen-7B-ChatShots=0-shot, Model Size Group=< 7B Models2024.03 | — | 27 |