Comprehensive Examination on AGIEval (test)
62.3AccuracyGPT 4o
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT 4oModel Type=Proprietary2024.10 | 62.3 | — | — | — | — | — | — | — | — | — | — | |
| InternLM2-20BShots=0-shot, Model Size Group=13 ~ 20B Models2024.03 | 53 | — | — | — | — | — | — | — | — | — | — | |
| Qwen-14BShots=0-shot, Model Size Group=13 ~ 20B Models2024.03 | 52 | — | — | — | — | — | — | — | — | — | — | |
| InternLM2-Chat-20B-SFTParameter size group=13~20B Models, Evaluation protocol=0-shot, Alignment=SFT2024.03 | 51.2 | — | — | — | — | — | — | — | — | — | — | |
| InternLM2-Chat-20BParameter size group=13~20B Models, Evaluation protocol=0-shot2024.03 | 50.3 | — | — | — | — | — | — | — | — | — | — | |
| InternLM2-7BShots=0-shot, Model Size Group=<= 7B Models2024.03 | 49.9 | — | — | — | — | — | — | — | — | — | — | |
| InternLM2-Chat-7B-SFTParameter size group=< 7B Models, Evaluation protocol=0-shot, Alignment=SFT2024.03 | 49 | — | — | — | — | — | — | — | — | — | — | |
| Baichuan-omniParameters=7B, Modality=Omni-modal2024.10 | 47.7 | — | — | — | — | — | — | — | — | — | — | |
| ChatGLM3-6B-BaseShots=0-shot, Model Size Group=<= 7B Models2024.03 | 47.5 | — | — | — | — | — | — | — | — | — | — | |
| InternLM2-Chat-7BParameter size group=< 7B Models, Evaluation protocol=0-shot2024.03 | 47.2 | — | — | — | — | — | — | — | — | — | — | |
| Qwen-14B-ChatParameter size group=13~20B Models, Evaluation protocol=0-shot2024.03 | 46.5 | — | — | — | — | — | — | — | — | — | — | |
| VITAParameters=8x7B, Modality=Omni-modal2024.10 | 46.2 | — | — | — | — | — | — | — | — | — | — | |
| Qwen-7BShots=0-shot, Model Size Group=<= 7B Models2024.03 | 45.6 | — | — | — | — | — | — | — | — | — | — | |
| ChatGLM3-6BParameter size group=< 7B Models, Evaluation protocol=0-shot2024.03 | 44.2 | — | — | — | — | — | — | — | — | — | — | |
| Mixtral-8x7B-Instruct-v0.1Parameter size group=13~20B Models, Evaluation protocol=0-shot, Alignment=Instruct2024.03 | 41.7 | — | — | — | — | — | — | — | — | — | — | |
| Mixtral-8x7B-v0.1Shots=0-shot, Model Size Group=13 ~ 20B Models2024.03 | 40.9 | — | — | — | — | — | — | — | — | — | — | |
| Baichuan2-13B-ChatParameter size group=13~20B Models, Evaluation protocol=0-shot2024.03 | 40 | — | — | — | — | — | — | — | — | — | — | |
| GPT-3.5Model type=API Models, Evaluation protocol=0-shot2024.03 | 39.9 | — | — | — | — | — | — | — | — | — | — | |
| Qwen-7B-ChatParameter size group=< 7B Models, Evaluation protocol=0-shot2024.03 | 39.7 | — | — | — | — | — | — | — | — | — | — | |
| Qwen1.5-ChatParameters=7B, Modality=Pure text2024.10 | 39.3 | — | — | — | — | — | — | — | — | — | — | |
| InternLM2-7B-BaseShots=0-shot, Model Size Group=<= 7B Models2024.03 | 38.7 | — | — | — | — | — | — | — | — | — | — | |
| InternLM2-20B-BaseShots=0-shot, Model Size Group=13 ~ 20B Models2024.03 | 38.7 | — | — | — | — | — | — | — | — | — | — | |
| Llama3-InstructParameters=8B, Modality=Pure text2024.10 | 38.4 | — | — | — | — | — | — | — | — | — | — | |
| Baichuan2-13B-BaseShots=0-shot, Model Size Group=13 ~ 20B Models2024.03 | 37.4 | — | — | — | — | — | — | — | — | — | — | |
| Baichuan2-7B-ChatParameter size group=< 7B Models, Evaluation protocol=0-shot2024.03 | 35.3 | — | — | — | — | — | — | — | — | — | — | |
| Mistral-7B-Instruct-v0.2Parameter size group=< 7B Models, Evaluation protocol=0-shot, Alignment=Instruct2024.03 | 34.5 | — | — | — | — | — | — | — | — | — | — | |
| Baichuan2-7B-BaseShots=0-shot, Model Size Group=<= 7B Models2024.03 | 34.5 | — | — | — | — | — | — | — | — | — | — | |
| MAP-NeoParameters=7B, Modality=Pure text2024.10 | 33.9 | — | — | — | — | — | — | — | — | — | — | |
| Mistral-7B-v0.1Shots=0-shot, Model Size Group=<= 7B Models2024.03 | 32.9 | — | — | — | — | — | — | — | — | — | — | |
| Llama2-13B-ChatParameter size group=13~20B Models, Evaluation protocol=0-shot2024.03 | 30.9 | — | — | — | — | — | — | — | — | — | — | |
| Llama2-7B-ChatParameter size group=< 7B Models, Evaluation protocol=0-shot2024.03 | 28.5 | — | — | — | — | — | — | — | — | — | — | |
| Llama2-13BShots=0-shot, Model Size Group=13 ~ 20B Models2024.03 | 27.9 | — | — | — | — | — | — | — | — | — | — | |
| MoE w/ optimal ARModel Type=MoE, Activation rate=20.07, Compute=2.86e21, Data reuse=-2025.06 | 22.07 | — | — | — | — | — | — | — | — | — | — | |
| Llama2-7BShots=0-shot, Model Size Group=<= 7B Models2024.03 | 21.3 | — | — | — | — | — | — | — | — | — | — | |
| MoE w/ optimal ARModel Type=MoE, Activation rate=20.07, Compute=2.86e21, Data reuse=strict2025.06 | 21.02 | — | — | — | — | — | — | — | — | — | — | |
| Dense baselineModel Type=Dense, Compute=5.45e212025.06 | 20.89 | — | — | — | — | — | — | — | — | — | — | |
| OLMoParameters=7B, Modality=Pure text2024.10 | 19.9 | — | — | — | — | — | — | — | — | — | — | |
| RouteMoAMetric Category=Accuracy (%)2026.01 | — | 58.1 | 37.68 | 49.19 | 77.78 | 67.84 | 69.36 | 27.12 | 60.97 | 43.5 | 54.62 | |
| RouteMoAMetric Category=Cost ($)2026.01 | — | 4.18 | 7.77 | 6.33 | 3.46 | 4.15 | 3.33 | 6.08 | 7.57 | 7.91 | 5.64 | |
| RouteMoAMetric Category=Latency (s)2026.01 | — | 6.12 | 14.46 | 4.57 | 2.71 | 5.59 | 4.26 | 17.72 | 20.08 | 15.93 | 10.16 | |
| SMoAMetric Category=Accuracy (%)2026.01 | — | 53.33 | 37.68 | 49.59 | 80.39 | 60.8 | 64.26 | 27.12 | 64.1 | 39 | 52.92 | |
| SMoAMetric Category=Cost ($)2026.01 | — | 4.71 | 6.9 | 8.05 | 5.25 | 3.8 | 3.49 | 8.79 | 9.04 | 7.31 | 6.37 | |
| SMoAMetric Category=Latency (s)2026.01 | — | 11.25 | 15.41 | 10.9 | 9.4 | 9.79 | 9.86 | 19.09 | 19.85 | 15.91 | 13.5 |