Chinese Language Knowledge and Reasoning on CMMLU
77.01ScoreQwen3-4B
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen3-4BParams=4B, Shot setting=5 shot, Evaluation protocol=PPL2025.12 | 77.01 | |
| Qwen2.5-3BParams=3B, Shot setting=5 shot, Evaluation protocol=PPL2025.12 | 73.92 | |
| Qwen2-1.5BParams=1.5B, Shot setting=5 shot, Evaluation protocol=PPL2025.12 | 70.62 | |
| Qwen2.5-1.5BParams=1.5B, Shot setting=5 shot, Evaluation protocol=PPL2025.12 | 68.01 | |
| Qwen3-1.7BParams=1.7B, Shot setting=5 shot, Evaluation protocol=PPL2025.12 | 66.55 | |
| Qwen3-0.6BParams=0.6B, Shot setting=5 shot, Evaluation protocol=PPL2025.12 | 52.36 | |
| SmolLM3-3BParams=3B, Shot setting=5 shot, Evaluation protocol=PPL2025.12 | 49.35 | |
| PCMind-2.1-Kaiyuan-2BParams=2B, Shot setting=5 shot, Evaluation protocol=PPL2025.12 | 49.25 | |
| YuLan-Mini-2.4BParams=2.4B, Shot setting=5 shot, Evaluation protocol=PPL2025.12 | 48.14 | |
| Llama-3.2-3BParams=3B, Shot setting=5 shot, Evaluation protocol=PPL2025.12 | 44.33 | |
| Gemma2-2BParams=2B, Shot setting=5 shot, Evaluation protocol=PPL2025.12 | 39.63 | |
| SmolLM2-1.7BParams=1.7B, Shot setting=5 shot, Evaluation protocol=PPL2025.12 | 34.03 | |
| Llama-3.2-1BParams=1B, Shot setting=5 shot, Evaluation protocol=PPL2025.12 | 31.03 | |
| OLMo-2-0425-1BParams=1B, Shot setting=5 shot, Evaluation protocol=PPL2025.12 | 28.62 |