Multi-task Language Understanding on CMMLU
89.44AccuracyQwen3 Model
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Qwen3 ModelParameters=235B-25072025.09 | 89.44 | — | |
| BF16Backbone=DeepSeek-V3.1 671B2026.02 | 89.28 | — | |
| HiF4Backbone=DeepSeek-V3.1 671B2026.02 | 88.71 | -0.57 | |
| NVFP4Backbone=DeepSeek-V3.1 671B2026.02 | 87.76 | -1.52 | |
| NVFP4+PTSBackbone=DeepSeek-V3.1 671B2026.02 | 87.44 | -1.84 | |
| Qwen3 ModelParameters=8B2025.09 | 84.87 | — | |
| YiSize=34B2024.03 | 83.7 | — | |
| HiF4Backbone=LongCat 560B2026.02 | 82.74 | 1.09 | |
| BF16Backbone=LongCat 560B2026.02 | 81.65 | — | |
| MiniCPM-o 4.52026.04 | 79.6 | — | |
| Qwen3-8B-Instruct2026.04 | 78.7 | — | |
| Qwen3 ModelParameters=4B-25072025.09 | 76.76 | — | |
| YiSize=6B2024.03 | 75.5 | — | |
| Qwen3 ModelParameters=32B2025.09 | 73.64 | — | |
| AAPAParameters=4B2025.09 | 72.45 | — | |
| NVFP4+PTSBackbone=LongCat 560B2026.02 | 72.24 | -9.41 | |
| NVFP4Backbone=LongCat 560B2026.02 | 72.12 | -9.53 | |
| Qwen3 ModelParameters=4B2025.09 | 72.09 | — | |
| GPT-42024.03 | 71 | — | |
| QwenSize=14B2024.03 | 71 | — | |
| Baichuan-2Size=13B2024.03 | 61.97 | — | |
| SkyworkSize=13B2024.03 | 61.8 | — | |
| Qwen3 ModelParameters=1.7B2025.09 | 59.96 | — | |
| InternLMSize=20B2024.03 | 59 | — | |
| FalconSize=180B2024.03 | 58 | — | |
| GPT-3.52024.03 | 55.5 | — | |
| Llama2Size=70B2024.03 | 53.3 | — | |
| AAPAParameters=0.6B2025.09 | 46.6 | — | |
| FRAMEModel Size=3B, Training Tokens=1T2025.02 | 45.7 | — | |
| Qwen3 ModelParameters=0.6B2025.09 | 45.36 | — | |
| PDPCModel Size=3B, Training Tokens=1T2025.02 | 35.6 | — | |
| RandomModel Size=3B, Training Tokens=1T2025.02 | 27.5 | — | |
| Q3 -> Q1 -> Q4 -> Q2Model Size=3B, Training Tokens=1T, Curriculum Order=Q3 -> Q1 -> Q4 -> Q22025.02 | 26.4 | — |