# Chinese Language Knowledge and Reasoning on CMMLU Source: [Chinese Language Knowledge and Reasoning on CMMLU]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:03:54.140Z Evaluation of model performance on the Chinese Language Knowledge and Reasoning task within the CMMLU dataset, measured by Score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Chinese Language Knowledge and Reasoning | | Dataset (including any stated split) | CMMLU | | Metric | Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 14 of 14 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Score | Result date | Report paper | | --- | --- | --- | --- | | Qwen3-4B | 77.01 | Not recorded | 6937ad406cfd488761a4eb25 | | Qwen2.5-3B | 73.92 | Not recorded | 6937ad406cfd488761a4eb25 | | Qwen2-1.5B | 70.62 | Not recorded | 6937ad406cfd488761a4eb25 | | Qwen2.5-1.5B | 68.01 | Not recorded | 6937ad406cfd488761a4eb25 | | Qwen3-1.7B | 66.55 | Not recorded | 6937ad406cfd488761a4eb25 | | Qwen3-0.6B | 52.36 | Not recorded | 6937ad406cfd488761a4eb25 | | SmolLM3-3B | 49.35 | Not recorded | 6937ad406cfd488761a4eb25 | | PCMind-2.1-Kaiyuan-2B | 49.25 | Not recorded | 6937ad406cfd488761a4eb25 | | YuLan-Mini-2.4B | 48.14 | Not recorded | 6937ad406cfd488761a4eb25 | | Llama-3.2-3B | 44.33 | Not recorded | 6937ad406cfd488761a4eb25 | | Gemma2-2B | 39.63 | Not recorded | 6937ad406cfd488761a4eb25 | | SmolLM2-1.7B | 34.03 | Not recorded | 6937ad406cfd488761a4eb25 | | Llama-3.2-1B | 31.03 | Not recorded | 6937ad406cfd488761a4eb25 | | OLMo-2-0425-1B | 28.62 | Not recorded | 6937ad406cfd488761a4eb25 | ### Qwen3-4B Score: 77.01. Result ID: 6996c6bf5698665ec41250dc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params":"4B","Shot setting":"5 shot","Evaluation protocol":"PPL"}. This record does not assert independent reproduction. ### Qwen2.5-3B Score: 73.92. Result ID: 6996c6bf5698665ec41250d9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params":"3B","Shot setting":"5 shot","Evaluation protocol":"PPL"}. This record does not assert independent reproduction. ### Qwen2-1.5B Score: 70.62. Result ID: 6996c6bf5698665ec41250d7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Params":"1.5B","Shot setting":"5 shot","Evaluation protocol":"PPL"}. This record does not assert independent reproduction. ### Qwen2.5-1.5B Score: 68.01. Result ID: 6996c6bf5698665ec41250d8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params":"1.5B","Shot setting":"5 shot","Evaluation protocol":"PPL"}. This record does not assert independent reproduction. ### Qwen3-1.7B Score: 66.55. Result ID: 6996c6bf5698665ec41250db. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params":"1.7B","Shot setting":"5 shot","Evaluation protocol":"PPL"}. This record does not assert independent reproduction. ### Qwen3-0.6B Score: 52.36. Result ID: 6996c6bf5698665ec41250da. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params":"0.6B","Shot setting":"5 shot","Evaluation protocol":"PPL"}. This record does not assert independent reproduction. ### SmolLM3-3B Score: 49.35. Result ID: 6996c6bf5698665ec41250e3. [Reporting paper]() Recorded evaluation configuration: {"Params":"3B","Shot setting":"5 shot","Evaluation protocol":"PPL"}. This record does not assert independent reproduction. ### PCMind-2.1-Kaiyuan-2B Score: 49.25. Result ID: 6996c6bf5698665ec41250e4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params":"2B","Shot setting":"5 shot","Evaluation protocol":"PPL"}. This record does not assert independent reproduction. ### YuLan-Mini-2.4B Score: 48.14. Result ID: 6996c6bf5698665ec41250e2. [Reporting paper]() Recorded evaluation configuration: {"Params":"2.4B","Shot setting":"5 shot","Evaluation protocol":"PPL"}. This record does not assert independent reproduction. ### Llama-3.2-3B Score: 44.33. Result ID: 6996c6bf5698665ec41250df. [Reporting paper]() Recorded evaluation configuration: {"Params":"3B","Shot setting":"5 shot","Evaluation protocol":"PPL"}. This record does not assert independent reproduction. ### Gemma2-2B Score: 39.63. Result ID: 6996c6bf5698665ec41250dd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params":"2B","Shot setting":"5 shot","Evaluation protocol":"PPL"}. This record does not assert independent reproduction. ### SmolLM2-1.7B Score: 34.03. Result ID: 6996c6bf5698665ec41250e0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params":"1.7B","Shot setting":"5 shot","Evaluation protocol":"PPL"}. This record does not assert independent reproduction. ### Llama-3.2-1B Score: 31.03. Result ID: 6996c6bf5698665ec41250de. [Reporting paper]() Recorded evaluation configuration: {"Params":"1B","Shot setting":"5 shot","Evaluation protocol":"PPL"}. This record does not assert independent reproduction. ### OLMo-2-0425-1B Score: 28.62. Result ID: 6996c6bf5698665ec41250e1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Params":"1B","Shot setting":"5 shot","Evaluation protocol":"PPL"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.