Multi-task Language Understanding on MMLU (MMLU Score)
86.4MMLU ScoreGPT-4
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-42026.01 | 86.4 | |
| GRPOBackbone=LLaMA-3.1-8B2025.05 | 75 | |
| TI-DPOBackbone=LLaMA-3.1-8B2025.05 | 74 | |
| TPOBackbone=LLaMA-3.1-8B2025.05 | 73 | |
| CPOBackbone=LLaMA-3.1-8B2025.05 | 72 | |
| Dream-Instruct 7B + G-StarRefinement Strategy=Loop-based refinement2025.10 | 71.2 | |
| KTOBackbone=LLaMA-3.1-8B2025.05 | 71 | |
| GPT-3.52026.01 | 70 | |
| DPOBackbone=LLaMA-3.1-8B2025.05 | 70 | |
| Dream-Instruct 7BSource=Reproduced2025.10 | 69.9 | |
| TDPOBackbone=LLaMA-3.1-8B2025.05 | 69.5 | |
| SFTBackbone=LLaMA-3.1-8B2025.05 | 69 | |
| SIMPOBackbone=LLaMA-3.1-8B2025.05 | 68.5 | |
| IPOBackbone=LLaMA-3.1-8B2025.05 | 68 | |
| Dream-Instruct 7BSource=Originally Published2025.10 | 67 | |
| LLama-3-8BModel Size=8B, Tokens=15T2026.02 | 66.2 | |
| baselineBackbone=LLaMA-2 70B, #Bits=FP16, #Eff. (w)=162026.03 | 65.44 | |
| DenseAvg. bits=16, 5-shot=true2025.05 | 65.29 | |
| baselineBackbone=Qwen-2.5-3B, #Bits=FP16, #Eff. (w)=162026.03 | 65.12 | |
| SERQ (GPTQ)Backbone=LLaMA-2 70B, #Bits=W4A8, #Eff. (w)=3.432026.03 | 64.62 | |
| L2QERBackbone=LLaMA-2 70B, #Bits=W4A8, #Eff. (w)=3.552026.03 | 64.45 | |
| SERQ (RTN)Backbone=LLaMA-2 70B, #Bits=W4A8, #Eff. (w)=3.442026.03 | 64.45 | |
| DCLM-7B-8kModel Size=7B, Tokens=2.8T2026.02 | 63.7 | |
| SERQ (GPTQ)Backbone=Qwen-2.5-3B, #Bits=W4A8, #Eff. (w)=4.242026.03 | 63.25 | |
| SERQ (RTN)Backbone=LLaMA-2 70B, #Bits=W4A4, #Eff. (w)=3.652026.03 | 63.15 | |
| SERQ (GPTQ)Backbone=LLaMA-2 70B, #Bits=W4A4, #Eff. (w)=3.642026.03 | 63.09 | |
| DCLM-Bas. + CC-TablesModel Size=7B, Tokens=2.7T, alpha=0.82026.02 | 63 | |
| SERQ (RTN)Backbone=Qwen-2.5-3B, #Bits=W4A8, #Eff. (w)=4.242026.03 | 62.79 | |
| DCLM-Bas. + CC-TablesModel Size=7B, Tokens=2.7T, alpha=0.852026.02 | 62.7 | |
| DCLM-Bas. + CC-TablesModel Size=7B, Tokens=2.7T, alpha=0.92026.02 | 62.6 | |
| DCLM-Bas. + CC-TablesModel Size=7B, Tokens=2.7T, alpha=12026.02 | 62.3 | |
| baselineBackbone=LLaMA-3 8B, #Bits=FP16, #Eff. (w)=162026.03 | 62.13 | |
| SERQ-MXFP4Backbone=LLaMA-2 70B, #Bits=W4A4, #Eff. (w)=3.792026.03 | 61.64 | |
| L2QER-MXFP4Backbone=LLaMA-2 70B, #Bits=W4A4, #Eff. (w)=3.812026.03 | 61.48 | |
| Qwen2.5-DenseLLM=Qwen2.5, Model=Dense, Activated Parameters=1.5 B2026.05 | 61.1 | |
| SERQ (GPTQ)Backbone=LLaMA-3 8B, #Bits=W4A8, #Eff. (w)=6.522026.03 | 60.25 | |
| SERQ (GPTQ)Backbone=Qwen-2.5-3B, #Bits=W4A4, #Eff. (w)=4.242026.03 | 59.55 | |
| SERQ (RTN)Backbone=Qwen-2.5-3B, #Bits=W4A4, #Eff. (w)=4.242026.03 | 59.52 | |
| SERQ (RTN)Backbone=LLaMA-3 8B, #Bits=W4A8, #Eff. (w)=6.712026.03 | 58.49 | |
| L2QERBackbone=LLaMA-3 8B, #Bits=W4A8, #Eff. (w)=7.162026.03 | 57.81 | |
| L2QERBackbone=LLaMA-2 70B, #Bits=W4A4, #Eff. (w)=4.552026.03 | 56.44 | |
| DBF + PVAvg. bits=2.3, 5-shot=true2025.05 | 56.37 | |
| AQLM + PVAvg. bits=2.3, 5-shot=true2025.05 | 56.2 | |
| QTIPAvg. bits=2, 5-shot=true2025.05 | 56.2 | |
| SERQ-MXFP4Backbone=Qwen-2.5-3B, #Bits=W4A4, #Eff. (w)=4.372026.03 | 56.13 | |
| L2QER-MXFP4Backbone=Qwen-2.5-3B, #Bits=W4A4, #Eff. (w)=4.372026.03 | 55.87 | |
| baselineBackbone=LLaMA-3.2 3B, #Bits=FP16, #Eff. (w)=162026.03 | 54.06 | |
| L2QER-MXFP4Backbone=LLaMA-3 8B, #Bits=W4A4, #Eff. (w)=7.832026.03 | 53.82 | |
| SERQ (GPTQ)Backbone=LLaMA-3 8B, #Bits=W4A4, #Eff. (w)=7.752026.03 | 53.8 | |
| SERQ-MXFP4Backbone=LLaMA-3 8B, #Bits=W4A4, #Eff. (w)=7.632026.03 | 53.48 | |
| resiliparseTraining Scale=7B-2x (276B Tokens), fastText Thresholds=0.112026.02 | 52.9 | |
| DBF + PVAvg. bits=2, 5-shot=true2025.05 | 52.67 | |
| Union (Random)Training Scale=7B-2x (276B Tokens), fastText Thresholds=(0.11, 0.11, 0.11)2026.02 | 52.3 | |
| Vicuna-13B (all)#Token=370M2026.01 | 52.1 | |
| SERQ (RTN)Backbone=LLaMA-3 8B, #Bits=W4A4, #Eff. (w)=8.072026.03 | 51.84 | |
| resiliparseTraining Scale=7B-2x (276B Tokens), fastText Thresholds=0.152026.02 | 51.7 | |
| Qwen2.5-Dense2MoELLM=Qwen2.5, Model=Ours, Activated Parameters=1.2 B2026.05 | 51.67 | |
| SERQ (RTN)Backbone=LLaMA-3.2 3B, #Bits=W4A8, #Eff. (w)=4.242026.03 | 51.45 | |
| Union (Manual)Training Scale=7B-2x (276B Tokens), fastText Thresholds=(0.11, 0.11, 0.11)2026.02 | 51.4 | |
| L2QERBackbone=LLaMA-3.2 3B, #Bits=W4A8, #Eff. (w)=4.352026.03 | 51.24 | |
| Union (Random)Training Scale=7B-2x (276B Tokens), fastText Thresholds=(0.11, 0.15, 0.15)2026.02 | 51 | |
| Union (Manual)Training Scale=7B-2x (276B Tokens), fastText Thresholds=(0.11, 0.15, 0.15)2026.02 | 50.8 | |
| Llama2-Dense2MoELLM=Llama2, Model=Ours, Activated Parameters=5.7 B2026.05 | 50.8 | |
| SERQ (GPTQ)Backbone=LLaMA-3.2 3B, #Bits=W4A8, #Eff. (w)=4.242026.03 | 50.24 | |
| Alpaca-13B#Token=4.4M2026.01 | 48.1 | |
| resiliparseTraining Scale=7B-2x (276B Tokens), fastText Thresholds=0.182026.02 | 47.4 | |
| LLaMA-13B#Token=1T2026.01 | 47 | |
| L2QER-MXFP4Backbone=LLaMA-3.2 3B, #Bits=W4A4, #Eff. (w)=4.372026.03 | 46.85 | |
| LLM.int4()Backbone=LLaMA-3 8B, #Bits=W4A8, #Eff. (w)=-†2026.03 | 46.72 | |
| SERQ (RTN)Backbone=LLaMA-3.2 3B, #Bits=W4A4, #Eff. (w)=4.242026.03 | 46.36 | |
| Llama2-DenseLLM=Llama2, Model=Dense, Activated Parameters=7.0 B2026.05 | 45.8 | |
| SERQ (GPTQ)Backbone=LLaMA-3.2 3B, #Bits=W4A4, #Eff. (w)=4.242026.03 | 45.7 | |
| LLM.int4()Backbone=LLaMA-2 70B, #Bits=W4A8, #Eff. (w)=-†2026.03 | 43.56 | |
| Union (Manual)Training Scale=7B-1x (138B Tokens), fastText Thresholds=(0.11, 0.11, 0.11)2026.02 | 43.5 | |
| baselineBackbone=LLaMA-2 7B, #Bits=FP16, #Eff. (w)=162026.03 | 41.83 | |
| LLM.int4()Backbone=LLaMA-3.2 3B, #Bits=W4A8, #Eff. (w)=-†2026.03 | 41.48 | |
| SERQ-MXFP4Backbone=LLaMA-3.2 3B, #Bits=W4A4, #Eff. (w)=4.372026.03 | 41.33 | |
| SERQ (GPTQ)Backbone=LLaMA-2 7B, #Bits=W4A8, #Eff. (w)=4.242026.03 | 40.29 | |
| SERQ (RTN)Backbone=LLaMA-2 7B, #Bits=W4A8, #Eff. (w)=4.242026.03 | 40.21 | |
| resiliparseTraining Scale=7B-1x (138B Tokens), fastText Thresholds=0.112026.02 | 39.5 | |
| L2QERBackbone=LLaMA-2 7B, #Bits=W4A8, #Eff. (w)=4.352026.03 | 39.4 | |
| L2QERBackbone=LLaMA-3 8B, #Bits=W4A4, #Eff. (w)=11.442026.03 | 38.33 | |
| SERQ (RTN)Backbone=LLaMA-2 7B, #Bits=W4A4, #Eff. (w)=4.242026.03 | 38.03 | |
| trafilaturaTraining Scale=7B-1x (138B Tokens), fastText Thresholds=0.112026.02 | 37.7 | |
| SERQ (GPTQ)Backbone=LLaMA-2 7B, #Bits=W4A4, #Eff. (w)=4.242026.03 | 37.03 | |
| baselineBackbone=LLaMA-3.2 1B, #Bits=FP16, #Eff. (w)=162026.03 | 36.76 | |
| SERQ-MXFP4Backbone=LLaMA-2 7B, #Bits=W4A4, #Eff. (w)=4.372026.03 | 35.25 | |
| L2QER-MXFP4Backbone=LLaMA-2 7B, #Bits=W4A4, #Eff. (w)=4.372026.03 | 35.22 | |
| jusTextTraining Scale=7B-1x (138B Tokens), fastText Thresholds=0.112026.02 | 34.5 | |
| Union (Random)Training Scale=7B-1x (138B Tokens), fastText Thresholds=(0.11, 0.11, 0.11)2026.02 | 33.6 | |
| SERQ (RTN)Backbone=LLaMA-3.2 1B, #Bits=W4A8, #Eff. (w)=4.242026.03 | 32.77 | |
| L2QERBackbone=LLaMA-3.2 3B, #Bits=W4A4, #Eff. (w)=4.242026.03 | 32.25 | |
| L2QERBackbone=LLaMA-3.2 1B, #Bits=W4A8, #Eff. (w)=4.352026.03 | 31.23 | |
| L2QERBackbone=LLaMA-2 7B, #Bits=W4A4, #Eff. (w)=4.242026.03 | 29.63 | |
| SERQ (RTN)Backbone=LLaMA-3.2 1B, #Bits=W4A4, #Eff. (w)=4.242026.03 | 29.11 | |
| LLM.int4()Backbone=LLaMA-2 7B, #Bits=W4A8, #Eff. (w)=-†2026.03 | 29.01 | |
| SERQ-MXFP4Backbone=LLaMA-3.2 1B, #Bits=W4A4, #Eff. (w)=4.372026.03 | 27.23 | |
| SERQ (GPTQ)Backbone=LLaMA-3.2 1B, #Bits=W4A8, #Eff. (w)=4.242026.03 | 27.1 | |
| L2QER-MXFP4Backbone=LLaMA-3.2 1B, #Bits=W4A4, #Eff. (w)=4.372026.03 | 27.1 | |
| SERQ (GPTQ)Backbone=LLaMA-3.2 1B, #Bits=W4A4, #Eff. (w)=4.242026.03 | 26.34 |