Language Understanding on MMLU-Redux
85.75AccuracyInstruct Model (Q1)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Instruct Model (Q1)Model Scale=14B2026.01 | 85.75 | 0.1177 | 0.1071 | -0.7399 | |
| Ours (Label-Free Framework)Model Scale=14B2026.01 | 85.75 | 0.1101 | 0.0612 | -0.7474 | |
| Qwen3-32BTarget LLM=Public Models2026.05 | 85.7 | — | — | — | |
| Qwen3-14BPrecision=FP4 w. Attn-QAT2026.02 | 83.92 | — | — | — | |
| Qwen3-14BPrecision=BF162026.02 | 83.16 | — | — | — | |
| Base Model (Q0)Model Scale=14B2026.01 | 81.12 | 0.1084 | 0.0152 | -0.7028 | |
| DPA-Qwen3-32BTarget LLM=Qwen3-32B2026.05 | 80.6 | — | — | — | |
| Instruct Model (Q1)Model Scale=4B2026.01 | 80.26 | 0.1653 | 0.161 | -0.6373 | |
| Ours (Label-Free Framework)Model Scale=4B2026.01 | 80.26 | 0.1498 | 0.0965 | -0.6528 | |
| LLaVA-NeXT-Qwen3-32BTarget LLM=Qwen3-32B2026.05 | 79.5 | — | — | — | |
| Llama3.1-70BPrecision=BF162026.02 | 79.28 | — | — | — | |
| Llama3.1-70BPrecision=FP4 w. Attn-QAT2026.02 | 78.23 | — | — | — | |
| Qwen3-4BTarget LLM=Public Models2026.05 | 77.3 | — | — | — | |
| iLLaDA 8BModel Architecture=Diffusion, Instruction Tuning=true2026.06 | 76.4 | — | — | — | |
| Dream 7BModel Architecture=Diffusion, Instruction Tuning=true2026.06 | 76.3 | — | — | — | |
| Qwen2.5 7BModel Architecture=AR, Instruction Tuning=true2026.06 | 75.7 | — | — | — | |
| Base Model (Q0)Model Scale=4B2026.01 | 73.54 | 0.1379 | 0.0279 | -0.5976 | |
| Ours (Label-Free Framework)Model Scale=1.7B2026.01 | 70.44 | 0.232 | 0.2097 | -0.4724 | |
| Instruct Model (Q1)Model Scale=1.7B2026.01 | 70.26 | 0.2475 | 0.2461 | -0.4552 | |
| DPA-Qwen3-4BTarget LLM=Qwen3-4B2026.05 | 70.2 | — | — | — | |
| LLaDA 8BModel Architecture=Diffusion, Instruction Tuning=true2026.06 | 68.9 | — | — | — | |
| LLaVA-NeXT-Qwen3-4BTarget LLM=Qwen3-4B2026.05 | 66 | — | — | — | |
| Base Model (Q0)Model Scale=1.7B2026.01 | 64.46 | 0.1714 | 0.019 | -0.4732 | |
| Instruct Model (Q1)Model Scale=0.6B2026.01 | 54.58 | 0.3762 | 0.3742 | -0.1696 | |
| Ours (Label-Free Framework)Model Scale=0.6B2026.01 | 54.58 | 0.3571 | 0.3485 | -0.1887 | |
| Base Model (Q0)Model Scale=0.6B2026.01 | 53.16 | 0.2005 | 0.0277 | -0.331 | |
| Idefics2-8BTarget LLM=Public Models2026.05 | 50 | — | — | — | |
| Qwen2-VL-2BTarget LLM=Public Models2026.05 | 45.3 | — | — | — | |
| LLaVA-NeXT-7BTarget LLM=Public Models2026.05 | 43.6 | — | — | — | |
| DPA-LLaMA-3.2-3BTarget LLM=LLaMA-3.2-3B2026.05 | 43.6 | — | — | — | |
| LLaVA-1.5-7BTarget LLM=Public Models2026.05 | 42.6 | — | — | — | |
| LLaVA-NeXT-LLaMA-3.2-3BTarget LLM=LLaMA-3.2-3B2026.05 | 41 | — | — | — | |
| Cambrian-1-8BTarget LLM=Public Models2026.05 | 8.1 | — | — | — | |
| Histogram BinningModel Scale=0.6B, Category=Supervised Baseline2026.01 | — | 0.2365 | 0.0184 | -0.3093 | |
| Histogram BinningModel Scale=1.7B, Category=Supervised Baseline2026.01 | — | 0.1879 | 0.0389 | -0.5147 | |
| Histogram BinningModel Scale=4B, Category=Supervised Baseline2026.01 | — | 0.1403 | 0.0287 | -0.6623 | |
| Histogram BinningModel Scale=14B, Category=Supervised Baseline2026.01 | — | 0.1086 | 0.0303 | -0.749 | |
| Isotonic RegressionModel Scale=0.6B, Category=Supervised Baseline2026.01 | — | 0.2294 | 0.0178 | -0.3164 | |
| Isotonic RegressionModel Scale=1.7B, Category=Supervised Baseline2026.01 | — | 0.1696 | 0.0331 | -0.533 | |
| Isotonic RegressionModel Scale=4B, Category=Supervised Baseline2026.01 | — | 0.1239 | 0.0117 | -0.6788 | |
| Isotonic RegressionModel Scale=14B, Category=Supervised Baseline2026.01 | — | 0.0986 | 0.0411 | -0.7589 | |
| Temperature ScalingModel Scale=0.6B, Category=Supervised Baseline2026.01 | — | 0.2356 | 0.0546 | -0.3102 | |
| Temperature ScalingModel Scale=1.7B, Category=Supervised Baseline2026.01 | — | 0.1744 | 0.0591 | -0.5282 | |
| Temperature ScalingModel Scale=4B, Category=Supervised Baseline2026.01 | — | 0.1299 | 0.0563 | -0.6727 | |
| Temperature ScalingModel Scale=14B, Category=Supervised Baseline2026.01 | — | 0.1011 | 0.0679 | -0.7565 |