Language Understanding on Llama-3.1-70B Evaluation Suite (MMLU, WinoGrande, HellaSwag, ARC)
78.58MMLUFP16
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FP16q̄=16.000, tok/s=OOM, Base Model=Llama-3.1-70B2025.12 | 78.58 | 79.64 | 85.03 | 86.66 | 64.93 | 78.97 | |
| Qwen3-4BStatus=pretrained reference2026.05 | 73.1 | 72 | 75.8 | 86.2 | 64.6 | 74.34 | |
| AQLM-1x16q̄=2.055, tok/s=5.5, Base Model=Llama-3.1-70B2025.12 | 73.07 | 76.16 | 80.83 | 82.2 | 57.17 | 73.89 | |
| CodeGEMM-m1v4g32q̄=2.500, tok/s=49.1, Base Model=Llama-3.1-70B2025.12 | 71.21 | 76.64 | 79.43 | 82.41 | 56.06 | 73.15 | |
| CodeGEMM-m1v4g128q̄=2.125, tok/s=51.2, Base Model=Llama-3.1-70B2025.12 | 68.15 | 74.9 | 75.37 | 79.42 | 52.73 | 70.11 | |
| Qwen3-1.7BStatus=pretrained reference2026.05 | 62.6 | 66.1 | 67.1 | 81.9 | 55.5 | 66.63 | |
| AQLM-2x8q̄=2.002, tok/s=19.0, Base Model=Llama-3.1-70B2025.12 | 61.45 | 59.59 | 52.83 | 48.82 | 28.67 | 50.27 | |
| DO-ACPK=8, Training tokens=~4B2026.05 | 46.1 | 63.1 | 60.3 | 75.6 | 45.4 | 58.1 | |
| SFK=16, Training tokens=~4B2026.05 | 32.7 | 61.2 | 56.3 | 74 | 43.1 | 53.46 | |
| Random FFN + teacher attnTraining tokens=~4B2026.05 | 27.1 | 54.4 | 45.4 | 66 | 34.2 | 45.44 | |
| FlexRound-q2g128q̄=2.125, tok/s=41.7, Base Model=Llama-3.1-70B2025.12 | 26.7 | 53.59 | 50.67 | 25.13 | 24.83 | 36.58 | |
| D2D pruningPruning Path=Qwen3-32B -> 3.4B, Training tokens=~4B2026.05 | 26.6 | 60.5 | 57.5 | 73.1 | 41.5 | 51.84 | |
| GPTQ-q2g128q̄=2.125, tok/s=41.7, Base Model=Llama-3.1-70B2025.12 | 26.35 | 53.04 | 49.04 | 48.95 | 29.52 | 41.38 |