Multi-task Language Understanding on MMLU (Subject Accuracy Breakdown)
45.5Average AccuracyLLAMA-1-7B
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LLAMA-1-7BBits=FP162026.04 | 45.5 | 36.1 | 43.3 | 51.6 | 51.8 | |
| DenseModel=LLaMA-2-7B, Sparsity=0%, Evaluation=Zero-shot2026.07 | 45.2 | — | — | — | — | |
| PALSModel=LLaMA-2-7B, Sparsity=50%, Evaluation=Zero-shot2026.07 | 42.2 | — | — | — | — | |
| WandaModel=LLaMA-2-7B, Sparsity=50%, Evaluation=Zero-shot2026.07 | 41.5 | — | — | — | — | |
| MagnitudeModel=LLaMA-2-7B, Sparsity=50%, Evaluation=Zero-shot2026.07 | 36.4 | — | — | — | — | |
| LBLLMBits=W(1+1)A4, Backbone=LLAMA-1-7B2026.04 | 28.8 | 26.4 | 28.8 | 27.7 | 32 | |
| ARB-LLMBits=W(1+1)A4, Backbone=LLAMA-1-7B2026.04 | 27.7 | 30.2 | 25.4 | 30.5 | 26 | |
| AtomBits=W(1+1)A4, Backbone=LLAMA-1-7B2026.04 | 25.3 | 25.9 | 24.9 | 24 | 26.3 |