Language Understanding on MMLU (Overall Score)
69MMLU ScoreUnpruned
Evaluation Results
| Method | Links | |
|---|---|---|
| UnprunedBackbone=Llama-3.1-8B, FFN Sparsity=0%2026.04 | 69 | |
| Llama 3.1 8B Instruct# Bits=BF162026.05 | 68.34 | |
| FlexRound+LFQ# Bits=W42026.05 | 66.97 | |
| FlexRound+RILQ# Bits=W42026.05 | 66.86 | |
| FlexRound# Bits=W42026.05 | 66.19 | |
| OmniQuant+RILQ# Bits=W42026.05 | 66.07 | |
| OmniQuant+LFQ# Bits=W42026.05 | 65.48 | |
| OmniQuant# Bits=W42026.05 | 64.87 | |
| Block-AP+RILQ# Bits=W42026.05 | 64.62 | |
| Block-AP+LFQ# Bits=W42026.05 | 63.77 | |
| Block-AP# Bits=W42026.05 | 63.24 | |
| ARPipeline=Qwen3-0.6B2026.04 | 52.8 | |
| Base modelRuntime (min)=N/A2026.04 | 41.3 | |
| TIDE-SharedPipeline=Shared-Tokenizer2026.04 | 39.92 | |
| TIDE-CrossPipeline=Cross-Tokenizer2026.04 | 39.59 | |
| KLPipeline=Shared-Tokenizer2026.04 | 39.57 | |
| TIDE-SharedPipeline=Cross-Tokenizer2026.04 | 39.54 | |
| CALMPipeline=Cross-Tokenizer2026.04 | 39.42 | |
| SIFTRuntime (min)=215.62026.04 | 39.2 | |
| BD3LMPipeline=Qwen3-0.6B2026.04 | 39.15 | |
| TIDE-CrossPipeline=Shared-Tokenizer2026.04 | 39.09 | |
| POMERuntime (min)=115.42026.04 | 38.7 | |
| BLURRuntime (min)=114.72026.04 | 37.4 | |
| AdamWRuntime (min)=110.32026.04 | 36.6 | |
| MuonRuntime (min)=191.22026.04 | 36.1 | |
| Magnitude (channel)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.04 | 30 | |
| SCAR-ProtBackbone=Llama-3.1-8B, FFN Sparsity=50%2026.04 | 30 | |
| SCAR-LPBackbone=Llama-3.1-8B, FFN Sparsity=50%2026.04 | 29 | |
| SCAR-ConnBackbone=Llama-3.1-8B, FFN Sparsity=50%2026.04 | 29 | |
| FLAP (channel)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.04 | 28 | |
| Act. L2Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.04 | 28 | |
| RQBackbone=Llama-3.1-8B, FFN Sparsity=50%2026.04 | 28 | |
| Gaussian MI (analytic)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.04 | 28 | |
| RIA (channel)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.04 | 27 | |
| LLM-Pruner (channel)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.04 | 23 | |
| SparseGPT (channel)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.04 | 22 | |
| SlimLLM (channel)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.04 | 22 | |
| Wanda (channel)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.04 | 21 | |
| OWL (channel)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.04 | 21 | |
| RandomBackbone=Llama-3.1-8B, FFN Sparsity=50%2026.04 | 16 |