List your company
ResearchFormerly WizWand

Language Understanding on MMLU (Overall Score)

69MMLU ScoreUnpruned
13.8828.1942.556.81Apr 5, 2026Apr 13, 2026Apr 22, 2026May 1, 2026May 10, 2026May 19, 2026May 28, 2026
Updated 5mo ago

Evaluation results

MethodLinks
UnprunedBackbone=Llama-3.1-8B, FFN Sparsity=0%2026.0469
Llama 3.1 8B Instruct# Bits=BF162026.0568.34
FlexRound+LFQ# Bits=W42026.0566.97
FlexRound+RILQ# Bits=W42026.0566.86
FlexRound# Bits=W42026.0566.19
OmniQuant+RILQ# Bits=W42026.0566.07
OmniQuant+LFQ# Bits=W42026.0565.48
OmniQuant# Bits=W42026.0564.87
Block-AP+RILQ# Bits=W42026.0564.62
Block-AP+LFQ# Bits=W42026.0563.77
Block-AP# Bits=W42026.0563.24
ARPipeline=Qwen3-0.6B2026.0452.8
Base modelRuntime (min)=N/A2026.0441.3
TIDE-SharedPipeline=Shared-Tokenizer2026.0439.92
TIDE-CrossPipeline=Cross-Tokenizer2026.0439.59
KLPipeline=Shared-Tokenizer2026.0439.57
TIDE-SharedPipeline=Cross-Tokenizer2026.0439.54
CALMPipeline=Cross-Tokenizer2026.0439.42
SIFTRuntime (min)=215.62026.0439.2
BD3LMPipeline=Qwen3-0.6B2026.0439.15
TIDE-CrossPipeline=Shared-Tokenizer2026.0439.09
POMERuntime (min)=115.42026.0438.7
BLURRuntime (min)=114.72026.0437.4
AdamWRuntime (min)=110.32026.0436.6
MuonRuntime (min)=191.22026.0436.1
Magnitude (channel)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.0430
SCAR-ProtBackbone=Llama-3.1-8B, FFN Sparsity=50%2026.0430
SCAR-LPBackbone=Llama-3.1-8B, FFN Sparsity=50%2026.0429
SCAR-ConnBackbone=Llama-3.1-8B, FFN Sparsity=50%2026.0429
FLAP (channel)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.0428
Act. L2Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.0428
RQBackbone=Llama-3.1-8B, FFN Sparsity=50%2026.0428
Gaussian MI (analytic)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.0428
RIA (channel)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.0427
LLM-Pruner (channel)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.0423
SparseGPT (channel)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.0422
SlimLLM (channel)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.0422
Wanda (channel)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.0421
OWL (channel)Backbone=Llama-3.1-8B, FFN Sparsity=50%2026.0421
RandomBackbone=Llama-3.1-8B, FFN Sparsity=50%2026.0416