List your company
ResearchFormerly WizWand

Language Modeling Downstream Evaluation on Downstream tasks Average (test)

58.02Average ScoreBlockPruner
34.6240.69546.7752.845Jun 15, 2024Oct 14, 2024Feb 13, 2025Jun 15, 2025Oct 14, 2025Feb 13, 2026Jun 15, 2026
Updated 4mo ago

Evaluation results

MethodLinks
BlockPrunerModel=Mistral-7B-v0.3, Ratio (%)=25.122024.0658.02
ShortGPTModel=Mistral-7B-v0.3, Ratio (%)=24.072024.0657.4
BlockPrunerModel=Mistral-7B-v0.3, Ratio (%)=27.552024.0657.02
BlockPrunerModel=Mistral-7B-v0.3, Ratio (%)=29.982024.0656.1
ShortGPTModel=Mistral-7B-v0.3, Ratio (%)=27.082024.0653.26
BlockPrunerModel=LLaMA3-8B, Ratio (%)=24.862024.0652.36
BlockPrunerModel=LLaMA3-8B, Ratio (%)=27.582024.0650.08
BlockPrunerModel=Mistral-7B-v0.3, Ratio (%)=36.002024.0649.18
g=4, 2× widthTotal Parameters=4.71B, g=4, width=2×, Saved=33.8%2026.0648.2
g=4Total Parameters=3.50B, g=4, Saved=50.9%2026.0647.5
baseline (g=1)Total Parameters=7.12B, g=12026.0647.4
BlockPrunerModel=LLaMA3-8B, Ratio (%)=30.302024.0646.38
BlockPrunerModel=LLaMA3-8B, Ratio (%)=35.732024.0645.41
ShortGPTModel=LLaMA3-8B, Ratio (%)=27.162024.0644.51
OLMoE-1B-7B-0924Total Parameters=6.9B, training_tokens=≈20B2026.0644.4
ShortGPTModel=LLaMA3-8B, Ratio (%)=29.882024.0643.71
ShortGPTModel=LLaMA3-8B, Ratio (%)=35.312024.0642.85
ShortGPTModel=Mistral-7B-v0.3, Ratio (%)=30.092024.0642.79
ShortGPTModel=LLaMA3-8B, Ratio (%)=24.452024.0641.97
ShortGPTModel=Mistral-7B-v0.3, Ratio (%)=36.112024.0641.41
Relative Magnitude (RM)Model=LLaMA3-8B, Ratio (%)=24.452024.0639.34
Relative Magnitude (RM)Model=LLaMA3-8B, Ratio (%)=27.162024.0638.9
Relative Magnitude (RM)Model=Mistral-7B-v0.3, Ratio (%)=24.072024.0637.75
Relative Magnitude (RM)Model=LLaMA3-8B, Ratio (%)=29.882024.0637.48
Relative Magnitude (RM)Model=LLaMA3-8B, Ratio (%)=35.312024.0636.8
Relative Magnitude (RM)Model=Mistral-7B-v0.3, Ratio (%)=30.092024.0636.49
Relative Magnitude (RM)Model=Mistral-7B-v0.3, Ratio (%)=27.082024.0635.89
Relative Magnitude (RM)Model=Mistral-7B-v0.3, Ratio (%)=36.112024.0635.52