List your company
ResearchFormerly WizWand

Multi-task Language Understanding on MMLU (5-shot)

78.74Accuracy (5-shot)Full Precision
20.791235.835650.8865.9244Feb 6, 2026
Updated 8mo ago

Evaluation results

MethodLinks
Full PrecisionBackbone=Llama3-70B, Mixed-Precision=false, Bits=16, Quantizer Type=Uniform2026.0278.74
ScaleBITS + RTNBackbone=Llama3-70B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.0276.88
GuidedQuantBackbone=Llama3-70B, Mixed-Precision=false, Bits=3, Quantizer Type=Non-uniform2026.0276.86
GPTQ-g128Backbone=Llama3-70B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.0274.51
Full PrecisionBackbone=Llama3-8B, Mixed-Precision=false, Bits=16, Quantizer Type=Uniform2026.0265.34
ScaleBITS + RTNBackbone=Llama3-8B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.0259.35
GuidedQuantBackbone=Llama3-8B, Mixed-Precision=false, Bits=3, Quantizer Type=Non-uniform2026.0259.07
ScaleBITS + RTNBackbone=Llama3-70B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.0258.82
SlimLLM-g128Backbone=Llama3-8B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.0255.03
RTN-g128Backbone=Llama3-70B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.0254.56
GPTQ-g128Backbone=Llama3-8B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.0254.27
RTN-g128Backbone=Llama3-8B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.0246.45
Full PrecisionBackbone=Llama2-7B, Mixed-Precision=false, Bits=16, Quantizer Type=Uniform2026.0245.86
GuidedQuantBackbone=Llama3-70B, Mixed-Precision=false, Bits=2, Quantizer Type=Non-uniform2026.0245.55
ScaleBITS + RTNBackbone=Llama2-7B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.0243.48
GuidedQuantBackbone=Llama2-7B, Mixed-Precision=false, Bits=3, Quantizer Type=Non-uniform2026.0243.38
GPTQ-g128Backbone=Llama2-7B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.0241.43
SlimLLM-g128Backbone=Llama2-7B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.0239.79
ScaleBITS + RTNBackbone=Llama3-8B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.0239.36
RTN-g128Backbone=Llama2-7B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.0237.66
ScaleBITS + RTNBackbone=Llama2-7B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.0234.01
GuidedQuantBackbone=Llama2-7B, Mixed-Precision=false, Bits=2, Quantizer Type=Non-uniform2026.0229.37
GuidedQuantBackbone=Llama3-8B, Mixed-Precision=false, Bits=2, Quantizer Type=Non-uniform2026.0226.97
GPTQ-g128Backbone=Llama3-70B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.0226.63
SlimLLM-g128Backbone=Llama3-8B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.0225.51
GPTQ-g128Backbone=Llama3-8B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.0225.47
GPTQ-g128Backbone=Llama2-7B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.0224.95
RTN-g128Backbone=Llama2-7B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.0224.8
RTN-g128Backbone=Llama3-8B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.0224.1
SlimLLM-g128Backbone=Llama2-7B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.0223.69
RTN-g128Backbone=Llama3-70B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.0223.02