List your company
ResearchFormerly WizWand

Zero-shot Classification on WinoGrande, PiQA, HellaSwag, ARC-easy, ARC-challenge, and BoolQ

76.88Avg Zero-shot AccFull Precision
33.959245.102156.24567.3879Feb 6, 2026
Updated 8mo ago

Evaluation results

MethodLinks
Full PrecisionBackbone=Llama3-70B, Mixed-Precision=false, Bits=16, Quantizer Type=Uniform2026.0276.88
GuidedQuantBackbone=Llama3-70B, Mixed-Precision=false, Bits=3, Quantizer Type=Non-uniform2026.0275.51
ScaleBITS + RTNBackbone=Llama3-70B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.0275
GPTQ-g128Backbone=Llama3-70B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.0273.72
Full PrecisionBackbone=Llama3-8B, Mixed-Precision=false, Bits=16, Quantizer Type=Uniform2026.0270.69
GuidedQuantBackbone=Llama3-8B, Mixed-Precision=false, Bits=3, Quantizer Type=Non-uniform2026.0268.58
ScaleBITS + RTNBackbone=Llama3-8B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.0267.87
Full PrecisionBackbone=Llama2-7B, Mixed-Precision=false, Bits=16, Quantizer Type=Uniform2026.0266.55
RTN-g128Backbone=Llama3-70B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.0266.33
SlimLLM-g128Backbone=Llama3-8B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.0266.03
ScaleBITS + RTNBackbone=Llama2-7B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.0265.53
GuidedQuantBackbone=Llama2-7B, Mixed-Precision=false, Bits=3, Quantizer Type=Non-uniform2026.0265.12
GPTQ-g128Backbone=Llama3-8B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.0264.97
ScaleBITS + RTNBackbone=Llama3-70B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.0264.72
GuidedQuantBackbone=Llama3-70B, Mixed-Precision=false, Bits=2, Quantizer Type=Non-uniform2026.0264.39
GPTQ-g128Backbone=Llama2-7B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.0264.15
RTN-g128Backbone=Llama2-7B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.0263.65
SlimLLM-g128Backbone=Llama2-7B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.0263.06
RTN-g128Backbone=Llama3-8B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.0260.39
ScaleBITS + RTNBackbone=Llama2-7B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.0259.53
ScaleBITS + RTNBackbone=Llama3-8B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.0258.53
GuidedQuantBackbone=Llama3-8B, Mixed-Precision=false, Bits=2, Quantizer Type=Non-uniform2026.0256.21
GuidedQuantBackbone=Llama2-7B, Mixed-Precision=false, Bits=2, Quantizer Type=Non-uniform2026.0251.35
SlimLLM-g128Backbone=Llama2-7B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.0247
GPTQ-g128Backbone=Llama3-70B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.0246.6
GPTQ-g128Backbone=Llama2-7B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.0240.67
RTN-g128Backbone=Llama3-8B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.0237.16
SlimLLM-g128Backbone=Llama3-8B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.0237.13
GPTQ-g128Backbone=Llama3-8B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.0235.82
RTN-g128Backbone=Llama2-7B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.0235.8
RTN-g128Backbone=Llama3-70B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.0235.61