List your company
ResearchFormerly WizWand

Zero-shot Evaluation on 9 diverse tasks zero-shot

73.81Average Accuracy (Zero-shot)FloatingPoint
29.755641.192852.6364.0672May 25, 2026
Updated 5mo ago

Evaluation results

MethodLinks
FloatingPointBackbone=LLaMA-3 70B, W-A-KV Bits=16-16-162026.0573.81
OSTQuantBackbone=LLaMA-3 70B, W-A-KV Bits=4-16-16, Distillation=false2026.0573.69
INFOQUANTBackbone=LLaMA-3 70B, W-A-KV Bits=4-16-162026.0573.25
QuaRotBackbone=LLaMA-3 70B, W-A-KV Bits=4-16-162026.0572.93
SpinQuantBackbone=LLaMA-3 70B, W-A-KV Bits=4-16-162026.0572.9
OSTQuantBackbone=LLaMA-3 70B, W-A-KV Bits=4-4-16, Distillation=true2026.0572.21
OSTQuantBackbone=LLaMA-3 70B, W-A-KV Bits=4-4-4, Distillation=true2026.0571.69
FloatingPointBackbone=LLaMA-2 70B, W-A-KV Bits=16-16-162026.0571.59
OSTQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-16-16, Distillation=false2026.0571.48
INFOQUANTBackbone=LLaMA-2 70B, W-A-KV Bits=4-16-162026.0571.25
QuaRotBackbone=LLaMA-2 70B, W-A-KV Bits=4-16-162026.0571.21
SpinQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-16-162026.0571.12
GPTQBackbone=LLaMA-2 70B, W-A-KV Bits=4-16-162026.0570.96
OSTQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-16, Distillation=true2026.0570.92
AWQBackbone=LLaMA-2 70B, W-A-KV Bits=4-16-162026.0570.88
OSTQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-4, Distillation=true2026.0570.84
INFOQUANTBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-162026.0570.82
INFOQUANTBackbone=LLaMA-3 70B, W-A-KV Bits=4-4-162026.0570.71
SpinQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-162026.0570.58
SpinQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-42026.0570.57
INFOQUANTBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-42026.0570.35
QuaRotBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-42026.0570.3
INFOQUANTBackbone=LLaMA-3 70B, W-A-KV Bits=4-4-42026.0570.21
INFOQUANT*Backbone=LLaMA-2 70B, W-A-KV Bits=4-4-4, Rotation=global orthogonal2026.0570.1
INFOQUANT*Backbone=LLaMA-3 70B, W-A-KV Bits=4-4-4, Rotation=global orthogonal2026.0570.01
QuaRotBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-162026.0569.96
AWQBackbone=LLaMA-3 70B, W-A-KV Bits=4-16-162026.0568.92
FloatingPointBackbone=LLaMA-3 8B, W-A-KV Bits=16-16-162026.0568.09
OSTQuantBackbone=LLaMA-3 8B, W-A-KV Bits=4-16-16, Distillation=false2026.0567.8
FloatingPointBackbone=LLaMA-2 13B, W-A-KV Bits=16-16-162026.0567.61
INFOQUANTBackbone=LLaMA-3 8B, W-A-KV Bits=4-16-162026.0567.36
OSTQuantBackbone=LLaMA-2 13B, W-A-KV Bits=4-16-16, Distillation=false2026.0567.31
QuaRotBackbone=LLaMA-3 8B, W-A-KV Bits=4-16-162026.0567.27
INFOQUANTBackbone=LLaMA-2 13B, W-A-KV Bits=4-16-162026.0567.27
SpinQuantBackbone=LLaMA-2 13B, W-A-KV Bits=4-16-162026.0567.14
AWQBackbone=LLaMA-3 8B, W-A-KV Bits=4-16-162026.0567.03
SpinQuantBackbone=LLaMA-3 70B, W-A-KV Bits=4-4-162026.0566.99
QuaRotBackbone=LLaMA-2 13B, W-A-KV Bits=4-16-162026.0566.95
INFOQUANTBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-162026.0566.71
SpinQuantBackbone=LLaMA-3 8B, W-A-KV Bits=4-16-162026.0566.54
INFOQUANTBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-42026.0566.33
SpinQuantBackbone=LLaMA-3 70B, W-A-KV Bits=4-4-42026.0566.31
AWQBackbone=LLaMA-2 13B, W-A-KV Bits=4-16-162026.0566.25
OSTQuantBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-16, Distillation=true2026.0566.24
INFOQUANT*Backbone=LLaMA-2 13B, W-A-KV Bits=4-4-4, Rotation=global orthogonal2026.0566.12
INFOQUANTBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-162026.0565.74
INFOQUANTBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-42026.0565.57
QuaRotBackbone=LLaMA-3 70B, W-A-KV Bits=4-4-162026.0565.56
BASE-QBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-42026.0565.48
OSTQuantBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-4, Distillation=true2026.0565.41
BASE-QBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-42026.0565.39
OSTQuantBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-4, Distillation=true2026.0565.37
QuaRotBackbone=LLaMA-3 70B, W-A-KV Bits=4-4-42026.0565.33
FloatingPointBackbone=LLaMA-2 7B, W-A-KV Bits=16-16-162026.0565.21
QuaRotBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-42026.0565.16
OSTQuantBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-16, Distillation=true2026.0565.14
QuaRotBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-162026.0565.13
OSTQuant†Backbone=LLaMA-3 8B, W-A-KV Bits=4-4-4, Distillation=false2026.0565.13
INFOQUANT*Backbone=LLaMA-3 8B, W-A-KV Bits=4-4-4, Rotation=global orthogonal2026.0564.79
GPTQBackbone=LLaMA-2 13B, W-A-KV Bits=4-16-162026.0564.71
OSTQuantBackbone=LLaMA-2 7B, W-A-KV Bits=4-16-16, Distillation=false2026.0564.37
INFOQUANTBackbone=LLaMA-2 7B, W-A-KV Bits=4-16-162026.0564.34
QuaRotBackbone=LLaMA-2 7B, W-A-KV Bits=4-16-162026.0564.3
SpinQuantBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-42026.0564.13
SpinQuantBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-162026.0564.11
SpinQuantBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-42026.0564.1
OSTQuantBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-16, Distillation=true2026.0563.9
AWQBackbone=LLaMA-2 7B, W-A-KV Bits=4-16-162026.0563.89
SpinQuantBackbone=LLaMA-2 7B, W-A-KV Bits=4-16-162026.0563.59
SpinQuantBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-162026.0563.23
OSTQuantBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-4, Distillation=true2026.0563.18
INFOQUANTBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-42026.0563.16
INFOQUANTBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-162026.0562.84
INFOQUANT*Backbone=LLaMA-2 7B, W-A-KV Bits=4-4-4, Rotation=global orthogonal2026.0562.62
BASE-QBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-42026.0562.5
OSTQuant†Backbone=LLaMA-2 7B, W-A-KV Bits=4-4-4, Distillation=false2026.0562.45
SpinQuantBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-42026.0562.01
QuaRotBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-162026.0561.87
QuaRotBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-162026.0561.69
QuaRotBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-42026.0561.48
QuaRotBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-42026.0561.38
GPTQBackbone=LLaMA-3 8B, W-A-KV Bits=4-16-162026.0561.03
GPTQBackbone=LLaMA-2 7B, W-A-KV Bits=4-16-162026.0560.86
SpinQuantBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-162026.0557.37
GPTQBackbone=LLaMA-3 70B, W-A-KV Bits=4-16-162026.0531.45