Zero-shot Evaluation on 9 diverse tasks zero-shot
73.81Average Accuracy (Zero-shot)FloatingPoint
Evaluation Results
| Method | Links | |
|---|---|---|
| FloatingPointBackbone=LLaMA-3 70B, W-A-KV Bits=16-16-162026.05 | 73.81 | |
| OSTQuantBackbone=LLaMA-3 70B, W-A-KV Bits=4-16-16, Distillation=false2026.05 | 73.69 | |
| INFOQUANTBackbone=LLaMA-3 70B, W-A-KV Bits=4-16-162026.05 | 73.25 | |
| QuaRotBackbone=LLaMA-3 70B, W-A-KV Bits=4-16-162026.05 | 72.93 | |
| SpinQuantBackbone=LLaMA-3 70B, W-A-KV Bits=4-16-162026.05 | 72.9 | |
| OSTQuantBackbone=LLaMA-3 70B, W-A-KV Bits=4-4-16, Distillation=true2026.05 | 72.21 | |
| OSTQuantBackbone=LLaMA-3 70B, W-A-KV Bits=4-4-4, Distillation=true2026.05 | 71.69 | |
| FloatingPointBackbone=LLaMA-2 70B, W-A-KV Bits=16-16-162026.05 | 71.59 | |
| OSTQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-16-16, Distillation=false2026.05 | 71.48 | |
| INFOQUANTBackbone=LLaMA-2 70B, W-A-KV Bits=4-16-162026.05 | 71.25 | |
| QuaRotBackbone=LLaMA-2 70B, W-A-KV Bits=4-16-162026.05 | 71.21 | |
| SpinQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-16-162026.05 | 71.12 | |
| GPTQBackbone=LLaMA-2 70B, W-A-KV Bits=4-16-162026.05 | 70.96 | |
| OSTQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-16, Distillation=true2026.05 | 70.92 | |
| AWQBackbone=LLaMA-2 70B, W-A-KV Bits=4-16-162026.05 | 70.88 | |
| OSTQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-4, Distillation=true2026.05 | 70.84 | |
| INFOQUANTBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-162026.05 | 70.82 | |
| INFOQUANTBackbone=LLaMA-3 70B, W-A-KV Bits=4-4-162026.05 | 70.71 | |
| SpinQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-162026.05 | 70.58 | |
| SpinQuantBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-42026.05 | 70.57 | |
| INFOQUANTBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-42026.05 | 70.35 | |
| QuaRotBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-42026.05 | 70.3 | |
| INFOQUANTBackbone=LLaMA-3 70B, W-A-KV Bits=4-4-42026.05 | 70.21 | |
| INFOQUANT*Backbone=LLaMA-2 70B, W-A-KV Bits=4-4-4, Rotation=global orthogonal2026.05 | 70.1 | |
| INFOQUANT*Backbone=LLaMA-3 70B, W-A-KV Bits=4-4-4, Rotation=global orthogonal2026.05 | 70.01 | |
| QuaRotBackbone=LLaMA-2 70B, W-A-KV Bits=4-4-162026.05 | 69.96 | |
| AWQBackbone=LLaMA-3 70B, W-A-KV Bits=4-16-162026.05 | 68.92 | |
| FloatingPointBackbone=LLaMA-3 8B, W-A-KV Bits=16-16-162026.05 | 68.09 | |
| OSTQuantBackbone=LLaMA-3 8B, W-A-KV Bits=4-16-16, Distillation=false2026.05 | 67.8 | |
| FloatingPointBackbone=LLaMA-2 13B, W-A-KV Bits=16-16-162026.05 | 67.61 | |
| INFOQUANTBackbone=LLaMA-3 8B, W-A-KV Bits=4-16-162026.05 | 67.36 | |
| OSTQuantBackbone=LLaMA-2 13B, W-A-KV Bits=4-16-16, Distillation=false2026.05 | 67.31 | |
| QuaRotBackbone=LLaMA-3 8B, W-A-KV Bits=4-16-162026.05 | 67.27 | |
| INFOQUANTBackbone=LLaMA-2 13B, W-A-KV Bits=4-16-162026.05 | 67.27 | |
| SpinQuantBackbone=LLaMA-2 13B, W-A-KV Bits=4-16-162026.05 | 67.14 | |
| AWQBackbone=LLaMA-3 8B, W-A-KV Bits=4-16-162026.05 | 67.03 | |
| SpinQuantBackbone=LLaMA-3 70B, W-A-KV Bits=4-4-162026.05 | 66.99 | |
| QuaRotBackbone=LLaMA-2 13B, W-A-KV Bits=4-16-162026.05 | 66.95 | |
| INFOQUANTBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-162026.05 | 66.71 | |
| SpinQuantBackbone=LLaMA-3 8B, W-A-KV Bits=4-16-162026.05 | 66.54 | |
| INFOQUANTBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-42026.05 | 66.33 | |
| SpinQuantBackbone=LLaMA-3 70B, W-A-KV Bits=4-4-42026.05 | 66.31 | |
| AWQBackbone=LLaMA-2 13B, W-A-KV Bits=4-16-162026.05 | 66.25 | |
| OSTQuantBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-16, Distillation=true2026.05 | 66.24 | |
| INFOQUANT*Backbone=LLaMA-2 13B, W-A-KV Bits=4-4-4, Rotation=global orthogonal2026.05 | 66.12 | |
| INFOQUANTBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-162026.05 | 65.74 | |
| INFOQUANTBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-42026.05 | 65.57 | |
| QuaRotBackbone=LLaMA-3 70B, W-A-KV Bits=4-4-162026.05 | 65.56 | |
| BASE-QBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-42026.05 | 65.48 | |
| OSTQuantBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-4, Distillation=true2026.05 | 65.41 | |
| BASE-QBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-42026.05 | 65.39 | |
| OSTQuantBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-4, Distillation=true2026.05 | 65.37 | |
| QuaRotBackbone=LLaMA-3 70B, W-A-KV Bits=4-4-42026.05 | 65.33 | |
| FloatingPointBackbone=LLaMA-2 7B, W-A-KV Bits=16-16-162026.05 | 65.21 | |
| QuaRotBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-42026.05 | 65.16 | |
| OSTQuantBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-16, Distillation=true2026.05 | 65.14 | |
| QuaRotBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-162026.05 | 65.13 | |
| OSTQuant†Backbone=LLaMA-3 8B, W-A-KV Bits=4-4-4, Distillation=false2026.05 | 65.13 | |
| INFOQUANT*Backbone=LLaMA-3 8B, W-A-KV Bits=4-4-4, Rotation=global orthogonal2026.05 | 64.79 | |
| GPTQBackbone=LLaMA-2 13B, W-A-KV Bits=4-16-162026.05 | 64.71 | |
| OSTQuantBackbone=LLaMA-2 7B, W-A-KV Bits=4-16-16, Distillation=false2026.05 | 64.37 | |
| INFOQUANTBackbone=LLaMA-2 7B, W-A-KV Bits=4-16-162026.05 | 64.34 | |
| QuaRotBackbone=LLaMA-2 7B, W-A-KV Bits=4-16-162026.05 | 64.3 | |
| SpinQuantBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-42026.05 | 64.13 | |
| SpinQuantBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-162026.05 | 64.11 | |
| SpinQuantBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-42026.05 | 64.1 | |
| OSTQuantBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-16, Distillation=true2026.05 | 63.9 | |
| AWQBackbone=LLaMA-2 7B, W-A-KV Bits=4-16-162026.05 | 63.89 | |
| SpinQuantBackbone=LLaMA-2 7B, W-A-KV Bits=4-16-162026.05 | 63.59 | |
| SpinQuantBackbone=LLaMA-2 13B, W-A-KV Bits=4-4-162026.05 | 63.23 | |
| OSTQuantBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-4, Distillation=true2026.05 | 63.18 | |
| INFOQUANTBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-42026.05 | 63.16 | |
| INFOQUANTBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-162026.05 | 62.84 | |
| INFOQUANT*Backbone=LLaMA-2 7B, W-A-KV Bits=4-4-4, Rotation=global orthogonal2026.05 | 62.62 | |
| BASE-QBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-42026.05 | 62.5 | |
| OSTQuant†Backbone=LLaMA-2 7B, W-A-KV Bits=4-4-4, Distillation=false2026.05 | 62.45 | |
| SpinQuantBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-42026.05 | 62.01 | |
| QuaRotBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-162026.05 | 61.87 | |
| QuaRotBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-162026.05 | 61.69 | |
| QuaRotBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-42026.05 | 61.48 | |
| QuaRotBackbone=LLaMA-3 8B, W-A-KV Bits=4-4-42026.05 | 61.38 | |
| GPTQBackbone=LLaMA-3 8B, W-A-KV Bits=4-16-162026.05 | 61.03 | |
| GPTQBackbone=LLaMA-2 7B, W-A-KV Bits=4-16-162026.05 | 60.86 | |
| SpinQuantBackbone=LLaMA-2 7B, W-A-KV Bits=4-4-162026.05 | 57.37 | |
| GPTQBackbone=LLaMA-3 70B, W-A-KV Bits=4-16-162026.05 | 31.45 |