Common-sense Reasoning on ARC-e, ARC-c, BoolQ, HellaSwag, OpenBookQA, PIQA, SIQA, WinoGrande (0-shot)
73.410-shot Accuracy16-bit
Evaluation results
| Method | Links | |
|---|---|---|
| 16-bitModel Family=Qwen 2.5, Model Size=72B, Quantization Setting=16-bit2026.06 | 73.41 | |
| 16-bitModel Family=Llama, Model Size=3-70B, Quantization Setting=16-bit2026.06 | 73.09 | |
| OffQModel Family=Qwen 2.5, Model Size=72B, Quantization Setting=W4A4KV42026.06 | 72.68 | |
| ResQModel Family=Qwen 2.5, Model Size=72B, Quantization Setting=W4A4KV42026.06 | 71.98 | |
| OSTQuantModel Family=Llama, Model Size=3-70B, Quantization Setting=W4A4KV42026.06 | 71.16 | |
| ResQModel Family=Llama, Model Size=3-70B, Quantization Setting=W4A4KV42026.06 | 71.14 | |
| KurTailModel Family=Llama, Model Size=3-70B, Quantization Setting=W4A4KV42026.06 | 70.69 | |
| OffQModel Family=Llama, Model Size=3-70B, Quantization Setting=W4A4KV42026.06 | 70.63 | |
| 16-bitModel Family=Qwen 2.5, Model Size=14B, Quantization Setting=16-bit2026.06 | 70.61 | |
| 16-bitModel Family=Qwen 2.5, Model Size=32B, Quantization Setting=16-bit2026.06 | 70.44 | |
| QuaRotModel Family=Qwen 2.5, Model Size=72B, Quantization Setting=W4A4KV42026.06 | 70.28 | |
| OffQModel Family=Qwen 2.5, Model Size=32B, Quantization Setting=W4A4KV42026.06 | 69.59 | |
| ResQModel Family=Qwen 2.5, Model Size=32B, Quantization Setting=W4A4KV42026.06 | 69.55 | |
| OffQModel Family=Qwen 2.5, Model Size=14B, Quantization Setting=W4A4KV42026.06 | 69.2 | |
| ResQModel Family=Qwen 2.5, Model Size=14B, Quantization Setting=W4A4KV42026.06 | 69.16 | |
| DFRotModel Family=Llama, Model Size=3-70B, Quantization Setting=W4A4KV42026.06 | 68.98 | |
| 16-bitModel Family=Qwen 2.5, Model Size=7B, Quantization Setting=16-bit2026.06 | 68.45 | |
| QuaRotModel Family=Qwen 2.5, Model Size=32B, Quantization Setting=W4A4KV42026.06 | 67.9 | |
| QuaRotModel Family=Llama, Model Size=3-70B, Quantization Setting=W4A4KV42026.06 | 67.56 | |
| QuaRotModel Family=Qwen 2.5, Model Size=14B, Quantization Setting=W4A4KV42026.06 | 67.14 | |
| 16-bitModel Family=Llama, Model Size=3-8B, Quantization Setting=16-bit2026.06 | 67.09 | |
| 16-bitModel Family=Llama, Model Size=2-13B, Quantization Setting=16-bit2026.06 | 66.45 | |
| OffQModel Family=Qwen 2.5, Model Size=7B, Quantization Setting=W4A4KV42026.06 | 66.16 | |
| SpinQuantModel Family=Llama, Model Size=3-70B, Quantization Setting=W4A4KV42026.06 | 65.68 | |
| OffQModel Family=Llama, Model Size=3-8B, Quantization Setting=W4A4KV42026.06 | 65.49 | |
| ResQModel Family=Qwen 2.5, Model Size=7B, Quantization Setting=W4A4KV42026.06 | 65.29 | |
| ResQModel Family=Llama, Model Size=2-13B, Quantization Setting=W4A4KV42026.06 | 65.25 | |
| OffQModel Family=Llama, Model Size=2-13B, Quantization Setting=W4A4KV42026.06 | 65.25 | |
| KurTailModel Family=Llama, Model Size=2-13B, Quantization Setting=W4A4KV42026.06 | 65.18 | |
| SpinQuantModel Family=Llama, Model Size=2-13B, Quantization Setting=W4A4KV42026.06 | 64.81 | |
| OSTQuantModel Family=Llama, Model Size=3-8B, Quantization Setting=W4A4KV42026.06 | 64.7 | |
| KurTailModel Family=Llama, Model Size=3-8B, Quantization Setting=W4A4KV42026.06 | 64.63 | |
| OSTQuantModel Family=Llama, Model Size=2-13B, Quantization Setting=W4A4KV42026.06 | 64.19 | |
| 16-bitModel Family=Llama, Model Size=2-7B, Quantization Setting=16-bit2026.06 | 64.15 | |
| ResQModel Family=Llama, Model Size=3-8B, Quantization Setting=W4A4KV42026.06 | 63.91 | |
| DFRotModel Family=Llama, Model Size=2-13B, Quantization Setting=W4A4KV42026.06 | 63.83 | |
| 16-bitModel Family=Qwen 2.5, Model Size=3B, Quantization Setting=16-bit2026.06 | 63.81 | |
| QuaRotModel Family=Llama, Model Size=2-13B, Quantization Setting=W4A4KV42026.06 | 63.8 | |
| SpinQuantModel Family=Llama, Model Size=3-8B, Quantization Setting=W4A4KV42026.06 | 63.76 | |
| 16-bitModel Family=Llama, Model Size=3.2-3B, Quantization Setting=16-bit2026.06 | 62.73 | |
| DFRotModel Family=Llama, Model Size=3-8B, Quantization Setting=W4A4KV42026.06 | 62.35 | |
| OSTQuantModel Family=Llama, Model Size=2-7B, Quantization Setting=W4A4KV42026.06 | 62.11 | |
| QuaRotModel Family=Llama, Model Size=3-8B, Quantization Setting=W4A4KV42026.06 | 62.1 | |
| OffQModel Family=Llama, Model Size=2-7B, Quantization Setting=W4A4KV42026.06 | 61.99 | |
| ResQModel Family=Llama, Model Size=2-7B, Quantization Setting=W4A4KV42026.06 | 61.95 | |
| QUIKModel Family=Qwen 2.5, Model Size=72B, Quantization Setting=W4A4KV42026.06 | 61.9 | |
| OffQModel Family=Qwen 2.5, Model Size=3B, Quantization Setting=W4A4KV42026.06 | 61.47 | |
| KurTailModel Family=Llama, Model Size=2-7B, Quantization Setting=W4A4KV42026.06 | 61.31 | |
| ResQModel Family=Qwen 2.5, Model Size=3B, Quantization Setting=W4A4KV42026.06 | 61.13 | |
| SpinQuantModel Family=Llama, Model Size=2-7B, Quantization Setting=W4A4KV42026.06 | 60.98 | |
| DFRotModel Family=Llama, Model Size=2-7B, Quantization Setting=W4A4KV42026.06 | 60.97 | |
| 16-bitModel Family=Qwen 2.5, Model Size=1.5B, Quantization Setting=16-bit2026.06 | 60.85 | |
| OffQModel Family=Llama, Model Size=3.2-3B, Quantization Setting=W4A4KV42026.06 | 60.8 | |
| QuaRotModel Family=Llama, Model Size=2-7B, Quantization Setting=W4A4KV42026.06 | 60.75 | |
| QUIKModel Family=Llama, Model Size=2-13B, Quantization Setting=W4A4KV42026.06 | 60.21 | |
| QUIKModel Family=Qwen 2.5, Model Size=32B, Quantization Setting=W4A4KV42026.06 | 59.08 | |
| KurTailModel Family=Llama, Model Size=3.2-3B, Quantization Setting=W4A4KV42026.06 | 59.04 | |
| ResQModel Family=Llama, Model Size=3.2-3B, Quantization Setting=W4A4KV42026.06 | 58.99 | |
| QUIKModel Family=Llama, Model Size=3-70B, Quantization Setting=W4A4KV42026.06 | 58.15 | |
| SpinQuantModel Family=Llama, Model Size=3.2-3B, Quantization Setting=W4A4KV42026.06 | 57.89 | |
| QUIKModel Family=Qwen 2.5, Model Size=14B, Quantization Setting=W4A4KV42026.06 | 57.66 | |
| OffQModel Family=Qwen 2.5, Model Size=1.5B, Quantization Setting=W4A4KV42026.06 | 57.53 | |
| QUIKModel Family=Llama, Model Size=2-7B, Quantization Setting=W4A4KV42026.06 | 56.99 | |
| QuaRotModel Family=Llama, Model Size=3.2-3B, Quantization Setting=W4A4KV42026.06 | 56.06 | |
| ResQModel Family=Qwen 2.5, Model Size=1.5B, Quantization Setting=W4A4KV42026.06 | 55.26 | |
| 16-bitModel Family=Llama, Model Size=3.2-1B, Quantization Setting=16-bit2026.06 | 54.86 | |
| QUIKModel Family=Llama, Model Size=3-8B, Quantization Setting=W4A4KV42026.06 | 51.6 | |
| QUIKModel Family=Qwen 2.5, Model Size=3B, Quantization Setting=W4A4KV42026.06 | 51.19 | |
| OffQModel Family=Llama, Model Size=3.2-1B, Quantization Setting=W4A4KV42026.06 | 50.91 | |
| KurTailModel Family=Llama, Model Size=3.2-1B, Quantization Setting=W4A4KV42026.06 | 50.11 | |
| ResQModel Family=Llama, Model Size=3.2-1B, Quantization Setting=W4A4KV42026.06 | 50.11 | |
| QuaRotModel Family=Llama, Model Size=3.2-1B, Quantization Setting=W4A4KV42026.06 | 49.01 | |
| SpinQuantModel Family=Llama, Model Size=3.2-1B, Quantization Setting=W4A4KV42026.06 | 48.78 | |
| QUIKModel Family=Llama, Model Size=3.2-3B, Quantization Setting=W4A4KV42026.06 | 48.74 | |
| QuaRotModel Family=Qwen 2.5, Model Size=3B, Quantization Setting=W4A4KV42026.06 | 47.76 | |
| QUIKModel Family=Llama, Model Size=3.2-1B, Quantization Setting=W4A4KV42026.06 | 44.3 | |
| QUIKModel Family=Qwen 2.5, Model Size=7B, Quantization Setting=W4A4KV42026.06 | 41.48 | |
| GPTQModel Family=Llama, Model Size=3.2-3B, Quantization Setting=W4A4KV42026.06 | 40.34 | |
| GPTQModel Family=Llama, Model Size=3-8B, Quantization Setting=W4A4KV42026.06 | 39.79 | |
| GPTQModel Family=Llama, Model Size=2-7B, Quantization Setting=W4A4KV42026.06 | 38.89 | |
| GPTQModel Family=Qwen 2.5, Model Size=32B, Quantization Setting=W4A4KV42026.06 | 38.53 | |
| QuaRotModel Family=Qwen 2.5, Model Size=7B, Quantization Setting=W4A4KV42026.06 | 38.36 | |
| QuaRotModel Family=Qwen 2.5, Model Size=1.5B, Quantization Setting=W4A4KV42026.06 | 38.33 | |
| GPTQModel Family=Llama, Model Size=3.2-1B, Quantization Setting=W4A4KV42026.06 | 37.98 | |
| GPTQModel Family=Qwen 2.5, Model Size=14B, Quantization Setting=W4A4KV42026.06 | 36.93 | |
| GPTQModel Family=Llama, Model Size=2-13B, Quantization Setting=W4A4KV42026.06 | 35.83 | |
| QUIKModel Family=Qwen 2.5, Model Size=1.5B, Quantization Setting=W4A4KV42026.06 | 35.81 | |
| GPTQModel Family=Qwen 2.5, Model Size=1.5B, Quantization Setting=W4A4KV42026.06 | 35.21 | |
| GPTQModel Family=Qwen 2.5, Model Size=3B, Quantization Setting=W4A4KV42026.06 | 35.1 | |
| GPTQModel Family=Llama, Model Size=3-70B, Quantization Setting=W4A4KV42026.06 | 34.9 | |
| GPTQModel Family=Qwen 2.5, Model Size=7B, Quantization Setting=W4A4KV42026.06 | 34.85 | |
| GPTQModel Family=Qwen 2.5, Model Size=72B, Quantization Setting=W4A4KV42026.06 | 34.54 |