Zero-shot Commonsense Reasoning on ARC-Easy, ARC-Challenge, and HellaSwag Average
68.6Zero-Shot AccuracyBF16
Evaluation results
| Method | Links | |
|---|---|---|
| BF16Backbone=Llama-3, Parameters=8B, Precision=BF16, Protocol=Zero-shot2026.06 | 68.6 | |
| BF16Backbone=Llama-3, Parameters=3B, Precision=BF16, Protocol=Zero-shot2026.06 | 62 | |
| BF16Backbone=Qwen-2.5, Parameters=7B, Precision=BF16, Protocol=Zero-shot2026.06 | 57.7 | |
| BF16Backbone=Qwen-2.5, Parameters=3B, Precision=BF16, Protocol=Zero-shot2026.06 | 56.5 | |
| BF16Backbone=Qwen-2.5, Parameters=1.5B, Precision=BF16, Protocol=Zero-shot2026.06 | 54.9 | |
| BF16Backbone=Llama-3, Parameters=1B, Precision=BF16, Protocol=Zero-shot2026.06 | 52.6 | |
| Qronos ⊙ PiSOBackbone=Llama-3, Parameters=8B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 44 | |
| Qronos ⊙ PiSOBackbone=Llama-3, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 41.4 | |
| GPTQ ⊙ PiSOBackbone=Llama-3, Parameters=8B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 40.2 | |
| Qronos ⊙ PiSOBackbone=Qwen-2.5, Parameters=7B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 38.1 | |
| Qronos ⊙ PiSOBackbone=Llama-3, Parameters=1B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 37.5 | |
| GPTQ + PiSOBackbone=Qwen-2.5, Parameters=7B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 37.1 | |
| BeaconBackbone=Llama-3, Parameters=8B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 36.8 | |
| GPTQ ⊙ PiSOBackbone=Llama-3, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 36.3 | |
| Qronos + PiSOBackbone=Qwen-2.5, Parameters=7B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 36.1 | |
| GPTQ ⊙ PiSOBackbone=Llama-3, Parameters=1B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 35.5 | |
| GPTQ ⊙ PiSOBackbone=Qwen-2.5, Parameters=7B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 34.9 | |
| Qronos ⊙ PiSOBackbone=Qwen-2.5, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 34.8 | |
| GPTQ + PiSOBackbone=Llama-3, Parameters=8B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 34.7 | |
| GPTQ + PiSOBackbone=Qwen-2.5, Parameters=1.5B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 34.4 | |
| GPTQ ⊙ PiSOBackbone=Qwen-2.5, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 34.4 | |
| BeaconBackbone=Qwen-2.5, Parameters=7B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 33.9 | |
| BeaconBackbone=Llama-3, Parameters=1B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 33.8 | |
| GPTQ ⊙ PiSOBackbone=Qwen-2.5, Parameters=1.5B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 33.7 | |
| Qronos + absmaxBackbone=Llama-3, Parameters=8B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 33.5 | |
| RTN + absmaxBackbone=Llama-3, Parameters=8B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 33.4 | |
| BeaconBackbone=Llama-3, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 33.3 | |
| Qronos + PiSOBackbone=Llama-3, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 33.3 | |
| GPTQ + PiSOBackbone=Llama-3, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 33.2 | |
| GPTQ + PiSOBackbone=Llama-3, Parameters=1B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 33.1 | |
| RTN + absmaxBackbone=Llama-3, Parameters=1B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 33 | |
| GPTQ + absmaxBackbone=Qwen-2.5, Parameters=1.5B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 33 | |
| Qronos + PiSOBackbone=Llama-3, Parameters=8B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 33 | |
| RTN + absmaxBackbone=Qwen-2.5, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.9 | |
| GPTQ + PiSOBackbone=Qwen-2.5, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.9 | |
| Qronos + PiSOBackbone=Qwen-2.5, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.9 | |
| GPTQ + absmaxBackbone=Qwen-2.5, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.8 | |
| RTN + PiSOBackbone=Qwen-2.5, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.7 | |
| Qronos + absmaxBackbone=Llama-3, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.7 | |
| Qronos ⊙ PiSOBackbone=Qwen-2.5, Parameters=1.5B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.7 | |
| RTN + absmaxBackbone=Llama-3, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.5 | |
| BeaconBackbone=Qwen-2.5, Parameters=1.5B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.5 | |
| Qronos + absmaxBackbone=Qwen-2.5, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.5 | |
| BeaconBackbone=Qwen-2.5, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.4 | |
| RTN + absmaxBackbone=Qwen-2.5, Parameters=7B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.3 | |
| RTN + PiSOBackbone=Llama-3, Parameters=1B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.3 | |
| Qronos + PiSOBackbone=Qwen-2.5, Parameters=1.5B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.3 | |
| RTN + PiSOBackbone=Qwen-2.5, Parameters=7B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.2 | |
| GPTQ + absmaxBackbone=Qwen-2.5, Parameters=7B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.1 | |
| Qronos + absmaxBackbone=Qwen-2.5, Parameters=1.5B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.1 | |
| Qronos + absmaxBackbone=Qwen-2.5, Parameters=7B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.1 | |
| Qronos + PiSOBackbone=Llama-3, Parameters=1B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32.1 | |
| RTN + PiSOBackbone=Llama-3, Parameters=8B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 32 | |
| GPTQ + absmaxBackbone=Llama-3, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 31.9 | |
| RTN + absmaxBackbone=Qwen-2.5, Parameters=1.5B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 31.8 | |
| RTN + PiSOBackbone=Qwen-2.5, Parameters=1.5B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 31.8 | |
| RTN + PiSOBackbone=Llama-3, Parameters=3B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 31.6 | |
| GPTQ + absmaxBackbone=Llama-3, Parameters=1B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 31.6 | |
| Qronos + absmaxBackbone=Llama-3, Parameters=1B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 31.6 | |
| GPTQ + absmaxBackbone=Llama-3, Parameters=8B, Precision=2-bit weight-only, Protocol=Zero-shot2026.06 | 31.5 |