Zero-shot Evaluation on Zero-shot Benchmarks Average
68.6Average AccuracyBF16
Evaluation Results
| Method | Links | |
|---|---|---|
| BF16Bit-width=3-bit, Model=Llama-3 8B2026.06 | 68.6 | |
| BF16Bit-width=4-bit, Model=Llama-3 8B2026.06 | 68.6 | |
| GPTQ + PiSOBit-width=4-bit, Model=Llama-3 8B2026.06 | 68.3 | |
| GPTQ ⊙ PiSOBit-width=4-bit, Model=Llama-3 8B2026.06 | 68.1 | |
| Qronos + PiSOBit-width=4-bit, Model=Llama-3 8B2026.06 | 67.4 | |
| RTN + PiSOBit-width=4-bit, Model=Llama-3 8B2026.06 | 67.3 | |
| Qronos ⊙ PiSOBit-width=4-bit, Model=Llama-3 8B2026.06 | 67 | |
| GPTQ + absmaxBit-width=4-bit, Model=Llama-3 8B2026.06 | 65.6 | |
| Qronos + absmaxBit-width=4-bit, Model=Llama-3 8B2026.06 | 65.2 | |
| RTN + absmaxBit-width=4-bit, Model=Llama-3 8B2026.06 | 64.7 | |
| GPTQ + PiSOBit-width=3-bit, Model=Llama-3 8B2026.06 | 63 | |
| Qronos ⊙ PiSOBit-width=3-bit, Model=Llama-3 8B2026.06 | 62.7 | |
| GPTQ ⊙ PiSOBit-width=3-bit, Model=Llama-3 8B2026.06 | 62.6 | |
| BF16Bit-width=3-bit, Model=Llama-3 3B2026.06 | 62 | |
| BF16Bit-width=4-bit, Model=Llama-3 3B2026.06 | 62 | |
| Qronos + PiSOBit-width=3-bit, Model=Llama-3 8B2026.06 | 61.4 | |
| Qronos + PiSOBit-width=4-bit, Model=Llama-3 3B2026.06 | 61.2 | |
| Qronos ⊙ PiSOBit-width=4-bit, Model=Llama-3 3B2026.06 | 60.5 | |
| RTN + PiSOBit-width=4-bit, Model=Llama-3 3B2026.06 | 60.4 | |
| GPTQ + PiSOBit-width=4-bit, Model=Llama-3 3B2026.06 | 60.2 | |
| Qronos + absmaxBit-width=4-bit, Model=Llama-3 3B2026.06 | 60.1 | |
| GPTQ ⊙ PiSOBit-width=4-bit, Model=Llama-3 3B2026.06 | 60 | |
| RTN + PiSOBit-width=3-bit, Model=Llama-3 8B2026.06 | 59.9 | |
| BeaconBit-width=4-bit, Model=Llama-3 8B2026.06 | 59.7 | |
| GPTQ + absmaxBit-width=4-bit, Model=Llama-3 3B2026.06 | 59.4 | |
| Qronos + absmaxBit-width=3-bit, Model=Llama-3 8B2026.06 | 57.9 | |
| GPTQ ⊙ PiSOBit-width=3-bit, Model=Llama-3 3B2026.06 | 55.9 | |
| Qronos ⊙ PiSOBit-width=3-bit, Model=Llama-3 3B2026.06 | 55.6 | |
| RTN + absmaxBit-width=4-bit, Model=Llama-3 3B2026.06 | 55.6 | |
| Qronos + PiSOBit-width=3-bit, Model=Llama-3 3B2026.06 | 55.5 | |
| GPTQ + absmaxBit-width=3-bit, Model=Llama-3 8B2026.06 | 55.1 | |
| GPTQ + PiSOBit-width=3-bit, Model=Llama-3 3B2026.06 | 54.6 | |
| BeaconBit-width=3-bit, Model=Llama-3 8B2026.06 | 53.5 | |
| BF16Bit-width=3-bit, Model=Llama-3 1B2026.06 | 52.6 | |
| BF16Bit-width=4-bit, Model=Llama-3 1B2026.06 | 52.6 | |
| RTN + PiSOBit-width=3-bit, Model=Llama-3 3B2026.06 | 52.1 | |
| GPTQ + PiSOBit-width=4-bit, Model=Llama-3 1B2026.06 | 52 | |
| Qronos + PiSOBit-width=4-bit, Model=Llama-3 1B2026.06 | 51.5 | |
| Qronos + absmaxBit-width=3-bit, Model=Llama-3 3B2026.06 | 51.4 | |
| GPTQ ⊙ PiSOBit-width=4-bit, Model=Llama-3 1B2026.06 | 51.3 | |
| Qronos ⊙ PiSOBit-width=4-bit, Model=Llama-3 1B2026.06 | 51.1 | |
| Qronos + absmaxBit-width=4-bit, Model=Llama-3 1B2026.06 | 50.2 | |
| RTN + PiSOBit-width=4-bit, Model=Llama-3 1B2026.06 | 50 | |
| GPTQ + absmaxBit-width=3-bit, Model=Llama-3 3B2026.06 | 49.8 | |
| GPTQ + absmaxBit-width=4-bit, Model=Llama-3 1B2026.06 | 49.3 | |
| Qronos + PiSOBit-width=3-bit, Model=Llama-3 1B2026.06 | 48.6 | |
| Qronos ⊙ PiSOBit-width=3-bit, Model=Llama-3 1B2026.06 | 48 | |
| GPTQ + PiSOBit-width=3-bit, Model=Llama-3 1B2026.06 | 47.5 | |
| GPTQ ⊙ PiSOBit-width=3-bit, Model=Llama-3 1B2026.06 | 47 | |
| RTN + absmaxBit-width=4-bit, Model=Llama-3 1B2026.06 | 46.3 | |
| BeaconBit-width=4-bit, Model=Llama-3 3B2026.06 | 46.3 | |
| BeaconBit-width=4-bit, Model=Llama-3 1B2026.06 | 46.1 | |
| RTN + PiSOBit-width=3-bit, Model=Llama-3 1B2026.06 | 44.2 | |
| BeaconBit-width=3-bit, Model=Llama-3 1B2026.06 | 42.4 | |
| Qronos + absmaxBit-width=3-bit, Model=Llama-3 1B2026.06 | 40.6 | |
| BeaconBit-width=3-bit, Model=Llama-3 3B2026.06 | 40.1 | |
| GPTQ + absmaxBit-width=3-bit, Model=Llama-3 1B2026.06 | 39.6 | |
| RTN + absmaxBit-width=3-bit, Model=Llama-3 8B2026.06 | 38 | |
| RTN + absmaxBit-width=3-bit, Model=Llama-3 3B2026.06 | 35.7 | |
| RTN + absmaxBit-width=3-bit, Model=Llama-3 1B2026.06 | 32.8 |