Zero-shot Evaluation on Zero-shot Tasks Average
68.05AccuracyFP16
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FP16Bits=16-16, Backbone=LLaMA-3 8B, Evaluation Protocol=0-shot2026.04 | 68.05 | — | |
| FP16Bits=16-16, Backbone=LLaMA-2 13B, Evaluation Protocol=0-shot2026.04 | 67.77 | — | |
| ReSpinQuantBits=W4A4, Backbone=LLaMA-2 13B, Evaluation Protocol=0-shot2026.04 | 65.78 | — | |
| SpinQuantBits=W4A4, Backbone=LLaMA-2 13B, Evaluation Protocol=0-shot2026.04 | 65.58 | — | |
| OSTQuantBits=W4A4, Backbone=LLaMA-2 13B, Evaluation Protocol=0-shot2026.04 | 65.41 | — | |
| QuaRotBits=W4A4, Backbone=LLaMA-2 13B, Evaluation Protocol=0-shot2026.04 | 65.25 | — | |
| FP16Bits=16-16, Backbone=LLaMA-2 7B, Evaluation Protocol=0-shot2026.04 | 65.08 | — | |
| FlatQuantBits=W4A4, Backbone=LLaMA-2 13B, Evaluation Protocol=0-shot2026.04 | 65.04 | — | |
| ReSpinQuantBits=W4A4, Backbone=LLaMA-3 8B, Evaluation Protocol=0-shot2026.04 | 64.65 | — | |
| SpinQuantBits=W4A4, Backbone=LLaMA-3 8B, Evaluation Protocol=0-shot2026.04 | 64.53 | — | |
| OSTQuantBits=W4A4, Backbone=LLaMA-3 8B, Evaluation Protocol=0-shot2026.04 | 64.35 | — | |
| FP16Bits=16-16, Backbone=LLaMA-3.2 3B, Evaluation Protocol=0-shot2026.04 | 63.44 | — | |
| QuaRotBits=W4A4, Backbone=LLaMA-3 8B, Evaluation Protocol=0-shot2026.04 | 62.9 | — | |
| FlatQuantBits=W4A4, Backbone=LLaMA-3 8B, Evaluation Protocol=0-shot2026.04 | 62.72 | — | |
| ReSpinQuantBits=W4A4, Backbone=LLaMA-2 7B, Evaluation Protocol=0-shot2026.04 | 62.49 | — | |
| OSTQuantBits=W4A4, Backbone=LLaMA-2 7B, Evaluation Protocol=0-shot2026.04 | 62.32 | — | |
| QuaRotBits=W4A4, Backbone=LLaMA-2 7B, Evaluation Protocol=0-shot2026.04 | 62.03 | — | |
| FlatQuantBits=W4A4, Backbone=LLaMA-2 7B, Evaluation Protocol=0-shot2026.04 | 61.97 | — | |
| SpinQuantBits=W4A4, Backbone=LLaMA-2 7B, Evaluation Protocol=0-shot2026.04 | 61.6 | — | |
| ReSpinQuantBits=W4A4, Backbone=LLaMA-3.2 3B, Evaluation Protocol=0-shot2026.04 | 58.84 | — | |
| OSTQuantBits=W4A4, Backbone=LLaMA-3.2 3B, Evaluation Protocol=0-shot2026.04 | 58.67 | — | |
| FlatQuantBits=W4A4, Backbone=LLaMA-3.2 3B, Evaluation Protocol=0-shot2026.04 | 58.06 | — | |
| QuaRotBits=W4A4, Backbone=LLaMA-3.2 3B, Evaluation Protocol=0-shot2026.04 | 57.22 | — | |
| MDBFModel=Llama-3-8B, BPW=1.502026.03 | 57.2 | — | |
| SpinQuantBits=W4A4, Backbone=LLaMA-3.2 3B, Evaluation Protocol=0-shot2026.04 | 56.78 | — | |
| ReSpinQuantBits=W3A3, Backbone=LLaMA-2 13B, Evaluation Protocol=0-shot2026.04 | 56.58 | — | |
| MDBFModel=Llama-2-7B, BPW=1.502026.03 | 56.4 | — | |
| OSTQuantBits=W3A3, Backbone=LLaMA-2 13B, Evaluation Protocol=0-shot2026.04 | 56.36 | — | |
| DBFModel=Llama-3-8B, BPW=1.502026.03 | 56 | — | |
| FP16Bits=16-16, Backbone=LLaMA-3.2 1B, Evaluation Protocol=0-shot2026.04 | 55.75 | — | |
| SpinQuantBits=W3A3, Backbone=LLaMA-2 13B, Evaluation Protocol=0-shot2026.04 | 55.1 | — | |
| DBFModel=Llama-2-7B, BPW=1.502026.03 | 54.8 | — | |
| MDBFModel=Llama-3-8B, BPW=1.002026.03 | 51.8 | — | |
| FlatQuantBits=W3A3, Backbone=LLaMA-2 13B, Evaluation Protocol=0-shot2026.04 | 50.95 | — | |
| ReSpinQuantBits=W3A3, Backbone=LLaMA-3 8B, Evaluation Protocol=0-shot2026.04 | 50.74 | — | |
| DBFModel=Llama-3-8B, BPW=1.002026.03 | 50.5 | — | |
| ReSpinQuantBits=W3A3, Backbone=LLaMA-2 7B, Evaluation Protocol=0-shot2026.04 | 50.14 | — | |
| ReSpinQuantBits=W4A4, Backbone=LLaMA-3.2 1B, Evaluation Protocol=0-shot2026.04 | 49.57 | — | |
| MDBFModel=Llama-3.2-1B, BPW=1.502026.03 | 49.2 | — | |
| SpinQuantBits=W3A3, Backbone=LLaMA-2 7B, Evaluation Protocol=0-shot2026.04 | 49.15 | — | |
| DBFModel=Llama-3.2-1B, BPW=1.502026.03 | 49.1 | — | |
| DBFModel=TinyLlama-1.1B, BPW=1.502026.03 | 49.1 | — | |
| MDBFModel=TinyLlama-1.1B, BPW=1.502026.03 | 49 | — | |
| MDBFModel=Llama-2-7B, BPW=1.002026.03 | 48.9 | — | |
| OSTQuantBits=W4A4, Backbone=LLaMA-3.2 1B, Evaluation Protocol=0-shot2026.04 | 48.8 | — | |
| SpinQuantBits=W3A3, Backbone=LLaMA-3 8B, Evaluation Protocol=0-shot2026.04 | 48.77 | — | |
| OSTQuantBits=W3A3, Backbone=LLaMA-2 7B, Evaluation Protocol=0-shot2026.04 | 48.44 | — | |
| SpinQuantBits=W4A4, Backbone=LLaMA-3.2 1B, Evaluation Protocol=0-shot2026.04 | 48.35 | — | |
| FlatQuantBits=W4A4, Backbone=LLaMA-3.2 1B, Evaluation Protocol=0-shot2026.04 | 48.2 | — | |
| DBFModel=Llama-2-7B, BPW=1.002026.03 | 48 | — | |
| OSTQuantBits=W3A3, Backbone=LLaMA-3 8B, Evaluation Protocol=0-shot2026.04 | 47.34 | — | |
| MDBFModel=Llama-3.2-1B, BPW=1.002026.03 | 47.3 | — | |
| QuaRotBits=W4A4, Backbone=LLaMA-3.2 1B, Evaluation Protocol=0-shot2026.04 | 46.81 | — | |
| DBFModel=Llama-3.2-1B, BPW=1.002026.03 | 46.5 | — | |
| FlatQuantBits=W3A3, Backbone=LLaMA-2 7B, Evaluation Protocol=0-shot2026.04 | 45.43 | — | |
| DBFModel=TinyLlama-1.1B, BPW=1.002026.03 | 44.9 | — | |
| MDBFModel=TinyLlama-1.1B, BPW=1.002026.03 | 44.9 | — | |
| MDBFModel=Qwen3-0.6B, BPW=1.502026.03 | 44.6 | — | |
| QuaRotBits=W3A3, Backbone=LLaMA-2 13B, Evaluation Protocol=0-shot2026.04 | 43.67 | — | |
| ReSpinQuantBits=W3A3, Backbone=LLaMA-3.2 3B, Evaluation Protocol=0-shot2026.04 | 43.63 | — | |
| DBFModel=Qwen3-0.6B, BPW=1.502026.03 | 43.5 | — | |
| MDBFModel=Qwen3-0.6B, BPW=1.002026.03 | 42.6 | — | |
| SpinQuantBits=W3A3, Backbone=LLaMA-3.2 3B, Evaluation Protocol=0-shot2026.04 | 42.16 | — | |
| DBFModel=Qwen3-0.6B, BPW=1.002026.03 | 41.2 | — | |
| OSTQuantBits=W3A3, Backbone=LLaMA-3.2 3B, Evaluation Protocol=0-shot2026.04 | 41.16 | — | |
| QuaRotBits=W3A3, Backbone=LLaMA-2 7B, Evaluation Protocol=0-shot2026.04 | 40.94 | — | |
| RTNBits=W4A4, Backbone=LLaMA-3 8B, Evaluation Protocol=0-shot2026.04 | 36.74 | — | |
| ReSpinQuantBits=W3A3, Backbone=LLaMA-3.2 1B, Evaluation Protocol=0-shot2026.04 | 36.65 | — | |
| GPTQBits=W4A4, Backbone=LLaMA-3.2 3B, Evaluation Protocol=0-shot2026.04 | 36.51 | — | |
| GPTQBits=W4A4, Backbone=LLaMA-3 8B, Evaluation Protocol=0-shot2026.04 | 35.74 | — | |
| OSTQuantBits=W3A3, Backbone=LLaMA-3.2 1B, Evaluation Protocol=0-shot2026.04 | 35.71 | — | |
| RTNBits=W4A4, Backbone=LLaMA-3.2 3B, Evaluation Protocol=0-shot2026.04 | 35.15 | — | |
| SpinQuantBits=W3A3, Backbone=LLaMA-3.2 1B, Evaluation Protocol=0-shot2026.04 | 35.02 | — | |
| GPTQBits=W4A4, Backbone=LLaMA-2 7B, Evaluation Protocol=0-shot2026.04 | 34.89 | — | |
| FlatQuantBits=W3A3, Backbone=LLaMA-3.2 3B, Evaluation Protocol=0-shot2026.04 | 34.13 | — | |
| QuaRotBits=W3A3, Backbone=LLaMA-3 8B, Evaluation Protocol=0-shot2026.04 | 33.95 | — | |
| GPTQBits=W4A4, Backbone=LLaMA-3.2 1B, Evaluation Protocol=0-shot2026.04 | 33.81 | — | |
| FlatQuantBits=W3A3, Backbone=LLaMA-3 8B, Evaluation Protocol=0-shot2026.04 | 33.81 | — | |
| GPTQBits=W3A3, Backbone=LLaMA-2 7B, Evaluation Protocol=0-shot2026.04 | 33.78 | — | |
| RTNBits=W4A4, Backbone=LLaMA-2 7B, Evaluation Protocol=0-shot2026.04 | 33.69 | — | |
| RTNBits=W4A4, Backbone=LLaMA-3.2 1B, Evaluation Protocol=0-shot2026.04 | 33.52 | — | |
| QuaRotBits=W3A3, Backbone=LLaMA-3.2 3B, Evaluation Protocol=0-shot2026.04 | 32.94 | — | |
| RTNBits=W3A3, Backbone=LLaMA-3 8B, Evaluation Protocol=0-shot2026.04 | 32.71 | — | |
| RTNBits=W3A3, Backbone=LLaMA-2 13B, Evaluation Protocol=0-shot2026.04 | 32.59 | — | |
| RTNBits=W3A3, Backbone=LLaMA-2 7B, Evaluation Protocol=0-shot2026.04 | 32.56 | — | |
| FlatQuantBits=W3A3, Backbone=LLaMA-3.2 1B, Evaluation Protocol=0-shot2026.04 | 32.41 | — | |
| GPTQBits=W3A3, Backbone=LLaMA-3 8B, Evaluation Protocol=0-shot2026.04 | 32.31 | — | |
| QuaRotBits=W3A3, Backbone=LLaMA-3.2 1B, Evaluation Protocol=0-shot2026.04 | 32.07 | — | |
| GPTQBits=W3A3, Backbone=LLaMA-2 13B, Evaluation Protocol=0-shot2026.04 | 31.84 | — | |
| RTNBits=W3A3, Backbone=LLaMA-3.2 1B, Evaluation Protocol=0-shot2026.04 | 31.62 | — | |
| GPTQBits=W3A3, Backbone=LLaMA-3.2 3B, Evaluation Protocol=0-shot2026.04 | 31.48 | — | |
| GPTQBits=W4A4, Backbone=LLaMA-2 13B, Evaluation Protocol=0-shot2026.04 | 31.46 | — | |
| RTNBits=W3A3, Backbone=LLaMA-3.2 3B, Evaluation Protocol=0-shot2026.04 | 31.28 | — | |
| RTNBits=W4A4, Backbone=LLaMA-2 13B, Evaluation Protocol=0-shot2026.04 | 31.24 | — | |
| GPTQBits=W3A3, Backbone=LLaMA-3.2 1B, Evaluation Protocol=0-shot2026.04 | 31.13 | — | |
| BF16Algorithm=BF16, Bits=16, Backbone=Llama 3.1 70B Instruct2025.05 | — | 67.67 | |
| BF16Algorithm=BF16, Bits=16, Backbone=Llama 3.2 3B Instruct2025.05 | — | 63.79 | |
| BF16Algorithm=BF16, Bits=16, Backbone=Llama 3.1 8B Instruct2025.05 | — | 69.82 | |
| BF16Algorithm=BF16, Bits=16, Backbone=Llama 3.2 1B Instruct2025.05 | — | 54.79 | |
| BF16Model=Llama 3.1-8B, Precision=Full2026.06 | — | 68.6 |