Zero-shot Task Evaluation on tasks 0-shot
68.58AccuracyFP16
Evaluation Results
| Method | Links | |
|---|---|---|
| FP16Backbone=3-82026.06 | 68.58 | |
| FP16Backbone=2-132026.06 | 67.81 | |
| FP16Backbone=2-72026.06 | 64.85 | |
| BF16 (dense)Model=LLaMA-3 8B2026.05 | 63.94 | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 60.86 | |
| GPTQModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 60.55 | |
| LiftUQType=LQ-16/10, Bits=1.62, Backbone=2-132026.06 | 60.54 | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 60.52 | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 60.26 | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 60.17 | |
| GPTQModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 59.72 | |
| GPTQModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 59.44 | |
| GPTQModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 59.02 | |
| RTNModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 58.71 | |
| RTNModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 58.55 | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 58.34 | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 57.91 | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 57.59 | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 57.29 | |
| GPTQModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 57.14 | |
| LiftUQType=LQ-16/10, Bits=1.62, Backbone=3-82026.06 | 56.66 | |
| RTNModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 56.61 | |
| GPTQModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 56.51 | |
| GPTQModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 56.31 | |
| RTNModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 56.22 | |
| LiftUQType=LQ-16/10, Bits=1.62, Backbone=2-72026.06 | 56.19 | |
| GPTQModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 56.08 | |
| BF16 (dense)Model=LLaMA-3.2 1B-Instruct2026.05 | 53.22 | |
| ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 49.72 | |
| ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 49.38 | |
| ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 49.36 | |
| ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 49.31 | |
| PTQ1.61Type=UQ, Bits=1.61, Backbone=2-132026.06 | 49.21 | |
| GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 49.01 | |
| GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 48.77 | |
| GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 48.52 | |
| GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 48.27 | |
| RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 47.74 | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 47.55 | |
| ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 47.26 | |
| RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 47.18 | |
| RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 46.9 | |
| ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 46.84 | |
| RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 46.75 | |
| ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 46.7 | |
| ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 46.15 | |
| GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 45.88 | |
| GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 45.78 | |
| GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 45.76 | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 45.63 | |
| GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 45.62 | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 44.79 | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 44.64 | |
| PTQ1.61Type=UQ, Bits=1.61, Backbone=2-72026.06 | 44.14 | |
| PTQ1.61Type=UQ, Bits=1.61, Backbone=3-82026.06 | 43.99 | |
| GPTQModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 42.7 | |
| GPTQModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 42.44 | |
| RTNModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 40.85 | |
| GPTQModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 40.47 | |
| GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 40.05 | |
| ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 39.86 | |
| GPTQModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 39.52 | |
| ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 39.26 | |
| ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 39.09 | |
| RTNModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 38.81 | |
| ADMM-QModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 38.67 | |
| RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 38.61 | |
| GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 38.2 | |
| RTNModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 38.01 | |
| GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 37.74 | |
| RTNModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 37.69 | |
| RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 37.54 | |
| RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 37.14 | |
| RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 36.99 | |
| RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 36.94 | |
| GPTQModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 36.85 | |
| RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 36.67 | |
| RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 36.05 | |
| RTNModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 35.48 | |
| RTNModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 35.47 | |
| RTNModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 34.73 | |
| RTNModel=LLaMA-3.2 1B-Instruct, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 34.58 | |
| RTNModel=LLaMA-3 8B, Bit-width=W2A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 34.53 |