Zero-shot Evaluation on PIQA, WinoGrande, HellaSwag, ARC (Easy and Challenge), and LAMBADA (test)
77.07Average AccuracyFP16
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| FP16Model=LLAMA2-70B, Quantization=FP16, Evaluation Protocol=Zero-shot2024.03 | 77.07 | 82.7 | 77.98 | 83.84 | 80.98 | 57.34 | 79.58 | |
| QuaRotModel=LLAMA2-70B, Quantization=4-bit (A4W4KV4), Evaluation Protocol=Zero-shot2024.03 | 75.98 | 82.43 | 76.24 | 81.82 | 80.43 | 56.23 | 78.73 | |
| 16bitsAvg. bits=16, Group Size=-, Backbone=Llama3-70B2025.12 | 75.28 | — | — | — | — | — | — | |
| QuaRot-GPTQModel=Phi-3-mini, Precision=INT82024.03 | 73.18 | 80.25 | 74.19 | 78.54 | 80.35 | 57.08 | 68.64 | |
| BaselineModel=Phi-3-mini, Precision=FP162024.03 | 73.11 | 80.47 | 73.72 | 78.45 | 80.13 | 57.51 | 68.37 | |
| QuaRot-RTNModel=Phi-3-mini, Precision=INT82024.03 | 73.02 | 79.71 | 74.11 | 78.63 | 80.47 | 56.66 | 68.56 | |
| 16bitsAvg. bits=16, Group Size=-, Backbone=Llama2-70B2025.12 | 72.71 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2.5, Group Size=128, Backbone=Llama3-70B2025.12 | 72.68 | — | — | — | — | — | — | |
| QuaRot-GPTQModel=Phi-3-mini, Precision=INT62024.03 | 72.67 | 79.54 | 72.69 | 78.5 | 79.42 | 56.74 | 68.85 | |
| FP16Model=LLAMA2-13B, Quantization=FP16, Evaluation Protocol=Zero-shot2024.03 | 72.59 | 80.47 | 72.22 | 79.39 | 77.48 | 49.23 | 76.75 | |
| SignRoundV2Avg. bits=2.5, Group Size=128, Backbone=Llama3-70B2025.12 | 72.37 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2.5, Group Size=64, Backbone=Llama3-70B2025.12 | 72.32 | — | — | — | — | — | — | |
| QuaRot-RTNModel=Phi-3-mini, Precision=INT62024.03 | 71.98 | 79.54 | 73.01 | 77.46 | 79.21 | 55.12 | 67.53 | |
| SignRoundV2Avg. bits=2.5, Group Size=64, Backbone=Llama3-70B2025.12 | 71.97 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2, Group Size=64, Backbone=Llama3-70B2025.12 | 70.93 | — | — | — | — | — | — | |
| QuIP#Avg. bits=2, Group Size=-, Backbone=Llama2-70B2025.12 | 70.91 | — | — | — | — | — | — | |
| AQLMAvg. bits=2, Group Size=1x16, Backbone=Llama2-70B2025.12 | 70.84 | — | — | — | — | — | — | |
| SignRoundV2Avg. bits=2.5, Group Size=128, Backbone=Llama2-70B2025.12 | 70.6 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2.5, Group Size=64, Backbone=Llama2-70B2025.12 | 70.6 | — | — | — | — | — | — | |
| SignRoundV2Avg. bits=2.5, Group Size=64, Backbone=Llama2-70B2025.12 | 70.25 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2, Group Size=128, Backbone=Llama3-70B2025.12 | 70.16 | — | — | — | — | — | — | |
| AQLMAvg. bits=2, Group Size=1x16, Backbone=Llama3-70B2025.12 | 70.1 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2.5, Group Size=128, Backbone=Llama2-70B2025.12 | 70.04 | — | — | — | — | — | — | |
| AQLMAvg. bits=2, Group Size=2x8, Backbone=Llama2-70B2025.12 | 69.85 | — | — | — | — | — | — | |
| FP16Model=LLAMA2-7B, Quantization=FP16, Evaluation Protocol=Zero-shot2024.03 | 69.82 | 79.11 | 69.06 | 75.99 | 74.58 | 46.25 | 73.9 | |
| QuaRotModel=LLAMA2-13B, Quantization=4-bit (A4W4KV4), Evaluation Protocol=Zero-shot2024.03 | 69.79 | 78.89 | 70.24 | 76.37 | 72.98 | 46.59 | 73.67 | |
| EfficientQATAvg. bits=2, Group Size=64, Backbone=Llama2-70B2025.12 | 69.48 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2, Group Size=64, Backbone=Llama2-70B2025.12 | 69.3 | — | — | — | — | — | — | |
| SignRoundV2Avg. bits=2, Group Size=128, Backbone=Llama3-70B2025.12 | 69.02 | — | — | — | — | — | — | |
| EfficientQATAvg. bits=2, Group Size=128, Backbone=Llama2-70B2025.12 | 68.93 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2, Group Size=128, Backbone=Llama2-70B2025.12 | 68.82 | — | — | — | — | — | — | |
| 16bitsAvg. bits=16, Group Size=-, Backbone=Llama3-8B2025.12 | 68.64 | — | — | — | — | — | — | |
| SignRoundV2Avg. bits=2, Group Size=128, Backbone=Llama2-70B2025.12 | 68.39 | — | — | — | — | — | — | |
| EfficientQATAvg. bits=2, Group Size=64, Backbone=Llama3-70B2025.12 | 67.89 | — | — | — | — | — | — | |
| SignRoundV1 (SRV1)Avg. bits=2, Group Size=128, Backbone=Llama2-70B2025.12 | 67.7 | — | — | — | — | — | — | |
| EfficientQATAvg. bits=2, Group Size=128, Backbone=Llama3-70B2025.12 | 67.57 | — | — | — | — | — | — | |
| 16bitsAvg. bits=16, Group Size=-, Backbone=Llama2-13B2025.12 | 67.44 | — | — | — | — | — | — | |
| QuaRot-GPTQModel=Phi-3-mini, Precision=INT42024.03 | 66.34 | 75.35 | 67.88 | 72.95 | 72.98 | 48.12 | 60.78 | |
| QuaRotModel=LLAMA2-7B, Quantization=4-bit (A4W4KV4), Evaluation Protocol=Zero-shot2024.03 | 65.64 | 76.77 | 63.77 | 72.16 | 69.87 | 40.87 | 70.39 | |
| AQLMAvg. bits=2, Group Size=1x16, Backbone=Llama2-13B2025.12 | 64.95 | — | — | — | — | — | — | |
| SignRoundV1 (SRV1)Avg. bits=2, Group Size=128, Backbone=Llama3-70B2025.12 | 64.76 | — | — | — | — | — | — | |
| 16bitsAvg. bits=16, Group Size=-, Backbone=Llama2-7B2025.12 | 64.66 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2.5, Group Size=64, Backbone=Llama2-13B2025.12 | 64.57 | — | — | — | — | — | — | |
| EfficientQATAvg. bits=2, Group Size=64, Backbone=Llama2-13B2025.12 | 64.48 | — | — | — | — | — | — | |
| QuIP#Avg. bits=2, Group Size=-, Backbone=Llama2-13B2025.12 | 64.44 | — | — | — | — | — | — | |
| AQLMAvg. bits=2, Group Size=1x16, Backbone=Llama3-8B2025.12 | 64.1 | — | — | — | — | — | — | |
| SignRoundV2Avg. bits=2.5, Group Size=64, Backbone=Llama2-13B2025.12 | 64.09 | — | — | — | — | — | — | |
| EfficientQATAvg. bits=2, Group Size=128, Backbone=Llama2-13B2025.12 | 63.88 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2.5, Group Size=128, Backbone=Llama2-13B2025.12 | 63.73 | — | — | — | — | — | — | |
| SignRoundV2Avg. bits=2.5, Group Size=64, Backbone=Llama3-8B2025.12 | 63.67 | — | — | — | — | — | — | |
| SignRoundV2Avg. bits=2.5, Group Size=128, Backbone=Llama2-13B2025.12 | 63.65 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2.5, Group Size=64, Backbone=Llama3-8B2025.12 | 63.56 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2, Group Size=64, Backbone=Llama2-13B2025.12 | 62.81 | — | — | — | — | — | — | |
| SignRoundV2Avg. bits=2.5, Group Size=128, Backbone=Llama3-8B2025.12 | 62.76 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2.5, Group Size=128, Backbone=Llama3-8B2025.12 | 62.6 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2, Group Size=128, Backbone=Llama2-13B2025.12 | 62.34 | — | — | — | — | — | — | |
| AQLMAvg. bits=2, Group Size=2x8, Backbone=Llama2-13B2025.12 | 62.22 | — | — | — | — | — | — | |
| SignRoundV2Avg. bits=2, Group Size=128, Backbone=Llama2-13B2025.12 | 61.88 | — | — | — | — | — | — | |
| AQLMAvg. bits=2, Group Size=1x16, Backbone=Llama2-7B2025.12 | 61.85 | — | — | — | — | — | — | |
| EfficientQATAvg. bits=2, Group Size=64, Backbone=Llama3-8B2025.12 | 60.76 | — | — | — | — | — | — | |
| SignRoundV1 (SRV1)Avg. bits=2, Group Size=128, Backbone=Llama2-13B2025.12 | 60.72 | — | — | — | — | — | — | |
| QuIP#Avg. bits=2, Group Size=-, Backbone=Llama2-7B2025.12 | 60.61 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2.5, Group Size=128, Backbone=Llama2-7B2025.12 | 60.44 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2, Group Size=64, Backbone=Llama3-8B2025.12 | 60.38 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2.5, Group Size=64, Backbone=Llama2-7B2025.12 | 60.28 | — | — | — | — | — | — | |
| EfficientQATAvg. bits=2, Group Size=64, Backbone=Llama2-7B2025.12 | 60.14 | — | — | — | — | — | — | |
| SignRoundV2Avg. bits=2.5, Group Size=64, Backbone=Llama2-7B2025.12 | 60.08 | — | — | — | — | — | — | |
| OmniQAvg. bits=2, Group Size=128, Backbone=Llama3-70B2025.12 | 60.06 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2, Group Size=128, Backbone=Llama3-8B2025.12 | 59.97 | — | — | — | — | — | — | |
| SignRoundV2Avg. bits=2.5, Group Size=128, Backbone=Llama2-7B2025.12 | 59.96 | — | — | — | — | — | — | |
| EfficientQATAvg. bits=2, Group Size=128, Backbone=Llama2-7B2025.12 | 59.5 | — | — | — | — | — | — | |
| EfficientQATAvg. bits=2, Group Size=128, Backbone=Llama3-8B2025.12 | 59.37 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2, Group Size=64, Backbone=Llama2-7B2025.12 | 59.04 | — | — | — | — | — | — | |
| SignRoundV2Avg. bits=2, Group Size=128, Backbone=Llama3-8B2025.12 | 59.02 | — | — | — | — | — | — | |
| SignRoundV2 (Improved Recipe)Avg. bits=2, Group Size=128, Backbone=Llama2-7B2025.12 | 58.67 | — | — | — | — | — | — | |
| SignRoundV2Avg. bits=2, Group Size=128, Backbone=Llama2-7B2025.12 | 57.88 | — | — | — | — | — | — | |
| AQLMAvg. bits=2, Group Size=2x8, Backbone=Llama2-7B2025.12 | 57.61 | — | — | — | — | — | — | |
| QuaRot-RTNModel=Phi-3-mini, Precision=INT42024.03 | 56.12 | 68.39 | 58.64 | 60.6 | 65.87 | 39.25 | 43.99 | |
| SignRoundV1 (SRV1)Avg. bits=2, Group Size=128, Backbone=Llama3-8B2025.12 | 55.25 | — | — | — | — | — | — | |
| OmniQAvg. bits=2, Group Size=128, Backbone=Llama2-70B2025.12 | 54.87 | — | — | — | — | — | — | |
| DenseScale=1.3B, FLOPs=1.00, KV mem=1.002026.07 | 54.8 | — | — | — | — | — | — | |
| SignRoundV1 (SRV1)Avg. bits=2, Group Size=128, Backbone=Llama2-7B2025.12 | 54.5 | — | — | — | — | — | — | |
| TriRouteScale=1.3B, FLOPs=0.55, KV mem=0.402026.07 | 54.5 | — | — | — | — | — | — | |
| OmniQAvg. bits=2, Group Size=128, Backbone=Llama2-13B2025.12 | 53.56 | — | — | — | — | — | — | |
| Independent comboScale=1.3B, FLOPs=0.55, KV mem=0.402026.07 | 53.5 | — | — | — | — | — | — | |
| OmniQAvg. bits=2, Group Size=128, Backbone=Llama3-8B2025.12 | 52.66 | — | — | — | — | — | — | |
| MoD + KV-quantScale=1.3B, FLOPs=0.55, KV mem=0.402026.07 | 52.5 | — | — | — | — | — | — | |
| GPTQAvg. bits=2, Group Size=128, Backbone=Llama2-13B2025.12 | 48.29 | — | — | — | — | — | — | |
| DenseScale=410M, FLOPs=1.00, KV mem=1.002026.07 | 47.2 | — | — | — | — | — | — | |
| TriRouteScale=410M, FLOPs=0.55, KV mem=0.402026.07 | 47 | — | — | — | — | — | — | |
| OmniQAvg. bits=2, Group Size=128, Backbone=Llama2-7B2025.12 | 46.98 | — | — | — | — | — | — | |
| Independent comboScale=410M, FLOPs=0.55, KV mem=0.402026.07 | 46.2 | — | — | — | — | — | — | |
| MoD + KV-quantScale=410M, FLOPs=0.55, KV mem=0.402026.07 | 45.6 | — | — | — | — | — | — | |
| DenseScale=160M, FLOPs=1.00, KV mem=1.002026.07 | 42 | — | — | — | — | — | — | |
| TriRouteScale=160M, FLOPs=0.55, KV mem=0.402026.07 | 41.8 | — | — | — | — | — | — | |
| GPTQAvg. bits=2, Group Size=128, Backbone=Llama2-7B2025.12 | 41.56 | — | — | — | — | — | — | |
| Independent comboScale=160M, FLOPs=0.55, KV mem=0.402026.07 | 40.9 | — | — | — | — | — | — | |
| MoD + KV-quantScale=160M, FLOPs=0.55, KV mem=0.402026.07 | 40.3 | — | — | — | — | — | — | |
| AWQAvg. bits=2, Group Size=128, Backbone=Llama2-13B2025.12 | 35.99 | — | — | — | — | — | — | |
| AWQAvg. bits=2, Group Size=128, Backbone=Llama2-70B2025.12 | 35.49 | — | — | — | — | — | — |