Zero-shot Evaluation on Evaluation Tasks Zero-shot Aggregate
79.95Avg. AccuracyBaseline
Evaluation Results
| Method | Links | |
|---|---|---|
| BaselineBackbone=LLaMA3-70B, Precision=FP162026.05 | 79.95 | |
| TORQBackbone=LLaMA3-70B2026.05 | 76.28 | |
| FP16Model=Phi3-3.8B, Quantization Bits=None2026.02 | 75.41 | |
| BaselineBackbone=Qwen3-32B, Precision=FP162026.05 | 74.82 | |
| FP16Model=Qwen3-8B, Quantization Bits=None2026.02 | 74.15 | |
| OSTQuantBackbone=LLaMA3-70B2026.05 | 73.93 | |
| BaselineBackbone=Qwen3-14B, Precision=FP162026.05 | 73.81 | |
| TORQBackbone=Qwen3-32B2026.05 | 73.63 | |
| GlowQ-SZero-shot=true, Model=Vicuna-13B2026.03 | 73.24 | |
| BaselineBackbone=LLaMA3-8B, Precision=FP162026.05 | 73.18 | |
| QERAZero-shot=true, Model=Vicuna-13B2026.03 | 71.9 | |
| GlowQZero-shot=true, Model=Vicuna-13B2026.03 | 71.86 | |
| FP16Rank=-, Zero-shot=true, Model=Vicuna-13B2026.03 | 71.76 | |
| ZeroQuant-V2Zero-shot=true, Model=Vicuna-13B2026.03 | 71.67 | |
| L2QERRank=64, Zero-shot=true, Model=Vicuna-13B2026.03 | 71.67 | |
| SNRQ-SModel=L2-13B, Quantization Time (s)=786.42026.02 | 71.11 | |
| SNRQ-CModel=L2-13B, Quantization Time (s)=965.02026.02 | 71.06 | |
| GPTAQModel=L2-13B, Quantization Time (s)=1012.02026.02 | 71 | |
| SNRQ-S+RHTModel=L2-13B, Quantization Time (s)=970.22026.02 | 70.9 | |
| GPTAQ+RHTModel=L2-13B, Quantization Time (s)=1259.42026.02 | 70.86 | |
| QuIPModel=L2-13B, Quantization Time (s)=1096.22026.02 | 70.76 | |
| SNRQ-C+RHTModel=L2-13B, Quantization Time (s)=1106.32026.02 | 70.71 | |
| SNRQ-SModel=Qwen3-8B, Quantization Bits=3-bit, Quantization Time (s)=478.42026.02 | 70.56 | |
| TORQBackbone=Qwen3-14B2026.05 | 70.51 | |
| BaselineBackbone=Qwen3-8B, Precision=FP162026.05 | 70.43 | |
| SNRQ-CModel=Qwen3-8B, Quantization Bits=3-bit, Quantization Time (s)=480.22026.02 | 70.05 | |
| FlatQuantBackbone=Qwen3-14B2026.05 | 69.87 | |
| SNRQ-S+RHTModel=L3-8B, Quantization Time (s)=688.72026.02 | 69.78 | |
| SNRQ-SModel=L3-8B, Quantization Time (s)=542.82026.02 | 69.33 | |
| TORQBackbone=LLaMA3-8B2026.05 | 69.32 | |
| LDLQModel=Qwen3-8B, Quantization Bits=3-bit, Quantization Time (s)=419.52026.02 | 69.26 | |
| SNRQ-C+RHTModel=L3-8B, Quantization Time (s)=741.82026.02 | 69.01 | |
| SNRQ-SModel=Phi3-3.8B, Quantization Bits=3-bit, Quantization Time (s)=267.72026.02 | 69 | |
| GPTAQModel=Qwen3-8B, Quantization Bits=3-bit, Quantization Time (s)=670.42026.02 | 68.94 | |
| GPTAQModel=Phi3-3.8B, Quantization Bits=3-bit, Quantization Time (s)=3322026.02 | 68.74 | |
| GPTAQ+RHTModel=L3-8B, Quantization Time (s)=853.12026.02 | 68.73 | |
| GPTQModel=Qwen3-8B, Quantization Bits=3-bit, Quantization Time (s)=494.92026.02 | 68.49 | |
| SNRQ-CModel=Phi3-3.8B, Quantization Bits=3-bit, Quantization Time (s)=271.42026.02 | 68.47 | |
| FlatQuantBackbone=LLaMA3-8B2026.05 | 68.34 | |
| GPTQModel=Phi3-3.8B, Quantization Bits=3-bit, Quantization Time (s)=262.72026.02 | 67.84 | |
| BRQBackbone=LLaMA3-8B2026.05 | 67.76 | |
| TORQBackbone=Qwen3-8B2026.05 | 67.75 | |
| QuIPModel=L3-8B, Quantization Time (s)=763.82026.02 | 67.6 | |
| SNRQ-CModel=L3-8B, Quantization Time (s)=594.42026.02 | 67.6 | |
| SNRQ-SModel=L2-7B, Quantization Time (s)=449.02026.02 | 67.34 | |
| SNRQ-C+RHTModel=L2-7B, Quantization Time (s)=697.32026.02 | 67.26 | |
| OSTQuantBackbone=Qwen3-32B2026.05 | 67.19 | |
| SNRQ-S+RHTModel=L2-7B, Quantization Time (s)=644.52026.02 | 67.18 | |
| GPTAQ+RHTModel=L2-7B, Quantization Time (s)=795.72026.02 | 67.02 | |
| LDLQModel=Phi3-3.8B, Quantization Bits=3-bit, Quantization Time (s)=231.92026.02 | 67.01 | |
| FlatQuantBackbone=Qwen3-8B2026.05 | 66.96 | |
| SNRQ-CModel=L2-7B, Quantization Time (s)=517.12026.02 | 66.82 | |
| GPTAQModel=L2-7B, Quantization Time (s)=590.42026.02 | 66.34 | |
| QuIPModel=L2-7B, Quantization Time (s)=704.82026.02 | 66.22 | |
| OSTQuantBackbone=LLaMA3-8B2026.05 | 65.92 | |
| GPTAQModel=L3-8B, Quantization Time (s)=715.02026.02 | 65.77 | |
| BRQBackbone=Qwen3-8B2026.05 | 64.73 | |
| OSTQuantBackbone=Qwen3-14B2026.05 | 64.06 | |
| QuaRotBackbone=LLaMA3-70B2026.05 | 62.87 | |
| QuaRotBackbone=LLaMA3-8B2026.05 | 62.13 | |
| QuaRotBackbone=Qwen3-32B2026.05 | 61.23 | |
| QuaRotBackbone=Qwen3-14B2026.05 | 60.21 | |
| GPTQBackbone=Qwen3-32B2026.05 | 57.62 | |
| QuaRotBackbone=Qwen3-8B2026.05 | 55.35 | |
| OSTQuantBackbone=Qwen3-8B2026.05 | 54.4 | |
| GPTQBackbone=Qwen3-14B2026.05 | 39.91 | |
| RTNBackbone=Qwen3-32B2026.05 | 38.4 | |
| GPTQBackbone=Qwen3-8B2026.05 | 35.54 | |
| RTNBackbone=Qwen3-14B2026.05 | 35.22 | |
| GPTQBackbone=LLaMA3-70B2026.05 | 34.82 | |
| GPTQBackbone=LLaMA3-8B2026.05 | 33.6 | |
| RTNBackbone=LLaMA3-8B2026.05 | 32.03 | |
| RTNBackbone=LLaMA3-70B2026.05 | 30.92 | |
| RTNBackbone=Qwen3-8B2026.05 | 30.73 |