Zero-shot Evaluation on Eight datasets average
63.54AccuracySurrogate-Assisted Layer Contribution Estimation
Evaluation results
| Method | Links | |
|---|---|---|
| Surrogate-Assisted Layer Contribution EstimationBase Model=Meta-LLaMA-3-8B, Parameter Size=7.4B, Evaluation Protocol=Zero-shot2026.02 | 63.54 | |
| Shortened-LLaMABase Model=Meta-LLaMA-3-8B, Parameter Size=7.4B, Evaluation Protocol=Zero-shot2026.02 | 62.99 | |
| GPTAQ + MaCaBackbone=Qwen3-8B-IT, Bit-width=4bit2026.02 | 61.89 | |
| GPTAQ + MaCaBackbone=Gemma3-12B-IT, Bit-width=4bit2026.02 | 61.43 | |
| SLEBBase Model=Meta-LLaMA-3-8B, Parameter Size=7.4B, Evaluation Protocol=Zero-shot2026.02 | 61.09 | |
| GPTQ + MaCaBackbone=Qwen3-8B-IT, Bit-width=4bit2026.02 | 60.96 | |
| GPTAQBackbone=Qwen3-8B-IT, Bit-width=4bit2026.02 | 60.92 | |
| ShortGPTBase Model=Meta-LLaMA-3-8B, Parameter Size=7.4B, Evaluation Protocol=Zero-shot2026.02 | 60.54 | |
| GPTAQBackbone=Gemma3-12B-IT, Bit-width=4bit2026.02 | 59.88 | |
| GPTQ + MaCaBackbone=Gemma3-12B-IT, Bit-width=4bit2026.02 | 58.84 | |
| FP16 BaselineBackbone=LLaMA3-1B, Weight precision=FP16, Activation precision=FP16, Zero-shot=true2026.05 | 58.5 | |
| GPTQBackbone=Qwen3-8B-IT, Bit-width=4bit2026.02 | 58.4 | |
| GPTAQ + MaCaBackbone=Qwen3-4B-IT, Bit-width=4bit2026.02 | 57.21 | |
| GPTQBackbone=Gemma3-12B-IT, Bit-width=4bit2026.02 | 57.08 | |
| WinQ + JacQuantBackbone=LLaMA3-1B, Weight precision=W2, Activation precision=A16, Zero-shot=true2026.05 | 56.8 | |
| ParetoQ + JacQuantBackbone=LLaMA3-1B, Weight precision=W2, Activation precision=A16, Zero-shot=true2026.05 | 56.6 | |
| WinQBackbone=LLaMA3-1B, Weight precision=W2, Activation precision=A16, Zero-shot=true2026.05 | 56.6 | |
| ParetoQ + Rotation TrickBackbone=LLaMA3-1B, Weight precision=W2, Activation precision=A16, Zero-shot=true2026.05 | 56.1 | |
| WinQ + JacQuantBackbone=LLaMA3-1B, Weight precision=W1.58, Activation precision=A16, Zero-shot=true2026.05 | 56.1 | |
| GPTQ + MaCaBackbone=Qwen3-4B-IT, Bit-width=4bit2026.02 | 56.08 | |
| GPTAQ + MaCaBackbone=Qwen3-8B-IT, Bit-width=3bit2026.02 | 55.88 | |
| ParetoQBackbone=LLaMA3-1B, Weight precision=W2, Activation precision=A16, Zero-shot=true2026.05 | 55.7 | |
| WinQBackbone=LLaMA3-1B, Weight precision=W1.58, Activation precision=A16, Zero-shot=true2026.05 | 55.6 | |
| GPTAQBackbone=Qwen3-4B-IT, Bit-width=4bit2026.02 | 55.39 | |
| ParetoQ + JacQuantBackbone=LLaMA3-1B, Weight precision=W1.58, Activation precision=A16, Zero-shot=true2026.05 | 55.3 | |
| GPTQBackbone=Qwen3-4B-IT, Bit-width=4bit2026.02 | 55.01 | |
| GPTAQ + MaCaBackbone=Gemma3-4B-IT, Bit-width=4bit2026.02 | 54.97 | |
| GPTAQ + MaCaBackbone=Gemma3-12B-IT, Bit-width=3bit2026.02 | 54.81 | |
| ParetoQ + Rotation TrickBackbone=LLaMA3-1B, Weight precision=W1.58, Activation precision=A16, Zero-shot=true2026.05 | 54.7 | |
| GPTAQBackbone=Gemma3-4B-IT, Bit-width=4bit2026.02 | 54.5 | |
| ParetoQBackbone=LLaMA3-1B, Weight precision=W1.58, Activation precision=A16, Zero-shot=true2026.05 | 54.3 | |
| GPTAQ + MaCaBackbone=LLaMA3.1-8B-IT, Bit-width=4bit2026.02 | 54.11 | |
| WinQ + JacQuantBackbone=LLaMA3-1B, Weight precision=W1, Activation precision=A16, Zero-shot=true2026.05 | 53.5 | |
| WinQBackbone=LLaMA3-1B, Weight precision=W1, Activation precision=A16, Zero-shot=true2026.05 | 52.6 | |
| ParetoQ + JacQuantBackbone=LLaMA3-1B, Weight precision=W1, Activation precision=A16, Zero-shot=true2026.05 | 52.4 | |
| GPTAQBackbone=Gemma3-12B-IT, Bit-width=3bit2026.02 | 52.06 | |
| GPTAQBackbone=LLaMA3.1-8B-IT, Bit-width=4bit2026.02 | 52.06 | |
| ParetoQ + Rotation TrickBackbone=LLaMA3-1B, Weight precision=W1, Activation precision=A16, Zero-shot=true2026.05 | 52 | |
| GPTQ + MaCaBackbone=Gemma3-4B-IT, Bit-width=4bit2026.02 | 51.95 | |
| ParetoQBackbone=LLaMA3-1B, Weight precision=W1, Activation precision=A16, Zero-shot=true2026.05 | 51.9 | |
| GPTAQ + MaCaBackbone=Qwen3-4B-IT, Bit-width=3bit2026.02 | 51.24 | |
| GPTQ + MaCaBackbone=Gemma3-12B-IT, Bit-width=3bit2026.02 | 51.21 | |
| GPTQBackbone=Gemma3-4B-IT, Bit-width=4bit2026.02 | 50.94 | |
| GPTQ + MaCaBackbone=Qwen3-8B-IT, Bit-width=3bit2026.02 | 49.8 | |
| GPTQ + MaCaBackbone=Qwen3-8B-IT, Bit-width=2bit/g1282026.02 | 49.39 | |
| GPTQBackbone=Gemma3-12B-IT, Bit-width=3bit2026.02 | 47.46 | |
| GPTAQBackbone=Qwen3-8B-IT, Bit-width=3bit2026.02 | 47.21 | |
| Surrogate-Assisted Layer Contribution EstimationBase Model=Meta-LLaMA-3-8B, Parameter Size=6.1B, Evaluation Protocol=Zero-shot2026.02 | 46.76 | |
| GPTAQ + MaCaBackbone=LLaMA3.1-8B-IT, Bit-width=3bit2026.02 | 46.57 | |
| GPTQ + MaCaBackbone=Qwen3-4B-IT, Bit-width=3bit2026.02 | 46.28 | |
| GPTAQ + MaCaBackbone=Gemma3-4B-IT, Bit-width=3bit2026.02 | 46.2 | |
| GPTAQ + MaCaBackbone=LLaMA3.2-3B-IT, Bit-width=4bit2026.02 | 45.75 | |
| GPTQ + MaCaBackbone=Qwen3-4B-IT, Bit-width=2bit/g1282026.02 | 45.51 | |
| SLEBBase Model=Meta-LLaMA-3-8B, Parameter Size=6.1B, Evaluation Protocol=Zero-shot2026.02 | 45.26 | |
| GPTQ + MaCaBackbone=Gemma3-4B-IT, Bit-width=3bit2026.02 | 44.79 | |
| SliceGPTBase Model=Meta-LLaMA-3-8B, Parameter Size=7.4B, Evaluation Protocol=Zero-shot2026.02 | 44.65 | |
| GPTAQ + MaCaBackbone=Qwen3-8B-IT, Bit-width=2bit/g1282026.02 | 44.61 | |
| GPTQ + MaCaBackbone=LLaMA3.1-8B-IT, Bit-width=2bit/g1282026.02 | 44.52 | |
| GPTAQBackbone=Gemma3-4B-IT, Bit-width=3bit2026.02 | 44.31 | |
| GPTQBackbone=Qwen3-8B-IT, Bit-width=3bit2026.02 | 43.96 | |
| GPTQBackbone=Gemma3-4B-IT, Bit-width=3bit2026.02 | 43.38 | |
| GPTAQBackbone=Qwen3-4B-IT, Bit-width=3bit2026.02 | 43.3 | |
| GPTQBackbone=Qwen3-4B-IT, Bit-width=2bit/g1282026.02 | 43.23 | |
| GPTAQBackbone=LLaMA3.2-3B-IT, Bit-width=4bit2026.02 | 43.04 | |
| GPTAQBackbone=Qwen3-8B-IT, Bit-width=2bit/g1282026.02 | 43.02 | |
| GPTAQ + MaCaBackbone=LLaMA3.1-8B-IT, Bit-width=2bit/g1282026.02 | 42.35 | |
| GPTQBackbone=Qwen3-4B-IT, Bit-width=3bit2026.02 | 41.99 | |
| GPTAQBackbone=LLaMA3.1-8B-IT, Bit-width=3bit2026.02 | 41.97 | |
| GPTAQBackbone=LLaMA3.1-8B-IT, Bit-width=2bit/g1282026.02 | 41.67 | |
| GPTQBackbone=LLaMA3.1-8B-IT, Bit-width=2bit/g1282026.02 | 41.28 | |
| GPTAQ + MaCaBackbone=Gemma3-12B-IT, Bit-width=2bit/g1282026.02 | 40.78 | |
| GPTQBackbone=Qwen3-8B-IT, Bit-width=2bit/g1282026.02 | 40.62 | |
| GPTQ + MaCaBackbone=LLaMA3.2-3B-IT, Bit-width=2bit/g1282026.02 | 39.83 | |
| GPTAQ + MaCaBackbone=Qwen3-4B-IT, Bit-width=2bit/g1282026.02 | 39.7 | |
| GPTQ + MaCaBackbone=Gemma3-4B-IT, Bit-width=2bit/g1282026.02 | 39.26 | |
| GPTAQ + MaCaBackbone=LLaMA3.2-3B-IT, Bit-width=3bit2026.02 | 39.22 | |
| Surrogate-Assisted Layer Contribution EstimationBase Model=Meta-LLaMA-3-8B, Parameter Size=5.4B, Evaluation Protocol=Zero-shot2026.02 | 39.12 | |
| GPTAQ + MaCaBackbone=LLaMA3.2-3B-IT, Bit-width=2bit/g1282026.02 | 39.04 | |
| GPTAQBackbone=Gemma3-12B-IT, Bit-width=2bit/g1282026.02 | 38.96 | |
| GPTAQ + MaCaBackbone=Gemma3-4B-IT, Bit-width=2bit/g1282026.02 | 38.94 | |
| GPTAQBackbone=Gemma3-4B-IT, Bit-width=2bit/g1282026.02 | 38.91 | |
| GPTQBackbone=Gemma3-4B-IT, Bit-width=2bit/g1282026.02 | 38.66 | |
| GPTQ + MaCaBackbone=Gemma3-12B-IT, Bit-width=2bit/g1282026.02 | 38.64 | |
| GPTAQBackbone=LLaMA3.2-3B-IT, Bit-width=2bit/g1282026.02 | 38.59 | |
| RTN (PTQ)Backbone=LLaMA3-1B, Weight precision=W2, Activation precision=A16, Zero-shot=true2026.05 | 38.5 | |
| GPTQBackbone=Gemma3-12B-IT, Bit-width=2bit/g1282026.02 | 38.33 | |
| SpinQuant (PTQ)Backbone=LLaMA3-1B, Weight precision=W2, Activation precision=A16, Zero-shot=true2026.05 | 38.3 | |
| GPTQBackbone=LLaMA3.2-3B-IT, Bit-width=2bit/g1282026.02 | 37.97 | |
| SLEBBase Model=Meta-LLaMA-3-8B, Parameter Size=5.4B, Evaluation Protocol=Zero-shot2026.02 | 36.96 | |
| GPTQ (PTQ)Backbone=LLaMA3-1B, Weight precision=W2, Activation precision=A16, Zero-shot=true2026.05 | 36.8 | |
| GPTAQBackbone=LLaMA3.2-3B-IT, Bit-width=3bit2026.02 | 36.43 | |
| SliceGPTBase Model=Meta-LLaMA-3-8B, Parameter Size=6.1B, Evaluation Protocol=Zero-shot2026.02 | 36.2 | |
| RTN (PTQ)Backbone=LLaMA3-1B, Weight precision=W1.58, Activation precision=A16, Zero-shot=true2026.05 | 36.2 | |
| ShortGPTBase Model=Meta-LLaMA-3-8B, Parameter Size=6.1B, Evaluation Protocol=Zero-shot2026.02 | 35.98 | |
| GPTQ + MaCaBackbone=LLaMA3.1-8B-IT, Bit-width=3bit2026.02 | 35.8 | |
| Shortened-LLaMABase Model=Meta-LLaMA-3-8B, Parameter Size=5.4B, Evaluation Protocol=Zero-shot2026.02 | 35.76 | |
| ShortGPTBase Model=Meta-LLaMA-3-8B, Parameter Size=5.4B, Evaluation Protocol=Zero-shot2026.02 | 35.76 | |
| GPTQ + MaCaBackbone=LLaMA3.1-8B-IT, Bit-width=4bit2026.02 | 35.31 | |
| GPTQBackbone=LLaMA3.1-8B-IT, Bit-width=4bit2026.02 | 35.16 | |
| GPTQBackbone=LLaMA3.1-8B-IT, Bit-width=3bit2026.02 | 34.65 |