Zero-shot Evaluation on Downstream Tasks
76AccuracyFP16
Evaluation Results
| Method | Links | |
|---|---|---|
| FP16Weight bit=16, Pruning Ratio=0%, Reduction size=0x, Backbone=Nemotron-H 8B2025.12 | 76 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=0%, Reduction size=4x, Backbone=Bamba-v2 9B2025.12 | 75.1 | |
| FP16Weight bit=16, Pruning Ratio=0%, Reduction size=0x, Backbone=Bamba-v2 9B2025.12 | 74.6 | |
| BF16Model Size=8B, Precision=BF162025.05 | 74.4 | |
| BF16Model Size=8B2025.05 | 74.4 | |
| BF16Model Size=8B2025.05 | 74.1 | |
| FP16Weight bit=16, Pruning Ratio=0%, Reduction size=0x, Backbone=Llama-3.1 8B2025.12 | 74 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=0%, Reduction size=4x, Backbone=Llama-3.1 8B2025.12 | 73.6 | |
| BF16Stage 1=None, Model Size=8B, Quantization Bits=None2025.05 | 73.6 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=0%, Reduction size=4x, Backbone=Nemotron-H 8B2025.12 | 73.3 | |
| GPTAQStage 1=HIP, Bits=W4, Model Size=8B2025.05 | 72.7 | |
| FP16Weight bit=16, Pruning Ratio=0%, Reduction size=0x, Backbone=Qwen-2.5 7B2025.12 | 72.4 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=0%, Reduction size=4x, Backbone=Qwen-2.5 7B2025.12 | 72.4 | |
| QronosStage 1=None, Bits=W4, Model Size=8B2025.05 | 72.2 | |
| OPTQStage 1=HIP, Bits=W4, Model Size=8B2025.05 | 72.1 | |
| QronosStage 1=HIP, Bits=W4, Model Size=8B2025.05 | 72.1 | |
| GPFQStage 1=HIP, Bits=W4, Model Size=8B2025.05 | 71.9 | |
| OPTQStage 1=None, Bits=W4, Model Size=8B2025.05 | 71.8 | |
| GPFQStage 1=None, Bits=W4, Model Size=8B2025.05 | 71.5 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=15%, Reduction size=4.7x, Backbone=Llama-3.1 8B2025.12 | 71.4 | |
| GPTAQStage 1=None, Bits=W4, Model Size=8B2025.05 | 71.2 | |
| RTNStage 1=HIP, Bits=W4, Model Size=8B2025.05 | 70.8 | |
| FP16Weight bit=16, Pruning Ratio=0%, Reduction size=0x, Backbone=Mamba2 8B2025.12 | 70.6 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=15%, Reduction size=4.7x, Backbone=Nemotron-H 8B2025.12 | 70.5 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=15%, Reduction size=4.7x, Backbone=Bamba-v2 9B2025.12 | 70.3 | |
| BF16Stage 1=None, Model Size=4B, Quantization Bits=None2025.05 | 70.1 | |
| QronosTransformation=SmoothRot, Precision=W4A4KV16, Model Size=8B2025.05 | 69.4 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=0%, Reduction size=4x, Backbone=Mamba2 8B2025.12 | 69.3 | |
| QronosTransformation=QuaRot, Precision=W4A4KV16, Model Size=8B2025.05 | 68.9 | |
| FP16Weight bit=16, Pruning Ratio=0%, Reduction size=0x, Backbone=Llama-2 7B2025.12 | 68.8 | |
| GPTAQTransformation=SmoothRot, Precision=W4A4KV16, Model Size=8B2025.05 | 68.8 | |
| OPTQTransformation=SmoothRot, Precision=W4A4KV16, Model Size=8B2025.05 | 68.5 | |
| QronosTransformation=SpinQuant, Precision=W4A4KV16, Model Size=8B2025.05 | 68.4 | |
| GPTAQTransformation=SpinQuant, Precision=W4A4KV16, Model Size=8B2025.05 | 68.3 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=15%, Reduction size=4.7x, Backbone=Qwen-2.5 7B2025.12 | 68.1 | |
| GPTAQTransformation=QuaRot, Precision=W4A4KV16, Model Size=8B2025.05 | 68.1 | |
| QronosStage 1=HIP, Stage 2=Qronos, Model Size=8B, Quantization Bits=W3, Beta=0.92025.05 | 68 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=25%, Reduction size=5.3x, Backbone=Llama-3.1 8B2025.12 | 67.7 | |
| GPFQTransformation=SmoothRot, Precision=W4A4KV16, Model Size=8B2025.05 | 67.7 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=0%, Reduction size=4x, Backbone=Llama-2 7B2025.12 | 67.6 | |
| RTNStage 1=None, Bits=W4, Model Size=8B2025.05 | 67.6 | |
| OPTQStage 1=HIP, Stage 2=OPTQ, Model Size=8B, Quantization Bits=W3, Beta=0.92025.05 | 67.6 | |
| GPFQTransformation=QuaRot, Precision=W4A4KV16, Model Size=8B2025.05 | 67.6 | |
| BF16Model Size=3B, Precision=BF162025.05 | 67.5 | |
| BF16Model Size=3B2025.05 | 67.5 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=25%, Reduction size=5.3x, Backbone=Bamba-v2 9B2025.12 | 67.4 | |
| OPTQTransformation=SpinQuant, Precision=W4A4KV16, Model Size=8B2025.05 | 67 | |
| GPFQTransformation=SpinQuant, Precision=W4A4KV16, Model Size=8B2025.05 | 67 | |
| SVD-LLMone-pass=false, Fine-tuning=true, Weight bit=4 (FP16 embeddings/output), Pruning Ratio=15%, Reduction size=4.7x, Backbone=Qwen-2.5 7B2025.12 | 66.8 | |
| OPTQTransformation=QuaRot, Precision=W4A4KV16, Model Size=8B2025.05 | 66.7 | |
| BF16Model Size=3B2025.05 | 66.4 | |
| GPTAQStage 1=HIP, Stage 2=GPTAQ, Model Size=8B, Quantization Bits=W3, Beta=0.92025.05 | 66 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=15%, Reduction size=4.7x, Backbone=Mamba2 8B2025.12 | 65.8 | |
| QronosTransformation=SpinQuant, Precision=W4A4KV4, Model Size=8B2025.05 | 65.8 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=15%, Reduction size=4.7x, Backbone=Llama-2 7B2025.12 | 65.6 | |
| OPTQTransformation=SpinQuant, Precision=W4A4KV4, Model Size=8B2025.05 | 65.5 | |
| GPFQStage 1=HIP, Bits=W4, Model Size=3B2025.05 | 65.3 | |
| GPTAQTransformation=SpinQuant, Precision=W4A4KV4, Model Size=8B2025.05 | 65.3 | |
| QronosTransformation=SmoothRot, Precision=W4A4KV4, Model Size=8B2025.05 | 65.2 | |
| QronosStage 1=None, Bits=W4, Model Size=3B2025.05 | 64.8 | |
| QronosStage 1=HIP, Bits=W4, Model Size=3B2025.05 | 64.8 | |
| GPTAQTransformation=QuaRot, Precision=W4A4KV4, Model Size=8B2025.05 | 64.8 | |
| QronosTransformation=QuaRot, Precision=W4A4KV4, Model Size=8B2025.05 | 64.8 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=25%, Reduction size=5.3x, Backbone=Nemotron-H 8B2025.12 | 64.7 | |
| SVD-LLMone-pass=false, Fine-tuning=true, Weight bit=4 (FP16 embeddings/output), Pruning Ratio=25%, Reduction size=4.7x, Backbone=Qwen-2.5 7B2025.12 | 64.6 | |
| OPTQStage 1=HIP, Bits=W4, Model Size=3B2025.05 | 64.5 | |
| GPFQStage 1=None, Bits=W4, Model Size=3B2025.05 | 64.4 | |
| OPTQStage 1=HIP, Stage 2=OPTQ, Model Size=4B, Quantization Bits=W3, Beta=0.92025.05 | 64.4 | |
| MoDeGPTone-pass=false, Fine-tuning=true, Weight bit=4, Pruning Ratio=15%, Reduction size=4.7x, Backbone=Llama-3.1 8B2025.12 | 64.2 | |
| RTNTransformation=SpinQuant, Precision=W4A4KV16, Model Size=8B2025.05 | 64.2 | |
| QronosStage 1=HIP, Bits=W3, Model Size=8B2025.05 | 64.1 | |
| OPTQTransformation=QuaRot, Precision=W4A4KV4, Model Size=8B2025.05 | 64.1 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=25%, Reduction size=5.3x, Backbone=Qwen-2.5 7B2025.12 | 64 | |
| GPTAQStage 1=HIP, Bits=W4, Model Size=3B2025.05 | 63.9 | |
| BF16Stage 1=None, Model Size=1.7B, Quantization Bits=None2025.05 | 63.9 | |
| MoDeGPTone-pass=false, Fine-tuning=true, Weight bit=4, Pruning Ratio=15%, Reduction size=4.7x, Backbone=Llama-2 7B2025.12 | 63.7 | |
| GPTAQStage 1=None, Stage 2=GPTAQ, Model Size=4B, Quantization Bits=W3, Beta=0.92025.05 | 63.7 | |
| GPTAQTransformation=SmoothRot, Precision=W4A4KV4, Model Size=8B2025.05 | 63.7 | |
| OPTQStage 1=None, Bits=W4, Model Size=3B2025.05 | 63.6 | |
| RTNStage 1=HIP, Bits=W4, Model Size=3B2025.05 | 63.6 | |
| GPTAQStage 1=HIP, Stage 2=GPTAQ, Model Size=4B, Quantization Bits=W3, Beta=0.92025.05 | 63.6 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=25%, Reduction size=5.3x, Backbone=Llama-2 7B2025.12 | 63.5 | |
| QronosStage 1=HIP, Stage 2=Qronos, Model Size=4B, Quantization Bits=W3, Beta=0.92025.05 | 63.5 | |
| SVD-LLMone-pass=false, Fine-tuning=true, Weight bit=4 (FP16 embeddings/output), Pruning Ratio=15%, Reduction size=4.7x, Backbone=Llama-2 7B2025.12 | 63.2 | |
| GPTAQStage 1=HIP, Bits=W3, Model Size=8B2025.05 | 63.2 | |
| GPTAQStage 1=None, Bits=W4, Model Size=3B2025.05 | 63.1 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=35%, Reduction size=6.1x, Backbone=Llama-3.1 8B2025.12 | 62.7 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=35%, Reduction size=6.1x, Backbone=Bamba-v2 9B2025.12 | 62.7 | |
| RTNTransformation=QuaRot, Precision=W4A4KV16, Model Size=8B2025.05 | 62.6 | |
| RTNTransformation=SmoothRot, Precision=W4A4KV16, Model Size=8B2025.05 | 62.6 | |
| RTNStage 1=None, Bits=W4, Model Size=3B2025.05 | 62.3 | |
| GPFQStage 1=HIP, Stage 2=GPFQ, Model Size=8B, Quantization Bits=W3, Beta=0.92025.05 | 62.2 | |
| QronosTransformation=SpinQuant, Precision=W4A4KV16, Model Size=3B2025.05 | 62.1 | |
| GPFQTransformation=SmoothRot, Precision=W4A4KV16, Model Size=3B2025.05 | 62 | |
| UniQLone-pass=true, Fine-tuning=true, Weight bit=4, Pruning Ratio=25%, Reduction size=5.3x, Backbone=Mamba2 8B2025.12 | 61.8 | |
| OPTQStage 1=None, Stage 2=OPTQ, Model Size=8B, Quantization Bits=W3, Beta=0.92025.05 | 61.7 | |
| GPFQStage 1=HIP, Stage 2=GPFQ, Model Size=4B, Quantization Bits=W3, Beta=0.92025.05 | 61.6 | |
| QronosStage 1=None, Stage 2=Qronos, Model Size=8B, Quantization Bits=W3, Beta=0.92025.05 | 61.5 | |
| QronosTransformation=QuaRot, Precision=W4A4KV16, Model Size=3B2025.05 | 61.5 | |
| GPTAQTransformation=QuaRot, Precision=W4A4KV16, Model Size=3B2025.05 | 61.4 |