Language Modeling on WikiText-2 (Perplexity and Recovery)
4.88PerplexityFP16
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FP16Model=LLaMA-2-13B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 4.88 | — | |
| AAACModel=LLaMA-2-13B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A16, Selection group size S=162026.05 | 4.94 | — | |
| OmniQModel=LLaMA-2-13B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 4.95 | — | |
| QuIP#-FTModel=LLaMA-2-13B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 4.95 | — | |
| SqLLMModel=LLaMA-2-13B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 4.96 | — | |
| GPTVQModel=LLaMA-2-13B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 4.97 | — | |
| BF16Model=Qwen2.5-32B, Group Size (g)=16, Quantization Precision=W4A162026.05 | 4.97 | — | |
| RTNModel=LLaMA-2-13B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 4.98 | — | |
| QuIP#Model=LLaMA-2-13B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5 | — | |
| FP16Model=LLaMA-1-13B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.09 | — | |
| AAACModel=Qwen2.5-32B, Group Size (g)=16, Quantization Precision=W4A162026.05 | 5.09 | — | |
| IF4Model=Qwen2.5-32B, Group Size (g)=16, Quantization Precision=W4A162026.05 | 5.12 | — | |
| GPTVQModel=LLaMA-1-13B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.15 | — | |
| AWQModel=Qwen2.5-32B, Group Size (g)=16, Quantization Precision=W4A162026.05 | 5.15 | — | |
| OmniQModel=LLaMA-1-13B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.17 | — | |
| SqLLMModel=LLaMA-1-13B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.17 | — | |
| QuIP#-FTModel=LLaMA-1-13B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.17 | — | |
| AAACModel=LLaMA-1-13B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A16, Selection group size S=162026.05 | 5.17 | — | |
| RTNModel=Qwen2.5-32B, Group Size (g)=16, Quantization Precision=W4A162026.05 | 5.17 | — | |
| QuIP#Model=LLaMA-1-13B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.2 | — | |
| RTNModel=LLaMA-1-13B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.25 | — | |
| BF16Model=Qwen2.5-14B, Group Size (g)=16, Quantization Precision=W4A162026.05 | 5.25 | — | |
| FP16Model=Average, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.28 | — | |
| QuIP#-FTModel=Average, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.36 | 60 | |
| AAACModel=Average, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A16, Selection group size S=162026.05 | 5.36 | 60 | |
| OmniQModel=Average, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.37 | 55 | |
| SqLLMModel=Average, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.37 | 55 | |
| QuIP#Model=Average, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.42 | 30 | |
| GPTVQModel=Average, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.43 | 25 | |
| AAACModel=Qwen2.5-14B, Group Size (g)=16, Quantization Precision=W4A162026.05 | 5.45 | — | |
| FP16Model=LLaMA-2-7B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.47 | — | |
| RTNModel=Average, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.48 | — | |
| IF4Model=Qwen2.5-14B, Group Size (g)=16, Quantization Precision=W4A162026.05 | 5.54 | — | |
| AWQModel=Qwen2.5-14B, Group Size (g)=16, Quantization Precision=W4A162026.05 | 5.54 | — | |
| QuIP#-FTModel=LLaMA-2-7B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.56 | — | |
| SqLLMModel=LLaMA-2-7B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.57 | — | |
| OmniQModel=LLaMA-2-7B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.58 | — | |
| AAACModel=LLaMA-2-7B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A16, Selection group size S=162026.05 | 5.58 | — | |
| RTNModel=Qwen2.5-14B, Group Size (g)=16, Quantization Precision=W4A162026.05 | 5.59 | — | |
| GPTVQModel=LLaMA-2-7B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.62 | — | |
| QuIP#Model=LLaMA-2-7B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.66 | — | |
| FP16Model=LLaMA-1-7B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.68 | — | |
| RTNModel=LLaMA-2-7B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.72 | — | |
| QuIP#-FTModel=LLaMA-1-7B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.76 | — | |
| AAACModel=LLaMA-1-7B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A16, Selection group size S=162026.05 | 5.76 | — | |
| OmniQModel=LLaMA-1-7B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.77 | — | |
| SqLLMModel=LLaMA-1-7B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.77 | — | |
| QuIP#Model=LLaMA-1-7B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.83 | — | |
| RTNModel=LLaMA-1-7B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.96 | — | |
| GPTVQModel=LLaMA-1-7B, Group Size (g)=128, Weight/Activation Precision (W/A)=W4A162026.05 | 5.96 | — | |
| BF16Backbone=Llama-3.1-8B, Precision=BF162026.05 | 6.19 | — | |
| AAACBackbone=Llama-3.1-8B, Precision=W4A16, Group Size=16, S=162026.05 | 6.39 | — | |
| IF4Backbone=Llama-3.1-8B, Precision=W4A16, Group Size=162026.05 | 6.51 | — | |
| AWQBackbone=Llama-3.1-8B, Precision=W4A16, Group Size=162026.05 | 6.51 | — | |
| RTNBackbone=Llama-3.1-8B, Precision=W4A16, Group Size=162026.05 | 6.56 | — | |
| BF16Backbone=Qwen3.5-27B, Precision=BF162026.05 | 6.8 | — | |
| BF16Model=Qwen2.5-7B, Group Size (g)=16, Quantization Precision=W4A162026.05 | 6.8 | — | |
| AAACBackbone=Qwen3.5-27B, Precision=W4A16, Group Size=16, S=162026.05 | 6.88 | — | |
| IF4Backbone=Qwen3.5-27B, Precision=W4A16, Group Size=162026.05 | 6.92 | — | |
| AAACModel=Qwen2.5-7B, Group Size (g)=16, Quantization Precision=W4A162026.05 | 6.93 | — | |
| IF4Model=Qwen2.5-7B, Group Size (g)=16, Quantization Precision=W4A162026.05 | 6.97 | — | |
| AWQModel=Qwen2.5-7B, Group Size (g)=16, Quantization Precision=W4A162026.05 | 7 | — | |
| RTNBackbone=Qwen3.5-27B, Precision=W4A16, Group Size=162026.05 | 7.02 | — | |
| AWQBackbone=Qwen3.5-27B, Precision=W4A16, Group Size=162026.05 | 7.03 | — | |
| RTNModel=Qwen2.5-7B, Group Size (g)=16, Quantization Precision=W4A162026.05 | 7.03 | — | |
| BF16Model=Average, Group Size (g)=16, Quantization Precision=W4A162026.05 | 7.51 | — | |
| BF16Backbone=Llama-3.2-3B, Precision=BF162026.05 | 7.77 | — | |
| AAACModel=Average, Group Size (g)=16, Quantization Precision=W4A162026.05 | 7.78 | 47.8 | |
| IF4Model=Average, Group Size (g)=16, Quantization Precision=W4A162026.05 | 7.94 | 18.4 | |
| AWQModel=Average, Group Size (g)=16, Quantization Precision=W4A162026.05 | 7.97 | 12.6 | |
| AAACBackbone=Llama-3.2-3B, Precision=W4A16, Group Size=16, S=162026.05 | 7.99 | — | |
| RTNModel=Average, Group Size (g)=16, Quantization Precision=W4A162026.05 | 8.03 | — | |
| AWQBackbone=Llama-3.2-3B, Precision=W4A16, Group Size=162026.05 | 8.11 | — | |
| IF4Backbone=Llama-3.2-3B, Precision=W4A16, Group Size=162026.05 | 8.13 | — | |
| RTNBackbone=Llama-3.2-3B, Precision=W4A16, Group Size=162026.05 | 8.19 | — | |
| BF16Backbone=Qwen3.5-9B, Precision=BF162026.05 | 8.51 | — | |
| AAACBackbone=Qwen3.5-9B, Precision=W4A16, Group Size=16, S=162026.05 | 8.56 | — | |
| BF16Backbone=Average, Precision=BF162026.05 | 8.64 | — | |
| AAACBackbone=Average, Precision=W4A16, Group Size=16, S=162026.05 | 8.9 | 59.2 | |
| AWQBackbone=Qwen3.5-9B, Precision=W4A16, Group Size=162026.05 | 8.91 | — | |
| RTNBackbone=Qwen3.5-9B, Precision=W4A16, Group Size=162026.05 | 9.11 | — | |
| AWQBackbone=Average, Precision=W4A16, Group Size=162026.05 | 9.11 | 28.1 | |
| IF4Backbone=Average, Precision=W4A16, Group Size=162026.05 | 9.21 | 12.3 | |
| RTNBackbone=Average, Precision=W4A16, Group Size=162026.05 | 9.29 | — | |
| IF4Backbone=Qwen3.5-9B, Precision=W4A16, Group Size=162026.05 | 9.38 | — | |
| BF16Backbone=Qwen3.5-4B, Precision=BF162026.05 | 9.42 | — | |
| BF16Backbone=Llama-3.2-1B, Precision=BF162026.05 | 9.71 | — | |
| AAACBackbone=Qwen3.5-4B, Precision=W4A16, Group Size=16, S=162026.05 | 9.92 | — | |
| AWQBackbone=Qwen3.5-4B, Precision=W4A16, Group Size=162026.05 | 10.16 | — | |
| AAACBackbone=Llama-3.2-1B, Precision=W4A16, Group Size=16, S=162026.05 | 10.21 | — | |
| IF4Backbone=Qwen3.5-4B, Precision=W4A16, Group Size=162026.05 | 10.21 | — | |
| RTNBackbone=Qwen3.5-4B, Precision=W4A16, Group Size=162026.05 | 10.25 | — | |
| AWQBackbone=Llama-3.2-1B, Precision=W4A16, Group Size=162026.05 | 10.46 | — | |
| IF4Backbone=Llama-3.2-1B, Precision=W4A16, Group Size=162026.05 | 10.61 | — | |
| RTNBackbone=Llama-3.2-1B, Precision=W4A16, Group Size=162026.05 | 10.68 | — | |
| BF16Backbone=Qwen3.5-2B, Precision=BF162026.05 | 12.06 | — | |
| AAACBackbone=Qwen3.5-2B, Precision=W4A16, Group Size=16, S=162026.05 | 12.37 | — | |
| AWQBackbone=Qwen3.5-2B, Precision=W4A16, Group Size=162026.05 | 12.56 | — | |
| IF4Backbone=Qwen3.5-2B, Precision=W4A16, Group Size=162026.05 | 12.7 | — | |
| BF16Model=Qwen2.5-0.5B, Group Size (g)=16, Quantization Precision=W4A162026.05 | 13.03 | — |