Zero-shot NLU on BoolQ, ARC-E, ARC-C, and HellaSwag
83.2BoolQ AccuracyBF16
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| BF16Model Backbone=Llama-3.1-8B2025.12 | 83.2 | 82.5 | 55 | 79.3 | 75 | |
| GPTQ + 4/6Model Backbone=Llama-3.1-8B, Adaptive Block Scaling=4/6, Precision=FP42025.12 | 81.4 | 78.7 | 53 | 77.4 | 72.6 | |
| AWQModel Backbone=Llama-3.1-8B, Precision=FP42025.12 | 81.3 | 78.4 | 51.7 | 77.5 | 72.2 | |
| SmoothQuantModel Backbone=Llama-3.1-8B, Precision=FP42025.12 | 81.1 | 77.6 | 52.7 | 77.6 | 72.3 | |
| RTN + 4/6Model Backbone=Llama-3.1-8B, Adaptive Block Scaling=4/6, Precision=FP42025.12 | 80.9 | 80.2 | 52.6 | 78 | 72.2 | |
| AWQ + 4/6Model Backbone=Llama-3.1-8B, Adaptive Block Scaling=4/6, Precision=FP42025.12 | 80.4 | 80.2 | 53.6 | 78.2 | 73.1 | |
| SmoothQuant + 4/6Model Backbone=Llama-3.1-8B, Adaptive Block Scaling=4/6, Precision=FP42025.12 | 80.4 | 78.6 | 52.2 | 80.4 | 72.9 | |
| GPTQModel Backbone=Llama-3.1-8B, Precision=FP42025.12 | 80.3 | 78.3 | 53.9 | 77 | 72.4 | |
| RTNModel Backbone=Llama-3.1-8B, Precision=FP42025.12 | 80.2 | 77.5 | 52.5 | 77.7 | 72 | |
| BF16Model Backbone=Llama-3.2-1B2025.12 | 63.7 | 61.8 | 37 | 64.3 | 56.7 | |
| SmoothQuant + 4/6Model Backbone=Llama-3.2-1B, Adaptive Block Scaling=4/6, Precision=FP42025.12 | 61.6 | 58 | 35.6 | 61.6 | 54.2 | |
| GPTQModel Backbone=Llama-3.2-1B, Precision=FP42025.12 | 61.1 | 57.3 | 33.1 | 60.6 | 53 | |
| SmoothQuantModel Backbone=Llama-3.2-1B, Precision=FP42025.12 | 61.1 | 57.4 | 35.5 | 60.7 | 53.7 | |
| AWQ + 4/6Model Backbone=Llama-3.2-1B, Adaptive Block Scaling=4/6, Precision=FP42025.12 | 61 | 58.8 | 35.5 | 61.2 | 54.1 | |
| GPTQ + 4/6Model Backbone=Llama-3.2-1B, Adaptive Block Scaling=4/6, Precision=FP42025.12 | 60.2 | 57.5 | 33.8 | 60.7 | 53.1 | |
| AWQModel Backbone=Llama-3.2-1B, Precision=FP42025.12 | 59.8 | 58 | 34.2 | 60.9 | 53.2 | |
| RTNModel Backbone=Llama-3.2-1B, Precision=FP42025.12 | 58.6 | 57.3 | 34.3 | 60 | 52.3 | |
| RTN + 4/6Model Backbone=Llama-3.2-1B, Adaptive Block Scaling=4/6, Precision=FP42025.12 | 57.7 | 56.9 | 33 | 60 | 51.9 |