Commonsense Reasoning and Knowledge on ARC-C, BoolQ, Lambada, PIQA, Winogrande
53.58ARC-C AccuracyFP16
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FP16Avg. Bits=16.00, Base Model=Llama3.1-8B, Evaluation Protocol=0-shot, Quantization=None2025.08 | 53.58 | 81.99 | 75.47 | 80.09 | 74.03 | 73.03 | |
| MicroMixAvg. Bits=5.51, Base Model=Llama3.1-8B, Evaluation Protocol=0-shot, KV Cache Quantization=Asymmetric INT4 (group size 64)2025.08 | 52.22 | 80.64 | 74 | 79.38 | 70.96 | 71.44 | |
| FlatQuantAvg. Bits=4.19, Base Model=Llama3.1-8B, Evaluation Protocol=0-shot, KV Cache Quantization=Asymmetric INT4 (group size 64)2025.08 | 50.17 | 79.33 | 72.15 | 79.27 | 71.59 | 70.5 | |
| QUIKAvg. Bits=5.95, Base Model=Llama3.1-8B, Evaluation Protocol=0-shot, KV Cache Quantization=Asymmetric INT4 (group size 64)2025.08 | 49.66 | 77.77 | 71.08 | 78.51 | 67.88 | 68.98 | |
| INT6Avg. Bits=6.00, Base Model=Llama3.1-8B, Evaluation Protocol=0-shot, KV Cache Quantization=Asymmetric INT4 (group size 64)2025.08 | 49.15 | 76.73 | 68.62 | 78.07 | 69.06 | 68.32 | |
| AtomAvg. Bits=4.25, Base Model=Llama3.1-8B, Evaluation Protocol=0-shot, KV Cache Quantization=Asymmetric INT4 (group size 64)2025.08 | 47.95 | 79.94 | 72.81 | 78.35 | 70.72 | 69.95 | |
| AMXFP4Avg. Bits=5.00, Base Model=Llama3.1-8B, Evaluation Protocol=0-shot, KV Cache Quantization=Asymmetric INT4 (group size 64)2025.08 | 46.84 | 73.24 | 69.59 | 77.15 | 67.56 | 66.87 | |
| QuaRotAvg. Bits=4.12, Base Model=Llama3.1-8B, Evaluation Protocol=0-shot, KV Cache Quantization=Asymmetric INT4 (group size 64)2025.08 | 44.88 | 74.19 | 66.74 | 77.09 | 66.61 | 65.9 |