Zero-shot Reasoning and QA on (PIQA, HellaSwag, Winogrande, ARC-Challenge, ARC-Easy)
77.47PIQA Zero-Shot Accuracybfloat16
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| bfloat16Model=LLaMA-3.2-3B, Quantization=bfloat16, Evaluation Protocol=Zero-shot2026.04 | 77.47 | 73.62 | 69.61 | 45.9 | 71.71 | 67.67 | |
| Bit-by-BitModel=LLaMA-3.2-3B, Quantization=w2a16, Evaluation Protocol=Zero-shot2026.04 | 71.87 | 58.03 | 60.38 | 35.58 | 58.71 | 56.91 | |
| BitDistillerModel=LLaMA-3.2-3B, Quantization=w2a16, Evaluation Protocol=Zero-shot2026.04 | 70.65 | 57.42 | 59.78 | 34.71 | 58.34 | 56.18 | |
| EfficientQATModel=LLaMA-3.2-3B, Quantization=w2a16, Evaluation Protocol=Zero-shot2026.04 | 70.02 | 57.07 | 59.35 | 34.13 | 58.92 | 55.89 | |
| ParetoQModel=LLaMA-3.2-3B, Quantization=w2a16, Evaluation Protocol=Zero-shot2026.04 | 66.7 | 43.48 | 52.49 | 21.93 | 44.36 | 45.79 | |
| Bit-by-BitModel=LLaMA-3.2-3B, Quantization=w2a2, Evaluation Protocol=Zero-shot2026.04 | 66 | 49.3 | 56.91 | 31.4 | 54 | 51.52 | |
| BitDistillerModel=LLaMA-3.2-3B, Quantization=w2a2, Evaluation Protocol=Zero-shot2026.04 | 60.87 | 42.15 | 54.03 | 26.72 | 47.61 | 46.28 | |
| EfficientQATModel=LLaMA-3.2-3B, Quantization=w2a2, Evaluation Protocol=Zero-shot2026.04 | 56.53 | 34.76 | 52.17 | 21.84 | 35.23 | 40.1 | |
| ParetoQModel=LLaMA-3.2-3B, Quantization=w2a2, Evaluation Protocol=Zero-shot2026.04 | 51.8 | 25.76 | 48.78 | 23.55 | 27.53 | 35.48 |