Zero-shot Common-sense Reasoning Suite (ARC-e, ARC-c, BoolQ, PIQA, SIQA, HellaSwag, OBQA, WinoGrande)
74.2ARC-e AccuracyStatQAT-iterative
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| StatQAT-iterativeModel=Llama 3 8B, Type=FP4, Granularity=Tensor2026.05 | 74.2 | 42.3 | 77.9 | 75.4 | 43.7 | 54.5 | 29.2 | 68.4 | 58.2 | |
| StatQAT-iterativeModel=Llama 3.2 3B, Type=FP4, Granularity=Tensor2026.05 | 69.7 | 38.7 | 69.9 | 73 | 41.8 | 47.1 | 26.4 | 61.6 | 53.5 | |
| StatQAT-iterativeModel=MobileLLM 600M, Type=FP4, Granularity=Tensor2026.05 | 58.9 | 27.1 | 62.1 | 69.5 | 38.5 | 41.2 | 21.8 | 59.1 | 47.3 | |
| StatQAT-iterativeModel=Llama 3.2 1B, Type=FP4, Granularity=Tensor2026.05 | 58.6 | 28.3 | 64.2 | 70.5 | 39.6 | 42.4 | 23.4 | 57.9 | 48.1 | |
| StatQAT-analyticModel=MobileLLM 600M, Type=FP4, Granularity=Tensor2026.05 | 57.1 | 27.6 | 62.2 | 69.4 | 38.7 | 40 | 21.4 | 56.4 | 46.6 | |
| FP16 BaselineModel=MobileLLM 125M, Type=FP16 Baseline2026.05 | 46 | 20.3 | 57.8 | 64.7 | 38.2 | 32.7 | 18 | 52.5 | 41.3 | |
| StatQAT-iterativeModel=MobileLLM 125M, Type=FP4, Granularity=Tensor2026.05 | 44.6 | 20.1 | 54.3 | 61.3 | 37.8 | 31.7 | 18 | 53.1 | 40.1 | |
| StatQAT-analyticModel=MobileLLM 125M, Type=FP4, Granularity=Tensor2026.05 | 43.7 | 19.1 | 55.1 | 61.6 | 37.4 | 31.8 | 16.2 | 52 | 39.6 |