Zero-shot Commonsense Reasoning on ARC-e, ARC-c, HellaSwag, PIQA, WinoGrande (test)
82.87ARC-e AccuracyFP16
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FP16Model=LLaMA-3.1 8B-Instruct, Precision=FP162026.05 | 82.87 | 54.78 | 79.3 | 81.01 | 74.43 | 74.48 | |
| RaZeRModel=LLaMA-3.1 8B-Instruct, Quantization=W4A162026.05 | 81.69 | 54.96 | 78.85 | 81.06 | 73.24 | 73.96 | |
| 4over6Model=LLaMA-3.1 8B-Instruct, Quantization=W4A162026.05 | 81.57 | 55.03 | 78.94 | 81.07 | 73.24 | 73.97 | |
| SOARModel=Qwen3 8B, Quantization=W4A162026.05 | 81.44 | 56.99 | 73.96 | 77.8 | 69.69 | 71.98 | |
| FP16Model=Qwen3 8B, Precision=FP162026.05 | 80.89 | 56.66 | 74.86 | 77.58 | 67.64 | 71.53 | |
| NVFP4Model=LLaMA-3.1 8B-Instruct, Quantization=W4A162026.05 | 80.39 | 53.5 | 78.84 | 80.63 | 73.24 | 73.32 | |
| NVFP4Model=Qwen3 8B, Quantization=W4A162026.05 | 80.22 | 56.66 | 74.06 | 76.71 | 68.43 | 71.22 | |
| SOARModel=LLaMA-3.1 8B-Instruct, Quantization=W4A162026.05 | 80.18 | 53.92 | 79 | 80.2 | 73.8 | 73.42 | |
| RaZeRModel=Qwen3 8B, Quantization=W4A162026.05 | 79.97 | 56.48 | 74.24 | 77.2 | 68.03 | 71.18 | |
| 4over6Model=Qwen3 8B, Quantization=W4A162026.05 | 79.63 | 55.8 | 73.97 | 76.17 | 68.19 | 70.75 | |
| FP16Model=Qwen3 4B, Precision=FP162026.05 | 78.45 | 54.01 | 68.46 | 75.08 | 65.9 | 68.38 | |
| SOARModel=Qwen3 4B, Quantization=W4A162026.05 | 76.73 | 52.13 | 67.88 | 75.03 | 63.61 | 67.08 | |
| NVFP4Model=Qwen3 4B, Quantization=W4A162026.05 | 76.47 | 50.68 | 67.26 | 74.7 | 62.98 | 66.42 | |
| 4over6Model=Qwen3 4B, Quantization=W4A162026.05 | 76.26 | 51.62 | 67.26 | 74.76 | 64.4 | 66.86 | |
| RaZeRModel=Qwen3 4B, Quantization=W4A162026.05 | 75.93 | 51.45 | 67.54 | 75.19 | 63.77 | 66.78 | |
| BF16Backbone=LLaMA-3.2-3B, Evaluation Protocol=Zero-shot2026.01 | 74.5 | 42.2 | 55.2 | 76.8 | 69.1 | 63.6 | |
| HESTIABackbone=LLaMA-3.2-3B, Evaluation Protocol=Zero-shot2026.01 | 74.1 | 39.3 | 49.1 | 75.7 | 62.2 | 60.1 | |
| SOARModel=LLaMA-3.2 3B-Instruct, Quantization=W4A162026.05 | 71.42 | 45.73 | 71.3 | 76.22 | 68.75 | 66.68 | |
| FP16Model=LLaMA-3.2 3B-Instruct, Precision=FP162026.05 | 71.09 | 46.25 | 71.55 | 76.66 | 68.59 | 66.83 | |
| NVFP4Model=LLaMA-3.2 3B-Instruct, Quantization=W4A162026.05 | 70.62 | 45.13 | 71.24 | 76.5 | 67.64 | 66.23 | |
| 4over6Model=LLaMA-3.2 3B-Instruct, Quantization=W4A162026.05 | 70.54 | 44.8 | 70.84 | 76.5 | 68.19 | 66.17 | |
| RaZeRModel=LLaMA-3.2 3B-Instruct, Quantization=W4A162026.05 | 70.45 | 45.99 | 71.25 | 76.5 | 68.51 | 66.54 | |
| TequilaBackbone=LLaMA-3.2-3B, Evaluation Protocol=Zero-shot2026.01 | 70.2 | 34.6 | 46.4 | 73.9 | 62.7 | 57.6 | |
| TWNBackbone=LLaMA-3.2-3B, Evaluation Protocol=Zero-shot2026.01 | 69.2 | 35.1 | 46.2 | 73.4 | 58.6 | 56.5 | |
| INT2Backbone=LLaMA-3.2-3B, Evaluation Protocol=Zero-shot2026.01 | 68.7 | 34.8 | 42.3 | 72.7 | 55.8 | 54.9 | |
| AbsMeanBackbone=LLaMA-3.2-3B, Evaluation Protocol=Zero-shot2026.01 | 67.2 | 32.9 | 43.9 | 73.5 | 58.2 | 55.1 | |
| HESTIABackbone=LLaMA-3.2-1B, Evaluation Protocol=Zero-shot2026.01 | 66.5 | 35 | 41.8 | 72.6 | 57.7 | 54.7 | |
| BF16Backbone=LLaMA-3.2-1B, Evaluation Protocol=Zero-shot2026.01 | 65.4 | 31.3 | 47.7 | 74.2 | 60.3 | 55.8 | |
| TequilaBackbone=LLaMA-3.2-1B, Evaluation Protocol=Zero-shot2026.01 | 64.5 | 30.5 | 39.1 | 71 | 54.2 | 51.9 | |
| FP16Model=LLaMA-3.2 1B-Instruct, Precision=FP162026.05 | 63.72 | 37.54 | 61.59 | 74.92 | 61.64 | 59.88 | |
| SOARModel=LLaMA-3.2 1B-Instruct, Quantization=W4A162026.05 | 62.29 | 37.63 | 59.41 | 73.78 | 61.64 | 58.95 | |
| INT2Backbone=LLaMA-3.2-1B, Evaluation Protocol=Zero-shot2026.01 | 61.9 | 26.2 | 35.4 | 69.2 | 52.7 | 49.1 | |
| RaZeRModel=LLaMA-3.2 1B-Instruct, Quantization=W4A162026.05 | 61.7 | 37.54 | 60.02 | 73.83 | 62.19 | 59.06 | |
| NVFP4Model=LLaMA-3.2 1B-Instruct, Quantization=W4A162026.05 | 61.41 | 37.12 | 59.22 | 73.56 | 62.67 | 58.8 | |
| 4over6Model=LLaMA-3.2 1B-Instruct, Quantization=W4A162026.05 | 61.36 | 36.6 | 59.52 | 73.18 | 61.48 | 58.43 | |
| AbsMeanBackbone=LLaMA-3.2-1B, Evaluation Protocol=Zero-shot2026.01 | 60.3 | 25.9 | 36 | 68.3 | 54.1 | 48.9 | |
| SEQBackbone=LLaMA-3.2-3B, Evaluation Protocol=Zero-shot2026.01 | 49.8 | 23.1 | 30.3 | 64.5 | 52.9 | 44.1 | |
| LSQBackbone=LLaMA-3.2-3B, Evaluation Protocol=Zero-shot2026.01 | 43.1 | 20 | 29.4 | 59.9 | 52.2 | 40.9 | |
| DLTBackbone=LLaMA-3.2-1B, Evaluation Protocol=Zero-shot2026.01 | 42.4 | 17.4 | 25.6 | 56.3 | 51.3 | 38.6 | |
| SEQBackbone=LLaMA-3.2-1B, Evaluation Protocol=Zero-shot2026.01 | 42.1 | 18 | 27.3 | 60.4 | 51 | 39.8 | |
| TWNBackbone=LLaMA-3.2-1B, Evaluation Protocol=Zero-shot2026.01 | 40.7 | 22 | 28.4 | 60.1 | 49.2 | 40.1 | |
| LSQBackbone=LLaMA-3.2-1B, Evaluation Protocol=Zero-shot2026.01 | 37.6 | 17.7 | 25.8 | 57.4 | 50.6 | 37.8 | |
| DLTBackbone=LLaMA-3.2-3B, Evaluation Protocol=Zero-shot2026.01 | 36.1 | 16.1 | 26 | 57.2 | 49.6 | 37 |