Zero-shot Common Sense Reasoning Suite (PIQA, ARC-e, ARC-c, BoolQ, HellaSwag, Winogrande) (test dev)
79.35PIQAKirin
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| KirinBackbone=Llama2-13B, Evaluation Protocol=Zero-shot2026.02 | 79.35 | 56.4 | 43.94 | 66.21 | 75.94 | 68.98 | 65.14 | |
| SpikeQuantBackbone=Llama2-13B, Evaluation Protocol=Zero-shot2026.02 | 79.33 | 56.48 | 44.03 | 66.15 | 75.53 | 68.11 | 64.94 | |
| Fp16Backbone=Llama2-13B, Evaluation Protocol=Zero-shot2026.02 | 79.05 | 59.85 | 44.62 | 68.53 | 76.22 | 70.09 | 66.39 | |
| AtomBackbone=Llama2-13B, Evaluation Protocol=Zero-shot2026.02 | 77.69 | 57.58 | 42.92 | 67.46 | 73.77 | 68.51 | 64.66 | |
| Fp16Backbone=Llama2-7B, Evaluation Protocol=Zero-shot2026.02 | 77.37 | 52.53 | 41.38 | 73.12 | 72.99 | 66.85 | 64.04 | |
| SpikeQuantBackbone=Llama2-7B, Evaluation Protocol=Zero-shot2026.02 | 77.09 | 55.39 | 40.36 | 73.49 | 72.09 | 65.27 | 63.95 | |
| KirinBackbone=Llama2-7B, Evaluation Protocol=Zero-shot2026.02 | 76.77 | 54.55 | 40.53 | 73.91 | 72.1 | 66.22 | 64.01 | |
| QuantizationBackbone=Llama2-13B, Evaluation Protocol=Zero-shot2026.02 | 76.71 | 55.13 | 41.47 | 67.03 | 73.09 | 65.19 | 63.1 | |
| AtomBackbone=Llama2-7B, Evaluation Protocol=Zero-shot2026.02 | 76.28 | 52.1 | 38.99 | 69.79 | 69.81 | 63.69 | 61.78 | |
| QuantizationBackbone=Llama2-7B, Evaluation Protocol=Zero-shot2026.02 | 75.46 | 53.96 | 40.19 | 68.07 | 69.21 | 61.56 | 61.41 | |
| Fp16Backbone=Opt-2.7B, Evaluation Protocol=Zero-shot2026.02 | 74.81 | 54.34 | 31.31 | 60.28 | 60.59 | 60.93 | 57.04 | |
| KirinBackbone=Opt-2.7B, Evaluation Protocol=Zero-shot2026.02 | 74.81 | 53.91 | 31.48 | 61.69 | 58.83 | 59.91 | 56.77 | |
| SpikeQuantBackbone=Opt-2.7B, Evaluation Protocol=Zero-shot2026.02 | 74.21 | 53.79 | 31.66 | 61.19 | 59.11 | 59.75 | 56.62 | |
| Fp16Backbone=Opt-1.3B, Evaluation Protocol=Zero-shot2026.02 | 72.42 | 50.8 | 29.69 | 57.68 | 53.73 | 59.67 | 54 | |
| KirinBackbone=Opt-1.3B, Evaluation Protocol=Zero-shot2026.02 | 71.86 | 48.78 | 28.67 | 53.66 | 52.25 | 57.77 | 52.17 | |
| SpikeQuantBackbone=Opt-1.3B, Evaluation Protocol=Zero-shot2026.02 | 71.49 | 49.66 | 27.22 | 53.27 | 51.46 | 58.56 | 51.94 | |
| OPTQBackbone=Opt-2.7B, Evaluation Protocol=Zero-shot2026.02 | 71.38 | 48.19 | 27.82 | 55.43 | 49.86 | 54.82 | 51.25 | |
| QuantizationBackbone=Opt-2.7B, Evaluation Protocol=Zero-shot2026.02 | 71.16 | 50.84 | 28.75 | 59.85 | 55.57 | 60.46 | 54.44 | |
| OmniQuantBackbone=Llama2-13B, Evaluation Protocol=Zero-shot2026.02 | 69.69 | 47.39 | 33.1 | 62.84 | 58.96 | 55.8 | 54.63 | |
| OPTQBackbone=Opt-1.3B, Evaluation Protocol=Zero-shot2026.02 | 68.34 | 46.17 | 27.65 | 51.57 | 46.53 | 55.26 | 49.25 | |
| QuantizationBackbone=Opt-1.3B, Evaluation Protocol=Zero-shot2026.02 | 67.79 | 42.93 | 25.26 | 49.51 | 45.94 | 54.93 | 47.73 | |
| SpikeLLMBackbone=Llama2-13B, Evaluation Protocol=Zero-shot2026.02 | 66.49 | 55.3 | 30.12 | 64.16 | 47.43 | 51.54 | 52.51 | |
| OmniQuantBackbone=Llama2-7B, Evaluation Protocol=Zero-shot2026.02 | 66.15 | 45.2 | 31.14 | 63.51 | 56.44 | 53.43 | 52.65 | |
| SpikeLLMBackbone=Llama2-7B, Evaluation Protocol=Zero-shot2026.02 | 64.47 | 48.74 | 27.3 | 63.27 | 43.29 | 56.83 | 50.65 | |
| SmoothQuantBackbone=Llama2-13B, Evaluation Protocol=Zero-shot2026.02 | 64.47 | 41.75 | 30.89 | 62.29 | 46.68 | 51.7 | 49.63 | |
| LRQuantBackbone=Opt-2.7B, Evaluation Protocol=Zero-shot2026.02 | 64.04 | 47.81 | 20.65 | 54.1 | 33.59 | 53.51 | 45.62 | |
| OneBitBackbone=Opt-2.7B, Evaluation Protocol=Zero-shot2026.02 | 63.87 | 43.39 | 24.4 | 54.28 | 38.18 | 51.67 | 45.97 | |
| SmoothQuantBackbone=Llama2-7B, Evaluation Protocol=Zero-shot2026.02 | 63.11 | 40.03 | 31.57 | 58.47 | 43.38 | 52.8 | 48.23 | |
| OneBitBackbone=Opt-1.3B, Evaluation Protocol=Zero-shot2026.02 | 62.57 | 41.25 | 24.06 | 59.45 | 34.26 | 51.14 | 45.46 | |
| LRQuantBackbone=Opt-1.3B, Evaluation Protocol=Zero-shot2026.02 | 60.23 | 43.22 | 19.8 | 57.98 | 30.71 | 51.78 | 43.95 |