Commonsense Reasoning on PIQA, ARC-e, ARC-c, HellaSwag, and WinoGrande
74.1PIQA AccuracyBitNetV2-7B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| BitNetV2-7B#Tokens=100B, Quantization=W1.58A82026.06 | 74.1 | 58.54 | 32.94 | 61.08 | 61.48 | 57.63 | |
| FlashMorphModel Scale=1.7B, Attention Type=GLA, LS. Tokens=20M2026.06 | 73.3 | 74.5 | 44.7 | 61.2 | 62.3 | 63.2 | |
| FlashMorphModel Scale=1.7B, Attention Type=GDN, LS. Tokens=20M2026.06 | 73.3 | 74.4 | 44.1 | 61.5 | 63.9 | 63.4 | |
| FlashMorphModel Scale=1.7B, Attention Type=Lightning, LS. Tokens=20M2026.06 | 73.2 | 73.2 | 42.3 | 60.6 | 61 | 62.1 | |
| PostNAS*Model Scale=1.7B, Attention Type=GLA, LS. Tokens=50B2026.06 | 73.2 | 73.4 | 43.1 | 61.5 | 60.1 | 62.3 | |
| HALOModel Scale=1.7B, Attention Type=GDN, LS. Tokens=234M2026.06 | 73.2 | 75.8 | 45.7 | 61.1 | 63.1 | 63.8 | |
| PostNAS*Model Scale=1.7B, Attention Type=GDN, LS. Tokens=50B2026.06 | 73.1 | 75.1 | 45.1 | 61.7 | 62.3 | 63.4 | |
| PostNAS*Model Scale=1.7B, Attention Type=Lightning, LS. Tokens=50B2026.06 | 72.9 | 74.7 | 43 | 60.8 | 60.9 | 62.5 | |
| KL-LSModel Scale=1.7B, Attention Type=GDN, LS. Tokens=20B2026.06 | 72.9 | 74.3 | 47.6 | 61.6 | 62.3 | 63.7 | |
| UniformModel Scale=1.7B, Attention Type=GLA, LS. Tokens=N/A2026.06 | 72.7 | 74.4 | 45.5 | 60.8 | 63 | 63.3 | |
| HALOModel Scale=1.7B, Attention Type=Lightning, LS. Tokens=234M2026.06 | 72.6 | 73.5 | 42.8 | 60.6 | 61.5 | 62.2 | |
| HALOModel Scale=1.7B, Attention Type=GLA, LS. Tokens=234M2026.06 | 72.6 | 74.4 | 44.5 | 60.9 | 62.8 | 63 | |
| Qwen3Model Scale=1.7B, Attention Type=Full2026.06 | 72.5 | 72.4 | 43.5 | 60.4 | 61 | 62 | |
| KL-LSModel Scale=1.7B, Attention Type=GLA, LS. Tokens=20B2026.06 | 72.4 | 74.6 | 46.3 | 61 | 59.8 | 62.8 | |
| UniformModel Scale=1.7B, Attention Type=Lightning, LS. Tokens=N/A2026.06 | 72.3 | 74.7 | 45.1 | 60.5 | 62.2 | 62.9 | |
| KL-LSModel Scale=1.7B, Attention Type=Lightning, LS. Tokens=20B2026.06 | 72.3 | 74.2 | 42.8 | 60.4 | 59.1 | 61.8 | |
| UniformModel Scale=1.7B, Attention Type=GDN, LS. Tokens=N/A2026.06 | 72.3 | 74.3 | 44.5 | 61.5 | 62.4 | 63 | |
| Qwen3-8B + CAT-Q#Tokens=1M, Quantization=W1.58A82026.06 | 72.09 | 68.94 | 42.49 | 58.64 | 62.67 | 60.96 | |
| BitNetV2-3B#Tokens=100B, Quantization=W1.58A82026.06 | 71.33 | 55.56 | 30.55 | 57.19 | 58.72 | 54.67 | |
| BitNetV2-1.3B#Tokens=100B, Quantization=W1.58A82026.06 | 69.42 | 49.96 | 27.9 | 48.37 | 57.22 | 50.57 | |
| Qwen3-4B + CAT-Q#Tokens=1M, Quantization=W1.58A82026.06 | 68.65 | 62.64 | 35.69 | 51.48 | 63.51 | 56.39 | |
| Qwen3-1.7B + CAT-Q#Tokens=1M, Quantization=W1.58A82026.06 | 68.16 | 54.94 | 28.67 | 47.22 | 54.62 | 50.72 | |
| KL-LSModel Scale=0.6B, Attention Type=GDN, LS. Tokens=20B2026.06 | 67.9 | 61.1 | 35.3 | 47.4 | 56.7 | 53.7 | |
| FlashMorphModel Scale=0.6B, Attention Type=GDN, LS. Tokens=20M2026.06 | 67.8 | 63.1 | 33.5 | 47.5 | 56.4 | 53.7 | |
| HALOModel Scale=0.6B, Attention Type=GDN, LS. Tokens=234M2026.06 | 67.7 | 60.1 | 34.1 | 47.5 | 55.3 | 53 | |
| Qwen3Model Scale=0.6B, Attention Type=Full2026.06 | 67.6 | 60.6 | 34.1 | 47.3 | 55.7 | 53.1 | |
| KL-LSModel Scale=0.6B, Attention Type=Lightning, LS. Tokens=20B2026.06 | 67.5 | 62.1 | 33 | 46.2 | 54.9 | 52.7 | |
| FlashMorphModel Scale=0.6B, Attention Type=GLA, LS. Tokens=20M2026.06 | 67.5 | 63.9 | 32.9 | 46.9 | 54.1 | 53.1 | |
| HALOModel Scale=0.6B, Attention Type=Lightning, LS. Tokens=234M2026.06 | 67.4 | 63.5 | 32.4 | 46.4 | 56.1 | 53.2 | |
| HALOModel Scale=0.6B, Attention Type=GLA, LS. Tokens=234M2026.06 | 67.4 | 62.9 | 32.6 | 46.9 | 55.1 | 53 | |
| FlashMorphModel Scale=0.6B, Attention Type=Lightning, LS. Tokens=20M2026.06 | 67.3 | 62.8 | 32 | 46.4 | 55.2 | 52.7 | |
| UniformModel Scale=0.6B, Attention Type=GLA, LS. Tokens=N/A2026.06 | 67.3 | 63.1 | 33.1 | 46.8 | 55.3 | 53.1 | |
| KL-LSModel Scale=0.6B, Attention Type=GLA, LS. Tokens=20B2026.06 | 67.2 | 61.8 | 33.8 | 46.9 | 54.9 | 52.9 | |
| UniformModel Scale=0.6B, Attention Type=GDN, LS. Tokens=N/A2026.06 | 67.1 | 59.6 | 32.3 | 47.6 | 55.3 | 52.4 | |
| UniformModel Scale=0.6B, Attention Type=Lightning, LS. Tokens=N/A2026.06 | 66.9 | 62.3 | 32.9 | 46.1 | 55.6 | 52.8 |