Commonsense Reasoning and Knowledge Understanding on ARC, HellaSwag, LAMBADA, PIQA, WinoGrande, and MMLU
83.42ARC-e AccuracyQwen3-8B
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Qwen3-8BArchitecture=Transformer, Training Tokens (B)=360002025.10 | 83.42 | 56.74 | 74.9 | 61.11 | 76.61 | 68.11 | 74.93 | 70.83 | |
| FalconMamba-7BArchitecture=Linear/Subquadratic, Training Tokens (B)=55002025.10 | 83.33 | 58.45 | 80.22 | 61.6 | 80.2 | 75.53 | 60.27 | 71.37 | |
| NHA-Qwen2.5-7BArchitecture=Hybrid, Training Tokens (B)=10, Full-attention layers=42025.10 | 82.74 | 55.97 | 76.49 | 64.54 | 78.73 | 71.03 | 68.8 | 71.19 | |
| Mistral-7BArchitecture=Transformer, Training Tokens (B)=80002025.10 | 80.85 | 54.01 | 81.08 | 69.49 | 80.9 | 73.64 | 62.5 | 71.78 | |
| NHA-Llama-3-8BArchitecture=Hybrid, Training Tokens (B)=10, Full-attention layers=42025.10 | 80.76 | 52.47 | 78.93 | 67.26 | 79.71 | 72.85 | 60.21 | 70.31 | |
| Qwen2.5-7BArchitecture=Transformer, Training Tokens (B)=180002025.10 | 80.43 | 51.45 | 78.93 | 64.7 | 78.78 | 73.09 | 74.18 | 71.65 | |
| Llama-3-8BArchitecture=Transformer, Training Tokens (B)=150002025.10 | 80.3 | 53.07 | 79.14 | 68.89 | 79.6 | 72.77 | 65.34 | 71.3 | |
| Zamba2-7BArchitecture=Hybrid, Training Tokens (B)=2100, Full-attention layers=92025.10 | 80.13 | 56.23 | 81.52 | 59.09 | 79.05 | 77.19 | 67.27 | 71.5 | |
| StripedHyena-7BArchitecture=Hybrid, Full-attention layers=162025.10 | 78.62 | 53.75 | 79.11 | 60.78 | 80.03 | 70.32 | 54.07 | 68.1 | |
| Mamba-7BArchitecture=Linear/Subquadratic, Training Tokens (B)=12002025.10 | 77.61 | 46.84 | 77.93 | 65.53 | 79.87 | 71.74 | 33.19 | 64.67 | |
| Griffin-7BArchitecture=Linear/Subquadratic, Training Tokens (B)=3002025.10 | 75.4 | 47.9 | 78.6 | — | 81 | 72.6 | 39.3 | — | |
| RWKV-6-World-7BArchitecture=Linear/Subquadratic, Training Tokens (B)=14202025.10 | 73.61 | 43.86 | 75.17 | 69.26 | 78.35 | 67.72 | 43.39 | 64.48 | |
| Zamba-7BArchitecture=Hybrid, Training Tokens (B)=1000, Full-attention layers=132025.10 | 71.72 | 45.56 | 80.81 | 64.84 | 80.41 | 72.14 | 57.68 | 67.59 |