Commonsense Reasoning on Commonsense Reasoning Task
53.63HellaSwag AccuracyKDA
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| KDANumber of Parameters=1.3B, Training Tokens=100B, Batch Size=1M2026.05 | 53.63 | 52.73 | 59.43 | 30.12 | 71.93 | 57.38 | 59.73 | 83.5 | 58.56 | |
| MDNNumber of Parameters=1.3B, Training Tokens=100B, Batch Size=1M2026.05 | 53.5 | 52.97 | 59.55 | 30.52 | 71.87 | 58.48 | 59.63 | 84 | 58.82 | |
| CombaNumber of Parameters=1.3B, Training Tokens=100B, Batch Size=1M2026.05 | 53.01 | 51 | 59.55 | 29.95 | 72.31 | 56.51 | 62.02 | 83.6 | 58.49 | |
| Mamba2Number of Parameters=1.3B, Training Tokens=100B, Batch Size=1M2026.05 | 52.69 | 49.66 | 58.88 | 29.01 | 71.11 | 54.54 | 60.49 | 80.4 | 57.1 | |
| TransformerNumber of Parameters=1.3B, Training Tokens=100B, Batch Size=1M2026.05 | 52.56 | 51.25 | 58.59 | 27.82 | 71.22 | 58.88 | 61.16 | 82.1 | 57.95 | |
| GDNNumber of Parameters=1.3B, Training Tokens=100B, Batch Size=1M2026.05 | 52.37 | 50.63 | 58.25 | 28.33 | 72.47 | 56.51 | 58.87 | 82.1 | 57.44 | |
| KDANumber of Parameters=400M, Training Tokens=15B, Batch Size=0.5M2026.05 | 35.95 | 36.62 | 47.14 | 23.89 | 65.79 | 53.28 | 56.57 | 73.2 | 49.06 | |
| CombaNumber of Parameters=400M, Training Tokens=15B, Batch Size=0.5M2026.05 | 35.78 | 34.31 | 47.05 | 24.66 | 65.78 | 51.54 | 58.32 | 73.8 | 48.91 | |
| MDNNumber of Parameters=400M, Training Tokens=15B, Batch Size=0.5M2026.05 | 35.6 | 37.43 | 46.93 | 25.17 | 66.43 | 50.28 | 59.25 | 74.3 | 49.42 | |
| Mamba2Number of Parameters=400M, Training Tokens=15B, Batch Size=0.5M2026.05 | 35.08 | 29.69 | 46.68 | 23.55 | 65.18 | 52.09 | 59.14 | 71.4 | 47.85 | |
| GDNNumber of Parameters=400M, Training Tokens=15B, Batch Size=0.5M2026.05 | 34.9 | 34.85 | 46.13 | 24.91 | 65.56 | 52.33 | 57.86 | 71.5 | 48.51 | |
| TransformerNumber of Parameters=400M, Training Tokens=15B, Batch Size=0.5M2026.05 | 34.4 | 33.24 | 45.62 | 24.23 | 64.42 | 52.17 | 59.48 | 70.9 | 48.06 |