Multiple-choice QA on PIQA, OpenBookQA, HellaSwag, and ARC
68.39PIQA AccuracyNexusFormer
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| NexusFormerSize=640M2026.04 | 68.39 | 33.6 | 45 | 59.67 | 30.72 | 47.48 | |
| TokenFormerSize=640M2026.04 | 66.53 | 34 | 39.93 | 51.51 | 27.9 | 43.97 | |
| NexusFormerSize=240M2026.04 | 66.27 | 33 | 39.89 | 50.8 | 28.67 | 43.73 | |
| TokenFormerSize=240M2026.04 | 65.99 | 32.6 | 39.94 | 52.35 | 28.67 | 43.91 | |
| TokenFormerSize=170M2026.04 | 65.2 | 32.6 | 38.4 | 46.8 | 27.2 | 42.04 | |
| NexusFormerSize=480M2026.04 | 64.47 | 30.6 | 41.41 | 50.97 | 27.73 | 43.04 | |
| NexusFormerSize=170M2026.04 | 64.42 | 32.4 | 35.53 | 49.33 | 26.88 | 41.71 | |
| Qwen3Size=640M2026.04 | 60.34 | 29 | 29.72 | 44.74 | 24.83 | 37.73 | |
| Qwen3Size=480M2026.04 | 59.74 | 30.4 | 32.18 | 42.97 | 25.34 | 38.13 | |
| TokenFormerSize=480M2026.04 | 56.91 | 28.8 | 31.53 | 40.36 | 25.93 | 36.71 | |
| Qwen3Size=170M2026.04 | 53.32 | 25 | 26.42 | 34.89 | 22.44 | 32.41 | |
| Transformer++Size=240M2026.04 | 51.9 | 28 | 26.27 | 27.57 | 26.19 | 31.99 | |
| Qwen3Size=240M2026.04 | 51.41 | 25 | 26.26 | 32.83 | 23.81 | 31.86 | |
| Transformer++Size=640M2026.04 | 51.14 | 25.4 | 25.9 | 28.2 | 24.2 | 30.97 | |
| Transformer++Size=480M2026.04 | 51.03 | 27.4 | 27.28 | 29.29 | 24.14 | 31.83 | |
| Transformer++Size=170M2026.04 | 50.82 | 26 | 25.5 | 27.57 | 26.88 | 31.35 |