General Language Understanding on AI2 OLMES (test)
40.4ARC-E AccuracyLooped-MoE
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Looped-MoEParams=216M, Training FLOPs=10^182026.05 | 40.4 | 24.3 | 63.9 | 30.9 | 25.4 | 26.8 | 55.2 | 38 | 51.7 | 39.6 | |
| BaseParams=246M, Training FLOPs=10^182026.05 | 39.3 | 25.3 | 51.4 | 29.3 | 26.2 | 27.6 | 57.8 | 40.5 | 50.5 | 38.7 | |
| LoopedParams=168M, Training FLOPs=10^182026.05 | 39.1 | 24.6 | 49.6 | 27.1 | 25.6 | 24.4 | 57.1 | 38.2 | 51.2 | 37.4 | |
| MoEParams=366M, Training FLOPs=10^182026.05 | 38.9 | 23 | 39 | 30 | 26.7 | 25.2 | 56.1 | 39.2 | 49.7 | 36.4 |