Zero-shot Downstream Evaluation on ARC, PIQA, HellaSwag, WinoGrande, LAMBADA, RACE
56.57ARC-E AccuracyUNIPOOL
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| UNIPOOLSetting=8E / top-1, Scale=830M, Zero-shot=true2026.05 | 56.57 | 25 | 68.77 | 36.9 | 52.49 | 47.37 | 32.63 | 45.67 | |
| UNIPOOLSetting=8E / top-1, Scale=978M, Zero-shot=true2026.05 | 54.34 | 22.27 | 69.21 | 36.19 | 52.17 | 44.94 | 29.38 | 44.07 | |
| Vanilla MoESetting=8E / top-1, Scale=978M, Zero-shot=true2026.05 | 53.24 | 23.21 | 68.01 | 35.83 | 52.01 | 44.63 | 30.43 | 43.91 | |
| UNIPOOLSetting=8E / top-1, Scale=469M, Zero-shot=true2026.05 | 53.16 | 21.42 | 67.3 | 33.9 | 52.72 | 42.19 | 31.1 | 43.11 | |
| Vanilla MoESetting=8E / top-1, Scale=830M, Zero-shot=true2026.05 | 52.53 | 23.89 | 68.93 | 35.36 | 52.33 | 43.14 | 30.53 | 43.82 | |
| UNIPOOLSetting=32E / top-4, Scale=182M, Zero-shot=true2026.05 | 52.44 | 22.27 | 67.41 | 34.32 | 50.51 | 40.77 | 30.62 | 42.62 | |
| UNIPOOLSetting=8E / top-1, Scale=650M, Zero-shot=true2026.05 | 52.02 | 22.61 | 67.9 | 35.55 | 52.49 | 44.28 | 31.67 | 43.79 | |
| Vanilla MoESetting=8E / top-1, Scale=650M, Zero-shot=true2026.05 | 51.94 | 21.25 | 67.03 | 34.53 | 53.04 | 43.74 | 29.76 | 43.04 | |
| Vanilla MoESetting=8E / top-1, Scale=469M, Zero-shot=true2026.05 | 50.51 | 21.08 | 66.32 | 32.72 | 51.14 | 40.21 | 29.38 | 41.62 | |
| Vanilla MoESetting=32E / top-4, Scale=182M, Zero-shot=true2026.05 | 50.08 | 21.08 | 66.43 | 32.91 | 51.54 | 39.41 | 29 | 41.49 | |
| UNIPOOLSetting=16E / top-2, Scale=182M, Zero-shot=true2026.05 | 49.24 | 20.22 | 65.45 | 32.33 | 54.22 | 37.86 | 29.19 | 41.22 | |
| Vanilla MoESetting=16E / top-2, Scale=182M, Zero-shot=true2026.05 | 48.82 | 21.59 | 65.07 | 31.83 | 49.72 | 36.48 | 28.8 | 40.33 | |
| UNIPOOLSetting=8E / top-1, Scale=182M, Zero-shot=true2026.05 | 46.72 | 20.48 | 64.36 | 30.66 | 50.99 | 34.56 | 29.47 | 39.61 | |
| Vanilla MoESetting=8E / top-1, Scale=182M, Zero-shot=true2026.05 | 45.71 | 19.97 | 63.11 | 29.98 | 50.99 | 32.78 | 28.61 | 38.74 |