Zero-shot Commonsense Reasoning on Reasoning Suite (ARC-E, BoolQ, HSwag, LAMBADA, OBQA, PIQA, SocIQA, WinoGr.)
45.5ARC-E AccuracyPathMoE
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| PathMoERouting=Path-Constrained Routing, Throughput (k tok/s/GPU)=55.71, Memory (GiB)=66.742026.03 | 45.5 | 55.44 | 47.82 | 47.89 | 32.4 | 66.21 | 38.49 | 53.43 | 48.4 | |
| PathB4-MoERouting=Path-Constrained Routing, Throughput (k tok/s/GPU)=55.67, Memory (GiB)=66.832026.03 | 44.7 | 60.4 | 47.95 | 49.45 | 31.6 | 66.32 | 40.94 | 55.64 | 49.62 | |
| Indep-MoERouting=Independent Routing, Throughput (k tok/s/GPU)=55.21, Memory (GiB)=66.942026.03 | 44.57 | 56.45 | 45.99 | 46.19 | 29.8 | 66.87 | 38.84 | 51.54 | 47.53 | |
| LowRank-MoERouting=Path-Constrained Routing, Throughput (k tok/s/GPU)=53.84, Memory (GiB)=66.962026.03 | 44.36 | 59.97 | 46.33 | 46.11 | 31.8 | 66.65 | 39.2 | 52.96 | 48.42 | |
| Recurrent-MoERouting=Recurrent Routing, Throughput (k tok/s/GPU)=54.65, Memory (GiB)=69.262026.03 | 44.07 | 54.5 | 46.05 | 47.12 | 30.8 | 67.74 | 38.79 | 53.67 | 47.84 | |
| PathB8-MoERouting=Path-Constrained Routing, Throughput (k tok/s/GPU)=55.20, Memory (GiB)=66.942026.03 | 43.73 | 59.97 | 46.65 | 46.92 | 31.4 | 66.81 | 40.69 | 54.93 | 48.89 | |
| X-MoERouting=Independent Routing, Throughput (k tok/s/GPU)=56.37, Memory (GiB)=67.062026.03 | 43.27 | 60.55 | 46.26 | 45.22 | 32.2 | 67.63 | 39.61 | 52.8 | 48.44 | |
| MonoB8-MoERouting=Path-Constrained Routing, Throughput (k tok/s/GPU)=56.53, Memory (GiB)=62.862026.03 | 42.55 | 57.8 | 46.36 | 46.73 | 31.2 | 65.67 | 37.56 | 54.06 | 47.74 | |
| Rand-MoERouting=Independent Routing, Throughput (k tok/s/GPU)=75.17, Memory (GiB)=47.492026.03 | 40.49 | 61.1 | 37.95 | 38.83 | 28.2 | 64.47 | 38.89 | 50.99 | 45.12 |