Zero-shot Evaluation on LAMBDA, RACE, ARC-E, and ARC-C
61.73LAMBDA ScoreReMoE
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| ReMoEExperts=64, K=8, Sparsity-Ratio=1/8, Model Size=182M, Evaluation Protocol=Zero-shot2025.11 | 61.73 | 28.15 | 49.12 | 19.06 | 39.52 | |
| SeqTopKExperts=128, K=4, Sparsity-Ratio=1/32, Model Size=182M, Evaluation Protocol=Zero-shot2025.11 | 60.82 | 28.7 | 48.07 | 23.41 | 40.25 | |
| SeqTopKExperts=64, K=8, Sparsity-Ratio=1/8, Model Size=182M, Evaluation Protocol=Zero-shot2025.11 | 60.67 | 28.55 | 48.42 | 22.36 | 40 | |
| TopKExperts=64, K=8, Sparsity-Ratio=1/8, Model Size=182M, Evaluation Protocol=Zero-shot2025.11 | 60.15 | 27.61 | 47.19 | 20.06 | 38.75 | |
| ReMoEExperts=128, K=4, Sparsity-Ratio=1/32, Model Size=182M, Evaluation Protocol=Zero-shot2025.11 | 59.94 | 28.01 | 47.71 | 21.07 | 39.18 | |
| TopKExperts=128, K=4, Sparsity-Ratio=1/32, Model Size=182M, Evaluation Protocol=Zero-shot2025.11 | 59.3 | 27.33 | 46.66 | 20.73 | 38.51 |