Expert Routing Consistency Analysis on 11 domains
78.16SRPLLaMA-MoE-v2
Evaluation Results
| Method | Links | |
|---|---|---|
| LLaMA-MoE-v2MoE=all, A:T=1:4, S:A=02025.05 | 78.16 | |
| Yuan2.0MoE=all, A:T=1:16, S:A=02025.05 | 63.48 | |
| PowerMoEMoE=all, A:T=1:5, S:A=02025.05 | 55.17 | |
| Qwen3MoE=all, A:T=1:16, S:A=02025.05 | 54.14 | |
| Phi-3.5-MoEMoE=all, A:T=1:8, S:A=02025.05 | 51.98 | |
| OLMoEMoE=all, A:T=1:8, S:A=02025.05 | 50.91 | |
| GRIN-MoEMoE=all, A:T=1:8, S:A=02025.05 | 50.39 | |
| Mixtral-8x7BMoE=all, A:T=1:4, S:A=02025.05 | 49.36 | |
| MiniCPM-MoEMoE=all, A:T=1:4, S:A=02025.05 | 48.85 | |
| JetMoEMoE=all, A:T=1:4, S:A=02025.05 | 47.45 | |
| LLaMA-MoE-v1MoE=all, A:T=1:4, S:A=02025.05 | 45.29 | |
| XVERSE-MoEMoE=all, A:T=3:32, S:A=1:32025.05 | 38.58 | |
| Jamba-MiniMoE=every 2, A:T=1:8, S:A=02025.05 | 38.08 | |
| DeepSeek-V2-LiteMoE=after 1st, A:T=3:32, S:A=1:32025.05 | 37.92 | |
| DeepSeekMoEMoE=after 1st, A:T=3:32, S:A=1:32025.05 | 36.94 | |
| Qwen2MoE=all, A:T=1:8, S:A=1:12025.05 | 36.74 | |
| NLLB-MoE (decoder)MoE=every 4, A:T=1:64, S:A=02025.05 | 31.35 | |
| Qwen1.5-MoEMoE=all, A:T=1:15, S:A=1:12025.05 | 30.71 | |
| OpenMoEMoE=every 6, A:T=1:16, S:A=1:22025.05 | 28.77 | |
| NLLB-MoE (encoder)MoE=every 4, A:T=1:64, S:A=02025.05 | 25.24 | |
| SwitchTF (encoder)MoE=every 2, A:T=1:128, S:A=02025.05 | 19.33 | |
| SwitchTF (decoder)MoE=every 2, A:T=1:128, S:A=02025.05 | 19.27 |