Question Answering on PIQA (test)
85AccuracyExpanded Drop
Evaluation Results
| Method | Links | |
|---|---|---|
| Expanded DropModel=Mixtral-8x7B-Instruct, Capacity Factor=1.52025.03 | 85 | |
| BaselineModel=Mixtral-8x7B-Instruct, Capacity Factor=1.52025.03 | 84.8 | |
| Token DropModel=Mixtral-8x7B-Instruct, Capacity Factor=1.52025.03 | 83.3 | |
| Expert DropModel=Mixtral-8x7B-Instruct, Capacity Factor=1.52025.03 | 83.2 | |
| BaselineModel=DeepSeek-V2-Lite-Chat, Capacity Factor=2.02025.03 | 81.4 | |
| BaselineModel=OLMoE-Instruct, Capacity Factor=2.02025.03 | 80.2 | |
| BaselineModel=Qwen1.5-MoE-Chat, Capacity Factor=1.52025.03 | 79.9 | |
| Expanded DropModel=OLMoE-Instruct, Capacity Factor=2.02025.03 | 79.4 | |
| Expanded DropModel=DeepSeek-V2-Lite-Chat, Capacity Factor=2.02025.03 | 79.4 | |
| Expanded DropModel=Qwen1.5-MoE-Chat, Capacity Factor=1.52025.03 | 79.1 | |
| Token DropModel=Qwen1.5-MoE-Chat, Capacity Factor=1.52025.03 | 78.8 | |
| Expert DropModel=Qwen1.5-MoE-Chat, Capacity Factor=1.52025.03 | 78.7 | |
| Token DropModel=DeepSeek-V2-Lite-Chat, Capacity Factor=2.02025.03 | 78.3 | |
| Token DropModel=OLMoE-Instruct, Capacity Factor=2.02025.03 | 77.9 | |
| OracleModel Size=Large2026.04 | 77.8 | |
| OracleModel Size=Large, Noise Type=Uniform2026.04 | 77.8 | |
| REALMModel Size=Large, Expertise Distribution=Dist. 2, Noise Type=Uniform2026.04 | 77.67 | |
| Expert DropModel=DeepSeek-V2-Lite-Chat, Capacity Factor=2.02025.03 | 77.6 | |
| Expert DropModel=OLMoE-Instruct, Capacity Factor=2.02025.03 | 76.9 | |
| REALMModel Size=Large, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.04 | 76.5 | |
| REALMModel Size=Large, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.04 | 76.5 | |
| REALMModel Size=Large, Expertise Distribution=Dist. 1, Noise Type=Uniform2026.04 | 76.45 | |
| NOISYModel Size=Large, Expertise Distribution=Dist. 2, Noise Type=Uniform2026.04 | 75.1 | |
| OracleModel Size=Base, Noise Type=Uniform2026.04 | 67.68 | |
| REALMModel Size=Base, Expertise Distribution=Dist. 2, Noise Type=Uniform2026.04 | 66.55 | |
| REALMModel Size=Base, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.04 | 65.79 | |
| REALMModel Size=Base, Expertise Distribution=Dist. 1, Noise Type=Uniform2026.04 | 65.7 | |
| NOISYModel Size=Base, Expertise Distribution=Dist. 2, Noise Type=Uniform2026.04 | 63.33 | |
| OracleModel Size=Small, Noise Type=Uniform2026.04 | 52.88 | |
| REALMModel Size=Small, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.04 | 52.88 | |
| REALMModel Size=Small, Expertise Distribution=Dist. 1, Noise Type=Uniform2026.04 | 52.14 | |
| noisyModel Size=Large, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.04 | 51.99 | |
| NOISYModel Size=Large, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.04 | 51.99 | |
| REALMModel Size=Small, Expertise Distribution=Dist. 2, Noise Type=Uniform2026.04 | 51.6 | |
| NOISYModel Size=Base, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.04 | 51.47 | |
| NOISYModel Size=Small, Expertise Distribution=Dist. 2, Noise Type=Uniform2026.04 | 51.12 | |
| NOISYModel Size=Small, Expertise Distribution=Dist. 1, Noise Type=Uniform2026.04 | 51.08 | |
| NOISYModel Size=Small, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.04 | 50.36 | |
| NOISYModel Size=Base, Expertise Distribution=Dist. 1, Noise Type=Uniform2026.04 | 46.31 | |
| NOISYModel Size=Large, Expertise Distribution=Dist. 1, Noise Type=Uniform2026.04 | 26.27 |