List your company
ResearchFormerly WizWand

Question Answering on PIQA (test)

85AccuracyExpanded Drop
23.920839.777955.63571.4921Mar 7, 2025May 14, 2025Jul 21, 2025Sep 27, 2025Dec 4, 2025Feb 10, 2026Apr 19, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Expanded DropModel=Mixtral-8x7B-Instruct, Capacity Factor=1.52025.0385
BaselineModel=Mixtral-8x7B-Instruct, Capacity Factor=1.52025.0384.8
Token DropModel=Mixtral-8x7B-Instruct, Capacity Factor=1.52025.0383.3
Expert DropModel=Mixtral-8x7B-Instruct, Capacity Factor=1.52025.0383.2
BaselineModel=DeepSeek-V2-Lite-Chat, Capacity Factor=2.02025.0381.4
BaselineModel=OLMoE-Instruct, Capacity Factor=2.02025.0380.2
BaselineModel=Qwen1.5-MoE-Chat, Capacity Factor=1.52025.0379.9
Expanded DropModel=OLMoE-Instruct, Capacity Factor=2.02025.0379.4
Expanded DropModel=DeepSeek-V2-Lite-Chat, Capacity Factor=2.02025.0379.4
Expanded DropModel=Qwen1.5-MoE-Chat, Capacity Factor=1.52025.0379.1
Token DropModel=Qwen1.5-MoE-Chat, Capacity Factor=1.52025.0378.8
Expert DropModel=Qwen1.5-MoE-Chat, Capacity Factor=1.52025.0378.7
Token DropModel=DeepSeek-V2-Lite-Chat, Capacity Factor=2.02025.0378.3
Token DropModel=OLMoE-Instruct, Capacity Factor=2.02025.0377.9
OracleModel Size=Large2026.0477.8
OracleModel Size=Large, Noise Type=Uniform2026.0477.8
REALMModel Size=Large, Expertise Distribution=Dist. 2, Noise Type=Uniform2026.0477.67
Expert DropModel=DeepSeek-V2-Lite-Chat, Capacity Factor=2.02025.0377.6
Expert DropModel=OLMoE-Instruct, Capacity Factor=2.02025.0376.9
REALMModel Size=Large, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.0476.5
REALMModel Size=Large, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.0476.5
REALMModel Size=Large, Expertise Distribution=Dist. 1, Noise Type=Uniform2026.0476.45
NOISYModel Size=Large, Expertise Distribution=Dist. 2, Noise Type=Uniform2026.0475.1
OracleModel Size=Base, Noise Type=Uniform2026.0467.68
REALMModel Size=Base, Expertise Distribution=Dist. 2, Noise Type=Uniform2026.0466.55
REALMModel Size=Base, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.0465.79
REALMModel Size=Base, Expertise Distribution=Dist. 1, Noise Type=Uniform2026.0465.7
NOISYModel Size=Base, Expertise Distribution=Dist. 2, Noise Type=Uniform2026.0463.33
OracleModel Size=Small, Noise Type=Uniform2026.0452.88
REALMModel Size=Small, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.0452.88
REALMModel Size=Small, Expertise Distribution=Dist. 1, Noise Type=Uniform2026.0452.14
noisyModel Size=Large, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.0451.99
NOISYModel Size=Large, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.0451.99
REALMModel Size=Small, Expertise Distribution=Dist. 2, Noise Type=Uniform2026.0451.6
NOISYModel Size=Base, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.0451.47
NOISYModel Size=Small, Expertise Distribution=Dist. 2, Noise Type=Uniform2026.0451.12
NOISYModel Size=Small, Expertise Distribution=Dist. 1, Noise Type=Uniform2026.0451.08
NOISYModel Size=Small, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.0450.36
NOISYModel Size=Base, Expertise Distribution=Dist. 1, Noise Type=Uniform2026.0446.31
NOISYModel Size=Large, Expertise Distribution=Dist. 1, Noise Type=Uniform2026.0426.27