Grade School Math Reasoning on GSM8K (Avg.5)
61.33GSM8K AccuracyBaseline
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BaselineBase Model=Qwen1.5-MoE, Weight Bits=162026.04 | 61.33 | 46.1 | |
| BaselineBase Model=OLMoE, Weight Bits=162026.04 | 52.84 | 33.45 | |
| MoBiEBase Model=Qwen1.5-MoE, Weight Bits=1.352026.04 | 52.33 | 37.54 | |
| GPTQBase Model=Qwen1.5-MoE, Weight Bits=32026.04 | 52.01 | 37.48 | |
| GPTQBase Model=OLMoE, Weight Bits=32026.04 | 42.03 | 24.85 | |
| MoBiEBase Model=OLMoE, Weight Bits=1.462026.04 | 41.88 | 24.54 | |
| ARB-LLMBase Model=Qwen1.5-MoE, Weight Bits=1.112026.04 | 40.32 | 30.6 | |
| BiLLMBase Model=Qwen1.5-MoE, Weight Bits=1.112026.04 | 38.29 | 28.8 | |
| NoWagBase Model=Qwen1.5-MoE, Weight Bits=2.042026.04 | 35.54 | 26.94 | |
| ARB-LLMBase Model=OLMoE, Weight Bits=1.112026.04 | 30.02 | 16.97 | |
| BiLLMBase Model=OLMoE, Weight Bits=1.112026.04 | 27.21 | 14.85 | |
| MoEQuantBase Model=Qwen1.5-MoE, Weight Bits=22026.04 | 23.21 | 22.56 | |
| NoWagBase Model=OLMoE, Weight Bits=2.042026.04 | 21.14 | 11.77 | |
| MoEQuantBase Model=OLMoE, Weight Bits=22026.04 | 17.88 | 9.69 | |
| GPTQBase Model=Qwen1.5-MoE, Weight Bits=22026.04 | 15.42 | 16.96 | |
| AWQBase Model=Qwen1.5-MoE, Weight Bits=32026.04 | 14.79 | 15.1 | |
| GPTQBase Model=OLMoE, Weight Bits=22026.04 | 8.3 | 6.07 | |
| AWQBase Model=OLMoE, Weight Bits=32026.04 | 4.77 | 3.88 | |
| AWQBase Model=Qwen1.5-MoE, Weight Bits=22026.04 | 4.21 | 12.1 | |
| AWQBase Model=OLMoE, Weight Bits=22026.04 | 3.21 | 1.9 |