Mathematical Reasoning on MATH 500 (Speedup, Acceptance Length)
7.64SpeedupDraft-OPD
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Draft-OPDModel=Q3-8B, Thinking Mode=Disabled, Temperature=02026.05 | 7.64 | 6.57 | |
| Draft-OPDModel=Q3-4B, Thinking Mode=Disabled, Temperature=02026.05 | 7.22 | 6.6 | |
| DFlashModel=Q3-8B, Thinking Mode=Disabled, Temperature=02026.05 | 6.81 | 6.04 | |
| Draft-OPDModel=Q3-8B, Thinking Mode=Disabled, Temperature=0.62026.05 | 6.61 | 6.02 | |
| DFlashModel=Q3-4B, Thinking Mode=Disabled, Temperature=02026.05 | 6.35 | 6.04 | |
| Draft-OPDModel=Q3-4B, Thinking Mode=Disabled, Temperature=0.62026.05 | 6.35 | 6.13 | |
| DFlashModel=Q3-8B, Thinking Mode=Disabled, Temperature=0.62026.05 | 6.18 | 5.62 | |
| EAGLE-3Model=Q3-8B, Thinking Mode=Disabled, Temperature=02026.05 | 6.07 | 5.99 | |
| DFlashModel=Q3-4B, Thinking Mode=Disabled, Temperature=0.62026.05 | 5.87 | 5.73 | |
| Draft-OPDModel=Q3-8B, Thinking Mode=Enabled, Temperature=02026.05 | 5.8 | 5.73 | |
| EAGLE-3Model=Q3-4B, Thinking Mode=Disabled, Temperature=02026.05 | 5.7 | 5.84 | |
| Draft-OPDModel=Q3-4B, Thinking Mode=Enabled, Temperature=02026.05 | 5.55 | 5.96 | |
| EAGLE-3Model=Q3-8B, Thinking Mode=Disabled, Temperature=0.62026.05 | 5.17 | 5.69 | |
| Draft-OPDModel=Q3-8B, Thinking Mode=Enabled, Temperature=0.62026.05 | 5.07 | 5.05 | |
| DFlashModel=Q3-8B, Thinking Mode=Enabled, Temperature=02026.05 | 5.01 | 5.19 | |
| EAGLE-3Model=Q3-4B, Thinking Mode=Disabled, Temperature=0.62026.05 | 4.93 | 5.66 | |
| Draft-OPDModel=Q3-4B, Thinking Mode=Enabled, Temperature=0.62026.05 | 4.91 | 5.13 | |
| DFlashModel=Q3-4B, Thinking Mode=Enabled, Temperature=02026.05 | 4.88 | 5.51 | |
| DFlashModel=Q3-8B, Thinking Mode=Enabled, Temperature=0.62026.05 | 4.57 | 4.66 | |
| DFlashModel=Q3-4B, Thinking Mode=Enabled, Temperature=0.62026.05 | 4.46 | 4.77 | |
| EAGLE-3Model=Q3-8B, Thinking Mode=Enabled, Temperature=02026.05 | 4.45 | 5.64 | |
| EAGLE-3Model=Q3-4B, Thinking Mode=Enabled, Temperature=02026.05 | 4.15 | 5.33 | |
| EAGLE-3Model=Q3-8B, Thinking Mode=Enabled, Temperature=0.62026.05 | 4.09 | 5.33 | |
| EAGLE-3Model=Q3-4B, Thinking Mode=Enabled, Temperature=0.62026.05 | 3.77 | 5.03 |