Mathematical Reasoning on GSM8K 8B Instruct (test)
58.6AccuracyProb Margin + Both
Evaluation results
| Method | Links | |
|---|---|---|
| Prob Margin + BothToken Selection (T.S.)=-, Position Selection (P.S.)=L&E, Randomness Step=all, Compute Budget (T)=322026.04 | 58.6 | |
| Top1 Prob + BothToken Selection (T.S.)=-, Position Selection (P.S.)=L&E, Randomness Step=all, Compute Budget (T)=322026.04 | 56.8 | |
| Top1 Prob + PlannerToken Selection (T.S.)=-, Position Selection (P.S.)=L, Randomness Step=1st, Compute Budget (T)=322026.04 | 55 | |
| Init. PositionToken Selection (T.S.)=-, Position Selection (P.S.)=U, Randomness Step=1st, Compute Budget (T)=322026.04 | 52.6 | |
| Top1 Prob + EOS Temp.Token Selection (T.S.)=-, Position Selection (P.S.)=E, Randomness Step=all, Compute Budget (T)=322026.04 | 50.9 | |
| Prob MarginToken Selection (T.S.)=-, Position Selection (P.S.)=-, Randomness Step=-, Compute Budget (T)=322026.04 | 47.2 | |
| Top1 ProbToken Selection (T.S.)=-, Position Selection (P.S.)=-, Randomness Step=-, Compute Budget (T)=322026.04 | 46.6 | |
| Temperature SamplingToken Selection (T.S.)=T, Position Selection (P.S.)=-, Randomness Step=all, Compute Budget (T)=322026.04 | 45 | |
| Ancestral SamplingToken Selection (T.S.)=-, Position Selection (P.S.)=U, Randomness Step=all, Compute Budget (T)=322026.04 | 42.1 |