Mathematical Reasoning on Countdown 8B Instruct (test)
46.1AccuracyTop1 Prob + EOS Temp.
Evaluation results
| Method | Links | |
|---|---|---|
| Top1 Prob + EOS Temp.Token Selection (T.S.)=-, Position Selection (P.S.)=E, Randomness Step=all, Compute Budget (T)=322026.04 | 46.1 | |
| Prob Margin + BothToken Selection (T.S.)=-, Position Selection (P.S.)=L&E, Randomness Step=all, Compute Budget (T)=322026.04 | 45.3 | |
| Top1 Prob + PlannerToken Selection (T.S.)=-, Position Selection (P.S.)=L, Randomness Step=1st, Compute Budget (T)=322026.04 | 44.1 | |
| Top1 Prob + BothToken Selection (T.S.)=-, Position Selection (P.S.)=L&E, Randomness Step=all, Compute Budget (T)=322026.04 | 43.8 | |
| Temperature SamplingToken Selection (T.S.)=T, Position Selection (P.S.)=-, Randomness Step=all, Compute Budget (T)=322026.04 | 43.4 | |
| Top1 ProbToken Selection (T.S.)=-, Position Selection (P.S.)=-, Randomness Step=-, Compute Budget (T)=322026.04 | 42.2 | |
| Prob MarginToken Selection (T.S.)=-, Position Selection (P.S.)=-, Randomness Step=-, Compute Budget (T)=322026.04 | 41.4 | |
| Init. PositionToken Selection (T.S.)=-, Position Selection (P.S.)=U, Randomness Step=1st, Compute Budget (T)=322026.04 | 30.1 | |
| Ancestral SamplingToken Selection (T.S.)=-, Position Selection (P.S.)=U, Randomness Step=all, Compute Budget (T)=322026.04 | 22.7 |