Logic Reasoning on Sudoku 8B Instruct (test)
71.7AccuracyProb Margin
Evaluation results
| Method | Links | |
|---|---|---|
| Prob MarginToken Selection (T.S.)=-, Position Selection (P.S.)=-, Randomness Step=-, Compute Budget (T)=322026.04 | 71.7 | |
| Top1 ProbToken Selection (T.S.)=-, Position Selection (P.S.)=-, Randomness Step=-, Compute Budget (T)=322026.04 | 71.2 | |
| Temperature SamplingToken Selection (T.S.)=T, Position Selection (P.S.)=-, Randomness Step=all, Compute Budget (T)=322026.04 | 69.9 | |
| Prob Margin + BothToken Selection (T.S.)=-, Position Selection (P.S.)=L&E, Randomness Step=all, Compute Budget (T)=322026.04 | 69.5 | |
| Top1 Prob + BothToken Selection (T.S.)=-, Position Selection (P.S.)=L&E, Randomness Step=all, Compute Budget (T)=322026.04 | 67 | |
| Top1 Prob + PlannerToken Selection (T.S.)=-, Position Selection (P.S.)=L, Randomness Step=1st, Compute Budget (T)=322026.04 | 65.2 | |
| Top1 Prob + EOS Temp.Token Selection (T.S.)=-, Position Selection (P.S.)=E, Randomness Step=all, Compute Budget (T)=322026.04 | 63.6 | |
| Init. PositionToken Selection (T.S.)=-, Position Selection (P.S.)=U, Randomness Step=1st, Compute Budget (T)=322026.04 | 48.3 | |
| Ancestral SamplingToken Selection (T.S.)=-, Position Selection (P.S.)=U, Randomness Step=all, Compute Budget (T)=322026.04 | 17.8 |