Sudoku
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Sudoku In-distribution (test)
100Accuracy
6
4x4 Sudoku
26.6Accuracy (Ngen=128)
6
Sudoku n=2 4x4 (eval)
100Accuracy
5
Sudoku
100Solve Rate
5
Sudoku (test)
99.9Accuracy
5
Sudoku
87.3RSR
5
Sudoku 9x9
0.11Accuracy
5
Sudoku n=3 9x9 (val)
100Accuracy
4
Sudoku Extreme
98.75Accuracy
4
Sudoku 4x4
95.7Success Rate
4
Sudoku
23.88Accuracy (128 tokens)
4
Sudoku (SDK) (test)
71.2Accuracy
4
Sudoku 4x4 with reasoning traces (expert) (val)
34Accuracy
3
Sudoku (n=5) 25x25 (val)
72Accuracy
3
Sudoku (n=4) 16x16 (val)
92Accuracy
3
Sudoku
18.37Accuracy
3
Sudoku Full 9x9
35.5Accuracy
3
Mini 4x4 Sudoku
93.5Accuracy
3
9 x 9 Sudoku
91Accuracy
2
Sudoku 4 x 4
95Accuracy
2
Sudoku
425Optimization Steps
2
Sudoku (final-step)
79.5Success Rate
2
Sudoku-Extreme 1000 puzzles (train)
100Train Exact Match
2
16x16 Maxi Sudoku (test)
0FSR
1
Sudoku 24-36-givens
70Accuracy
1