Sudoku
Benchmarks
TaskDatasetSOTA resultTrendResults
Sudoku Hard 30/81 digits visible (test)
58.4Exact Match Accuracy
10
Sudoku Med. 35/81 digits visible (test)
85.2Exact Match Accuracy
10
Sudoku Easy 40/81 digits visible (test)
96.3Exact Match Accuracy
10
Sudoku-Extreme (held-out)
65.8ODE Accuracy
10
Sudoku
100Accuracy (Scale 40)
10
Sudoku 4x4 mixed (val)
82Accuracy
9
Sudoku Hard
100Success Rate
9
Sudoku Easy
100Accuracy
9
Sudoku 8B Instruct (test)
71.7Accuracy
9
Sudoku 4x4
100Accuracy
9
Sudoku 3x3
100Accuracy
9
Sudoku
44.2Accuracy
9
Sudoku
12.15Average per-epoch runtime (ms)
9
Sudoku
92.4Accuracy
8
Sudoku
97.9Accuracy (Easy)
8
Sudoku
100Accuracy
8
Sudoku
99.6Success Rate
8
Sudoku Extreme
87.3RSR (%)
8
9x9 Sudoku (test)
52Cell Accuracy
7
Sudoku 5x5
2.7Final Reward
7
Sudoku 4x4
2.1Final Reward
7
Sudoku 3x3
160Final Reward
7
Sudoku (17-givens)
96.6Accuracy
7
Sudoku 4x4 expert (val)
66Accuracy
6
Sudoku OOD - RRN (test)
91.8Instance Solved Rate
6