Sudoku
Benchmarks
TaskDatasetSOTA resultTrendResults
Sudoku
94.3Accuracy
152
Sudoku
92.7Accuracy
141
Sudoku
96Pass@1
60
Sudoku
27.2Test Accuracy
42
Sudoku (test)
100Accuracy
33
Sudoku
91.9Accuracy (Sudoku Reasoning)
25
Sudoku 4x4 (test)
14.7Accuracy (Avg)
25
Sudoku (test)
91.7Accuracy
24
Sudoku Extreme
99.8Pass@1 Accuracy
21
Sudoku
91.82Accuracy
19
Sudoku (test)
100Accuracy
19
Sudoku 25x25 (90% masking)
52.2VPR (%)
18
Sudoku 25x25 80% masking
100VPR (%)
18
Sudoku 25x25 70% masking
100VPR (%)
18
Sudoku
99Success Rate (SR)
17
Sudoku (Su) (test)
34.3Loop Ratio
17
Sudoku 512 tokens
80.3Pass@1
15
Sudoku 256 tokens
92.9Pass@1
15
Sudoku
99.4Accuracy
14
Sudoku 2x2
1.3Final Reward
14
Sudoku Out Of Distribution
77.8Average @128
12
Sudoku In Distribution
97.1Average Score @128
12
Sudoku 10k 9x9 boards (val)
93.4Board Accuracy
12
Sudoku
57CSP Result Index 35
12
Sudoku
100Success Rate (pass@1)
12