COUNTDOWN
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Countdown (test)
87Score
10
Countdown 4,5,6-arg held-out difficulties (test)
25.1Accuracy
10
Countdown 8B Instruct (test)
46.1Accuracy
9
Countdown
100Accuracy
8
Countdown-34 (held-out)
81.26Accuracy
8
Countdown
0.61ROC-AUC (128)
8
Countdown
19.4Countdown
7
Countdown 0-shot (test)
71.5Pass@1 (Greedy)
7
Countdown (test)
57.7Accuracy
7
Countdown (eval)
5.27MAE
6
Countdown
49.8CD4 Score
6
Countdown
92Pass@1
6
Countdown
40.6Exact-match Accuracy (Ngen=128)
6
Countdown
40.4Average Diffusion Steps
6
Countdown (test)
74.7Accuracy Pass@1
5
Countdown (CTD) (test)
43.8Accuracy
4
Countdown
15.3Score (%)
4
Countdown CD 5
41.3Success Rate
4
Countdown 4
94.4Success Rate
4
Countdown (val)
41.9Accuracy
3
Countdown
175Optimization Steps
2