SCAN
Benchmarks
TaskDatasetSOTA resultTrendResults
SCAN HPD
82.88Accuracy
22
SCAN
242.24Overall Score
21
SCAN jump
100Accuracy
18
SCAN Around Right
100Exact-match Accuracy
16
SCAN (out-of-domain)
15.3Accuracy
15
SCAN Around Right (test)
95.7Accuracy
15
SCAN Around Right (val)
99.8Accuracy
15
SCAN Add Jump (test)
99.8Accuracy
15
SCAN Add Jump (val)
99.6Accuracy
15
SCAN Simple v1.0
1Accuracy
12
SCAN MCD3
80.2Exact Match Accuracy
12
SCAN (MCD2)
80.8Exact Match Accuracy
12
SCAN (MCD1)
0.674Exact-match Accuracy
12
SCAN Jump
100Exact-match Accuracy
11
SCAN (length)
99.7Accuracy
11
scan with board (test)
1.4AEE
9
SCAN around right v1.0
1Accuracy
8
SCAN around right
99.51Accuracy
7
SCAN
23.44Length
6
SCAN (MCD)
100Accuracy
6
SCAN Template
100Accuracy
6
SCAN (Length)
100Accuracy
6
SCAN 0-shot lexical
99Accuracy (0-shot)
6
SCAN 1-shot lexical
100Accuracy
6
SCAN (IID)
100Accuracy
6