Spider
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Spider Real
0.841Execution Accuracy
20
Spider no-easy
61.4Accuracy
20
Spider Lite 2.0 (test)
29.4Precision
20
Spider union (test)
69.6Precision (P)
20
Spider
100Recall
19
Spider 2.0
92.59AUPRC
18
Spider n=1034
84.3EM (1 Table)
18
Spider Out-of-Domain (dev)
88.4Accuracy (Total)
15
SPIDER
79.39DSC
13
Spider
4.46Speedup
12
Spider-TAS
64F1 Score
12
Spider
-0.78Kendall's τ
12
Spider (test)
0.723bits/Byte
12
Spider 2.0
36.6Precision
11
Spider
69.6Precision
11
Spider 1.0 (val)
64.8Accuracy (All)
11
Spider (dev)
75.5Exact Match Accuracy
11
Spider-Syn (dev)
62.6Exact Match Accuracy
11
SPIDER 2018 (test)
2.7Avg Tables
10
Spider hidden (test)
72.1Exact Match (EM)
10
Spider-SYN 1.0 (val)
66.9EM Accuracy
10
Spider ADVETA-ADD
50.6Exact Match (EM)
10
Spider (ADVETA-RPL)
35.8Exact Match (EM)
10
Spider2-Snow 80-case
43.75Execution Accuracy
9
Spider T2w (test)
83Global Dice Score
9