BIRD
Benchmarks
TaskDatasetSOTA resultTrendResults
BIRD (dev)
77.84Execution Accuracy (EA)
477
BIRD
65.12Execution Accuracy (EX)
83
BIRD
97.59Total Execution Accuracy
68
BIRD (test)
75.63EX
55
BIRD (dev)
65.8Exec Acc (Greedy)
44
BIRD
73Execution Accuracy
33
BIRD (Non-Synthesized Matched Set)
93.15ExM Accuracy
32
BIRD Synthesized Matched Set
90.39ExM Accuracy
32
BIRD
119True Positive Repairs (EM)
30
Bird
85.1Accuracy
29
Bird color video sequence
25.382PSNR
28
BIRD
69.1Accuracy
27
BIRD
80.36AUPRC
24
BIRD
31.8Execution Accuracy (Llama-8B)
21
BIRD n=1534
64.9EM (1T)
21
BIRD union (test)
54.4Precision
20
BIRD (dev)
100SRR
16
Bird
6.67Match Accuracy (MAT)
15
BIRD
100Recall (R)
15
BIRD Original (dev)
65.51Execution Accuracy (Simple)
14
BIRD Verified
69.41Execution Accuracy (Simple)
14
BIRD (test dev)
48.92Execution Accuracy (EX)
14
BIRD (test)
29Accuracy
13
BIRD
-0.11Kendall's τ
12
BIRD 2023 (test)
5.3Avg #tables
12