BIRD
Benchmarks
TaskDatasetSOTA resultTrendResults
BIRD
57.3Precision (P)
11
Bird
60.6Pass@1
11
BIRD (holdout test)
73Execution Accuracy
11
BIRD In-Domain (dev)
70.7Accuracy (Simple)
10
BIRD 3-table
56.25Accuracy
9
Bird-200 (test)
89.7EM
9
BIRD Movies
46.9Execution Accuracy
9
BIRD App Store
38.4Execution Accuracy
9
BIRD Computer Students
48.3Execution Accuracy
9
BIRD
99.1Capped Recall@25
9
Bird
96.1Recall@10
9
BIRD Interact
7.09Avg Turns (WRITE)
8
BIRD Movies
0.65Spearman Correlation
8
BIRD Apps
0.74Spearman Correlation
8
BIRD Computer
0.46Spearman Correlation
8
Bird (dev)
86.2Precision@5
8
BIRD 10-way 0-shot
96.5Accuracy
8
Bird
79.5mIoU
8
BIRD-147 2023 (test)
97.3SRR
7
Bird
25.3PSNR
7
BIRD
78.7California Schools Accuracy
6
BIRD
40.7Execution Accuracy
6
Bird
53.89Mean Best Fitness
6
Bird
0Minimum Best Fitness
6
Bird
0.088Std Dev of Mean Best Fitness
6