BIRD
Benchmarks
TaskDatasetSOTA resultTrendResults
BIRD
100Recall (R)
6
BIRD
90.1Recall@5
6
Bird
20.6Accuracy
6
BIRD
100Coverage
5
BIRD
0Cost Per Query
5
BIRD
9.2MAE (ATHENA)
5
Bird
94.9Accuracy
5
BIRD Route
79.62R@1
5
Bird
43.83Pass@1
5
BIRD-Interact
55.7Task Success Rate
4
BIRD (dev)
77.8Execution Accuracy
4
BIRD
127Original Pipeline Repair Count
4
BIRD hidden (test)
71.6Execution Accuracy
4
BIRD Many Columns (train)
3,823.7Token Cost (1k)
3
BIRD Medium Columns (train)
1,324.1Token Cost (1k Tokens)
3
BIRD Few Columns (train)
581.4Token Cost (1k)
3
BIRD
72Accuracy
3
BIRD 96 questions (subset)
93.8Execution Accuracy
2
BIRD official (test)
73.67Total Accuracy
2
Bird
—Primary metric
0