Beaver
Benchmarks
TaskDatasetSOTA resultTrendResults
Beaver
84.65Accuracy (Beaver Safety)
16
BEAVER Oracle dialect
36.4Execution Accuracy
9
BEAVER
43.5Capped Recall@25
9
BEAVER
32.6Recall@10
9
Beaver (dev)
17PR@5
8
BEAVER
23.1Recall@5
6
Beaver Response
84.4F1 Score
5
Beaver Prompt
77.2F1 Score
5
BEAVER
29.8Execution Accuracy (EX)
4