BBQ
Benchmarks
TaskDatasetSOTA resultTrendResults
BBQ
99.3Accuracy
175
BBQ Gender
82.4Accuracy
36
BBQ
92.12Accuracy
36
BBQ (test)
97.5Accuracy (Reasoning BBQ)
32
BBQ (test)
98.86Accuracy (amb)
20
BBQ Race
81.9Accuracy
18
BBQ Nationality
82.1Accuracy
18
BBQ Religion (test)
80.5Accuracy (ACC)
16
BBQ Age (test)
92.5Accuracy
16
BBQ averaged across gender, nationality, and religion domains
87.73Accuracy (Ambiguous)
16
BBQ (Bias Benchmark for QA) v1.0 (test)
93.1BBQ SES Score
16
BBQ SingleTurn
16.3Age Bias
12
BBQ
83.93Disambiguation TOP-1
12
BBQ Underspecified Intent
91.2Abstention F1
10
BBQ
9.21Harmful Content Rate (HCRate)
9
BBQ
1.84Steerability Score
9
BBQ
79Accuracy (All)
9
D_BBQ
99.5Accuracy
8
BBQ Overall Llama-3
80.7Accuracy
6
BBQ Unambiguous Questions
94Accuracy
5
BBQ Ambiguous Questions
97Accuracy
5
BBQ disambiguated questions
93Accuracy
5
BBQ (ambiguous)
95Accuracy
5
BBQ Gender
47.2Ambiguity Score
4
BBQ Ambig
85.04Accuracy
4