BBH
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
BBH Logical Deduction (Seven Objects) (test)
55.2Accuracy
22
BBH Sports Understanding
88Accuracy (BBH Sports)
21
BBH
85.01Accuracy
20
BBH (full)
0.913rho
20
BBH (test)
92.8Accuracy
20
BBH
38.7Accuracy
18
BBH Web of Lies
98Accuracy
18
BBH Disambiguation QA
71Accuracy (BBH Disambiguation QA)
18
BBH
77.3Boolean Expressions EM
18
BBH (Big-Bench Hard) (unseen)
98.8BBH Temporal Sequences
17
BBH-Algorithmic (test)
55.4ACC
16
BBH
83.03Acc
16
BBH
70.11Accuracy
15
BBH
81.62Accuracy
14
BBH
85.6Success Rate (BBH General Reasoning)
14
BBH
1Relative Cost
14
BBH
69.16Accuracy
14
BBH
73.2Accuracy (BBH)
12
BBH
52.1BBH Accuracy (0-shot)
12
BBH (unseen)
42.38Total Average Score
12
BBH
88.7Score
12
BBH n=2395 (test)
100Activation Score
11
BBH n=2395 (test)
100Activation
11
BBH matched-N (test)
86.25Accuracy (BBH matched-N)
11
BBH
6.12ECE
11