BigBench
Benchmarks
TaskDatasetSOTA resultTrendResults
BigBenchHard
100Accuracy (BigBenchHard)
37
BigBench Audio
97.6Accuracy
34
BigBench-Lite Topic Domains
66.8BBL Score
18
BigBenchHard
71.7Accuracy
18
BigBench 13 tasks (val)
58.7Accuracy
17
BIGBENCH II
100True WS Score
12
BigBench (test)
76.6Accuracy
10
BigBench Audio
83.7S2T Score
9
BigBench II
-0.37Accuracy Degradation (%)
9
BigBench (Lamb, SQuAD, CoQA, BBH, LSAT, LangID)
24Avg Score
8
BigBench Instruction Induction (BBII) (test)
60.14BBII Text Classification Score
6
BigBench Hard Hyperbaton
80.2Accuracy
5
BigBench Hard Snarks
0.554Accuracy
5
BigBench Hard Formal Fallacies
58.2Accuracy
5
BigBench Hard
31.1Score
5
BigBench Audio
80.73NFE
5
BigBench Extra Hard
14.3mean@4
4
BigBenchHard
63.23EM
4
BigBench-H SEM variant
95.08Accuracy
2
BigBench Emergent Suite (BBES)
67Navigate
2
BigBench (val)
—Primary metric
0