BIG-Bench
Benchmarks
TaskDatasetSOTA resultTrendResults
BIG-Bench Hard
91.1Accuracy
68
BIG-Bench Hard (BBH) (test)
87.3Average Accuracy
62
Big-Bench Hard (BBH)
82.3Accuracy
60
BIG-bench HHH Eval
87Overall Score
42
BIG-bench
81.6Accuracy (General)
36
BIG-Bench Hard (train)
91.9Accuracy
28
BIG-bench Hard Word Sorting (test)
45Test Accuracy
26
Big-Bench Hard (BBH)
88.1Exact Match Performance
22
BBH (BIG-Bench Hard)
87.5Average BBH Score
20
Big-bench Hard (BBH)
53.53Exact Match (EM)
20
BIG-bench-lite 24 tasks
3,777Score
17
Big-Bench Hard (test)
57.9Exact Match
17
Big-Bench Hard
85.7Hard Accuracy
15
BIG-bench (unseen)
86.96Known Unknowns Score
15
BIG-bench Hard Date Understanding (test)
75.2Test Accuracy
14
BIG-Bench
85.6Accuracy
12
BIG-Bench Hard MIX-14K
69.9Accuracy
12
BIG-bench Mimicked
99.7Sports Score
11
BIG-bench Original
99.4Sports Score
11
BIG Bench Audio Speech Modality
0.934Accuracy
10
BIG-Bench Extra Hard
37.8Score
10
BIG-bench
80Hindu Knowledge
10
BIG-Bench Hard (BBH) (test)
68.3Causal Judgement
10
Big-Bench Hard
71.02Accuracy
8
BIG-Bench Hard 5 subtasks (full)
86Accuracy
8