BIG-bench Hard
Benchmarks
TaskDatasetSOTA resultTrendResults
Big-Bench Hard (BBH) (val)
43.46Accuracy
36
Big Bench Hard 3-shot
41.46Accuracy
18
Big-Bench Hard Word Sorting
79.8Success Rate
4
Big-Bench Hard Counting
91.9Success Rate
4
BIG-bench Hard Temporal Sequences (test)
62Test Accuracy
4
BIG-Bench Hard Causal Judgment (OOD)
60.4Accuracy
3
BIG-Bench Hard Navigate (OOD)
57.6Accuracy
3