BBH
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
BBH
70.8Accuracy
6
BBH
58.4Pass@1
5
BBH en
56.52Accuracy
5
BBH T2T 1,000 items (test)
84.3Overall Accuracy
4
BBH S2T 1,000 items (test)
68.5Overall Score
4
BBH
27.9TPS
4
BBH
29.61Throughput (TPS)
4
BBH-LB
38.3Accuracy (BBH-LB)
4
BBH-7
100Boolean Expressions Accuracy
4
BBH
36.6Accuracy
4
BBH (test)
159.89Throughput (Tokens/s)
4
BBH 3
61.48Performance Score
4
BBH (test)
52.33Accuracy
4
BBH
67.3ASR
3
BBH Penguins in a Table
93Accuracy
3
BBH Geometric Shapes
53Accuracy
3
BBH Date Understanding
93Accuracy
3
BBH CoT
54.42Accuracy (BBH CoT)
3
BBH
41.9BBH Accuracy
3
BBH
0.476Normalized Log Accuracy
3
BBH 3-shot (test)
162.3Throughput (Tokens/s)
3
BBH
84Accuracy (BBH Symbolic Reasoning)
3
BBH 5-shot
75.69Accuracy
3
BBH
0.002MSE
3
BBH VoiceBench
54.3Score
2