QA Benchmarks
Benchmarks
TaskDatasetSOTA resultTrendResults
5 QA Benchmarks Average
40.9Average Exact Match
14
Average across QA benchmarks (test)
38.1EM
10
QA benchmarks
54.6TTFT (ms)
10
QA Benchmarks Zero-shot (BoolQ, Lambada, Piqa, OPQA, Winogrande, ARC-E, ARC-C, Hellaswag)
74.86BoolQ Accuracy
6