LongBench
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
LongBench-E Multi-Doc QA
49.6F1 Score
17
LongBench-E Single-Doc QA
43.3F1 Score
17
LongBench 4-task average
12.7Average Accuracy
17
LongBench v2
43.6Accuracy
17
LongBench
31.8Overall Average Score
17
LongBench Summary (test)
37.68Score
17
LongBench v2 (test)
86.7Finance Accuracy
16
LongBench
50.91SQA Score
16
LongBench (test)
37.68MAE
16
LongBench CodeQA v2
0.741Accuracy
16
LongBench-e (test)
68.42LCC (Language Comprehension Score)
16
LongBench v0.2
26.95Na.QA (NER QA)
16
LongBench
815,449.95Average Context Length (tokens)
16
LongBench
80.29Math Performance
15
LongBench
29.52NQ Score
15
LongBench
34.92NQA
15
LongBench v2 (val)
37.8Overall Accuracy
15
LongBench
27.36Narrative Score (Nrtv.)
14
LongBench 256 tokens v2
100Accuracy
14
LongBench Ministral-8B-Instruct
30.21NrtvQA
14
LongBench LLaMA-3.1-8B-Instruct (test)
32.8NrtvQA
14
LongBench
1Relative Cost
14
LongBench 1 host v1 (test)
46.232WQA Score
14
Longbench
0.657Comprehensiveness
14
LongBench
16GovReport Score
13