LongBench
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
LongBench Retrieval v2 (full)
0.484F1 Score
55
LongBench (test)
50.87Qasper Score
54
LongBench
82.62TREC Score
51
LongBench
33.39GovRep Score
51
LongBench
63.4Average Score
50
LongBench v1 (test)
30.7NrtvQA Score
48
LongBench N=162
31.5F1 Score
45
LongBench
57.67HotpotQA Acc
45
LongBench
46.25Average Score
43
LongBench
70.64Lcc Score
43
LongBench
14.67PCount
43
LongBench v2
46.5Overall Accuracy
41
LongBench
48.37Average Score
40
LongBench
31.42NQA
38
LongBench GovReport
34.61ROUGE-L
38
LongBench
30.04NrtvQA Score
37
LongBench
62.6Accuracy
37
LongBench-E 1.0 (test)
49.92S-Doc QA Perf.
37
LongBench V1
39.8S-Doc QA Accuracy
36
LongBench V1
31NQA
36
LongBench V2
56.77Overall Accuracy
33
LongBench v1 (test)
19.6NrtQA Score
31
LongBench
47.26Average Score
30
LongBench English
19.63Accuracy
30
LongBench
27.84NrtvQA Score
29