LongBench
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
LongBench QASPER
45.14F1 Score
10
LongBench NarrativeQA
27.4F1 Score
10
LongBench MuSiQue
22.32F1 Score
10
LongBench MFQA-zh
14.79F1 Score
10
LongBench MFQA-en
47.18F1 Score
10
LongBench HotpotQA
52.15F1 Score
10
LongBench 2WikiMQA
37.47F1 Score
10
LongBench v2 (full)
56.7Overall Accuracy
10
LongBench 48 items median 11K tokens
39.6F1 Score
10
LongBench avg
45.4Avg. F1
10
LongBench Short v2
61.66Average Score @4 (Short Context)
10
LongBench Pro
34.2F1 Score
10
LongBench E
52.28Single Doc QA
10
LongBench
37.66Qasper
10
LongBench Chat
72.6Point-wise Rate
10
LongBench
33.15LCC
9
LongBench Long v2
41.7Accuracy
9
LongBench Overall v2
42.5Accuracy
9
LongBench multi-page
63.6HQA Score
9
LongBench key_facts v2
100Accuracy
9
LongBench
73.7Accuracy
9
LongBench 32k
41.35LongBench 32k Score
9
LongBench
68.5F1 Score
9
LongBench v2
59.76Overall Score
9
LongBench Llama-2-7B-4K
62.54Code Completion
9