LongBench
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
LongBench Few-shot Learning
61.34Accuracy
5
LongBench Summarization
18.08Accuracy
5
LongBench-E
23.92S. QA Accuracy
5
LongBench
25GvR Score
5
LongBench-E
3.032Prefill Time (sec)
5
LongBench-SingleDoc (test)
26.8Score (2k tokens)
5
LongBench VI. Long Structured Data Understanding v2
66.7Accuracy
4
LongBench Code Repository Understanding v2
52Accuracy
4
LongBench IV. Long-dialogue History Understanding v2
61.6Accuracy
4
LongBench Long In-context Learning v2
49.4Accuracy
4
LongBench I. Single-Document QA v2
48Accuracy
4
LongBench 128K v2
64Overall Score
4
LongBench
25.7QQA
4
LongBench 493K v2 (L)
70Score
4
LongBench M 179K v2
68.9Overall Score
4
LongBench S 46K v2
70.2Score
4
LongBench 32K context slice v2
56.86Score
4
LongBench 16K context slice v2
57.89Score
4
LongBench
28.17Qasper
4
LongBench MuSiQue
0.389F1 Score
4
LongBench-Pro English
37.25Accuracy
4
LongBench
30.19Qasper
4
LongBench
32.03Single-Doc QA Score
4
LongBench Repobench (test)
2.26Avg Accepted Draft Tokens/Round
4
LongBench Samsum (test)
1.31Avg Accepted Draft Tokens per Round
4