# Long-context understanding on LongBench-E 1.0 (test) Source: [Long-context understanding on LongBench-E 1.0 (test)]() Collection index: [llms.txt]() Data updated: 2026-03-25T13:22:47.112Z Evaluation of language models on the LongBench-E 1.0 test set, reporting performance across various long-context understanding sub-tasks including Qasper, HotpotQA, MultiNews, and TriviaQA. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Long-context understanding | | Dataset (including any stated split) | LongBench-E 1.0 (test) | | Metric | Qasper Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Qasper Score](): Higher is better; unit: PERCENT. - [Multi Score](): Higher is better; unit: PERCENT. - [HotpotQA Score](): Higher is better; unit: PERCENT. - [2Wiki Score](): Higher is better; unit: PERCENT. - [Gov Report Score](): Higher is better; unit: PERCENT. - [MultiNews Score](): Higher is better; unit: PERCENT. - [TREC Score](): Higher is better; unit: PERCENT. - [TriviaQA Score](): Higher is better; unit: PERCENT. - [SAMSum Score](): Higher is better; unit: PERCENT. - [Precision Count Score](): Higher is better; unit: PERCENT. - [Retrieval Precision Score](): Higher is better; unit: PERCENT. - [LCC Score](): Higher is better; unit: PERCENT. - [Repository Precision Score](): Higher is better; unit: PERCENT. - [Average Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Qasper Score | Result date | Report paper | | --- | --- | --- | --- | | Exact (Baseline) | 44.56 | Not recorded | 68fa78b41e209131cea79e1f | | Exact (Baseline) | 43.39 | Not recorded | 68fa78b41e209131cea79e1f | | Exact (Baseline) | 42.87 | Not recorded | 68fa78b41e209131cea79e1f | | BalanceKV | 40.14 | Not recorded | 68fa78b41e209131cea79e1f | | Uniform | 39.28 | Not recorded | 68fa78b41e209131cea79e1f | | Uniform | 37.18 | Not recorded | 68fa78b41e209131cea79e1f | | Uniform | 37.07 | Not recorded | 68fa78b41e209131cea79e1f | | BalanceKV | 37.02 | Not recorded | 68fa78b41e209131cea79e1f | | SnapKV | 36.21 | Not recorded | 68fa78b41e209131cea79e1f | | BalanceKV | 35.75 | Not recorded | 68fa78b41e209131cea79e1f | | PyramidKV | 34.47 | Not recorded | 68fa78b41e209131cea79e1f | | SnapKV | 33.91 | Not recorded | 68fa78b41e209131cea79e1f | | PyramidKV | 33.86 | Not recorded | 68fa78b41e209131cea79e1f | | SnapKV | 32.95 | Not recorded | 68fa78b41e209131cea79e1f | | PyramidKV | 31.76 | Not recorded | 68fa78b41e209131cea79e1f | | StreamingLLM | 20.73 | Not recorded | 68fa78b41e209131cea79e1f | | StreamingLLM | 20.65 | Not recorded | 68fa78b41e209131cea79e1f | | StreamingLLM | 20.12 | Not recorded | 68fa78b41e209131cea79e1f | ### Exact (Baseline) Qasper Score: 44.56. Result ID: 69c3e1a547f34ce924ad6129. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Compression Rate":"None"}. This record does not assert independent reproduction. ### Exact (Baseline) Qasper Score: 43.39. Result ID: 69c3e1a547f34ce924ad612f. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Compression Rate":"None"}. This record does not assert independent reproduction. ### Exact (Baseline) Qasper Score: 42.87. Result ID: 69c3e1a547f34ce924ad6135. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Instruct","Compression Rate":"None"}. This record does not assert independent reproduction. ### BalanceKV Qasper Score: 40.14. Result ID: 69c3e1a547f34ce924ad612e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Compression Rate":"0.25"}. This record does not assert independent reproduction. ### Uniform Qasper Score: 39.28. Result ID: 69c3e1a547f34ce924ad612d. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Compression Rate":"0.25"}. This record does not assert independent reproduction. ### Uniform Qasper Score: 37.18. Result ID: 69c3e1a547f34ce924ad6139. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Instruct","Compression Rate":"0.25"}. This record does not assert independent reproduction. ### Uniform Qasper Score: 37.07. Result ID: 69c3e1a547f34ce924ad6133. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Compression Rate":"0.25"}. This record does not assert independent reproduction. ### BalanceKV Qasper Score: 37.02. Result ID: 69c3e1a547f34ce924ad6134. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Compression Rate":"0.25"}. This record does not assert independent reproduction. ### SnapKV Qasper Score: 36.21. Result ID: 69c3e1a547f34ce924ad612c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Compression Rate":"0.25"}. This record does not assert independent reproduction. ### BalanceKV Qasper Score: 35.75. Result ID: 69c3e1a547f34ce924ad613a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Instruct","Compression Rate":"0.25"}. This record does not assert independent reproduction. ### PyramidKV Qasper Score: 34.47. Result ID: 69c3e1a547f34ce924ad612b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Compression Rate":"0.25"}. This record does not assert independent reproduction. ### SnapKV Qasper Score: 33.91. Result ID: 69c3e1a547f34ce924ad6138. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Instruct","Compression Rate":"0.25"}. This record does not assert independent reproduction. ### PyramidKV Qasper Score: 33.86. Result ID: 69c3e1a547f34ce924ad6137. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Instruct","Compression Rate":"0.25"}. This record does not assert independent reproduction. ### SnapKV Qasper Score: 32.95. Result ID: 69c3e1a547f34ce924ad6132. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Compression Rate":"0.25"}. This record does not assert independent reproduction. ### PyramidKV Qasper Score: 31.76. Result ID: 69c3e1a547f34ce924ad6131. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Compression Rate":"0.25"}. This record does not assert independent reproduction. ### StreamingLLM Qasper Score: 20.73. Result ID: 69c3e1a547f34ce924ad6130. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Compression Rate":"0.25"}. This record does not assert independent reproduction. ### StreamingLLM Qasper Score: 20.65. Result ID: 69c3e1a547f34ce924ad6136. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Instruct","Compression Rate":"0.25"}. This record does not assert independent reproduction. ### StreamingLLM Qasper Score: 20.12. Result ID: 69c3e1a547f34ce924ad612a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Compression Rate":"0.25"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.