# Long-context language understanding on LongBench (Comprehensive Metric Suite) Source: [Long-context language understanding on LongBench (Comprehensive Metric Suite)]() Collection index: [llms.txt]() Data updated: 2026-06-30T07:42:43.639Z Comprehensive evaluation of long-context language understanding capabilities across a wide range of LongBench subtasks, including NrtvQA, Qasper, HotpotQA, TriviaQA, SAMSum, and others. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Long-context language understanding | | Dataset (including any stated split) | LongBench | | Metric | NrtvQA Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 37 of 37 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [NrtvQA Score](): Higher is better; unit: PERCENT. - [Qasper Score](): Higher is better; unit: PERCENT. - [MF-en Score](): Higher is better; unit: PERCENT. - [HotpotQA Score](): Higher is better; unit: PERCENT. - [2WikiMQA Score](): Higher is better; unit: PERCENT. - [Musique Score](): Higher is better; unit: PERCENT. - [GovRep Score](): Higher is better; unit: PERCENT. - [QMSum Score](): Higher is better; unit: PERCENT. - [MultiNews Score](): Higher is better; unit: PERCENT. - [TREC Score](): Higher is better; unit: PERCENT. - [TriviaQA Score](): Higher is better; unit: PERCENT. - [SAMSum Score](): Higher is better; unit: PERCENT. - [PCount](): Higher is better; unit: SCALAR. - [PRe Score](): Higher is better; unit: PERCENT. - [Lcc Score](): Higher is better; unit: PERCENT. - [RB-P Score](): Higher is better; unit: PERCENT. - [Average Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | NrtvQA Score | Result date | Report paper | | --- | --- | --- | --- | | K-VEC | 30.04 | Not recorded | 6a434d80ba73165dbb7439cb | | Pyramid | 29.62 | Not recorded | 6a434d80ba73165dbb7439cb | | SnapKV | 29.28 | Not recorded | 6a434d80ba73165dbb7439cb | | Ada-SnapKV | 29.23 | Not recorded | 6a434d80ba73165dbb7439cb | | Ada-Pyramid | 28.76 | Not recorded | 6a434d80ba73165dbb7439cb | | FullKV | 27.96 | Not recorded | 6a02b3f7dbc836b87d824593 | | SnapKV | 27.8 | Not recorded | 6a02b3f7dbc836b87d824593 | | RDKV | 27.75 | Not recorded | 6a02b3f7dbc836b87d824593 | | AdaKV | 27.64 | Not recorded | 6a02b3f7dbc836b87d824593 | | RDKV | 27.35 | Not recorded | 6a02b3f7dbc836b87d824593 | | RDKV | 26.79 | Not recorded | 6a02b3f7dbc836b87d824593 | | ThinK | 26.73 | Not recorded | 6a02b3f7dbc836b87d824593 | | AdaKV | 26.72 | Not recorded | 6a02b3f7dbc836b87d824593 | | Snap+Zip | 26.6 | Not recorded | 6a02b3f7dbc836b87d824593 | | SnapKV | 26.57 | Not recorded | 6a02b3f7dbc836b87d824593 | | ThinK | 26.48 | Not recorded | 6a02b3f7dbc836b87d824593 | | RDKV | 26.45 | Not recorded | 6a02b3f7dbc836b87d824593 | | SnapKV | 25.92 | Not recorded | 6a02b3f7dbc836b87d824593 | | Snap+Zip | 25.47 | Not recorded | 6a02b3f7dbc836b87d824593 | | AdaKV | 25.46 | Not recorded | 6a02b3f7dbc836b87d824593 | | ThinK | 25.3 | Not recorded | 6a02b3f7dbc836b87d824593 | | SLM | 24.97 | Not recorded | 6a434d80ba73165dbb7439cb | | RDKV | 24.94 | Not recorded | 6a02b3f7dbc836b87d824593 | | K-VEC | 24.55 | Not recorded | 6a434d80ba73165dbb7439cb | | Snap+Zip | 24.21 | Not recorded | 6a02b3f7dbc836b87d824593 | | SnapKV | 23.88 | Not recorded | 6a434d80ba73165dbb7439cb | | Snap+Zip | 23.84 | Not recorded | 6a02b3f7dbc836b87d824593 | | AdaKV | 23.78 | Not recorded | 6a02b3f7dbc836b87d824593 | | K-VEC | 23.45 | Not recorded | 6a434d80ba73165dbb7439cb | | ThinK | 22.88 | Not recorded | 6a02b3f7dbc836b87d824593 | | SnapKV | 21.46 | Not recorded | 6a02b3f7dbc836b87d824593 | | GemFilter | 20.71 | Not recorded | 6a434d80ba73165dbb7439cb | | SnapKV | 20.42 | Not recorded | 6a434d80ba73165dbb7439cb | | ThinK | 18.65 | Not recorded | 6a02b3f7dbc836b87d824593 | | Snap+Zip | 18.22 | Not recorded | 6a02b3f7dbc836b87d824593 | | SnapKV | 16.95 | Not recorded | 6a02b3f7dbc836b87d824593 | | AdaKV | 16.83 | Not recorded | 6a02b3f7dbc836b87d824593 | ### K-VEC NrtvQA Score: 30.04. Result ID: 6a437359fbc373592c93c40b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Instruct","Cache size":"1024"}. This record does not assert independent reproduction. ### Pyramid NrtvQA Score: 29.62. Result ID: 6a437359fbc373592c93c406. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Instruct","Cache size":"1024"}. This record does not assert independent reproduction. ### SnapKV NrtvQA Score: 29.28. Result ID: 6a437359fbc373592c93c407. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Instruct","Cache size":"1024"}. This record does not assert independent reproduction. ### Ada-SnapKV NrtvQA Score: 29.23. Result ID: 6a437359fbc373592c93c409. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Instruct","Cache size":"1024"}. This record does not assert independent reproduction. ### Ada-Pyramid NrtvQA Score: 28.76. Result ID: 6a437359fbc373592c93c408. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Instruct","Cache size":"1024"}. This record does not assert independent reproduction. ### FullKV NrtvQA Score: 27.96. Result ID: 6a02dc2c47f34ce924e56aad. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"Full"}. This record does not assert independent reproduction. ### SnapKV NrtvQA Score: 27.8. Result ID: 6a02dc2c47f34ce924e56ac2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"1024L"}. This record does not assert independent reproduction. ### RDKV NrtvQA Score: 27.75. Result ID: 6a02dc2c47f34ce924e56ac1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"512L"}. This record does not assert independent reproduction. ### AdaKV NrtvQA Score: 27.64. Result ID: 6a02dc2c47f34ce924e56ac3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"1024L"}. This record does not assert independent reproduction. ### RDKV NrtvQA Score: 27.35. Result ID: 6a02dc2c47f34ce924e56ac6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"1024L"}. This record does not assert independent reproduction. ### RDKV NrtvQA Score: 26.79. Result ID: 6a02dc2c47f34ce924e56ab7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"128L"}. This record does not assert independent reproduction. ### ThinK NrtvQA Score: 26.73. Result ID: 6a02dc2c47f34ce924e56ac4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"1024L"}. This record does not assert independent reproduction. ### AdaKV NrtvQA Score: 26.72. Result ID: 6a02dc2c47f34ce924e56abe. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"512L"}. This record does not assert independent reproduction. ### Snap+Zip NrtvQA Score: 26.6. Result ID: 6a02dc2c47f34ce924e56ac5. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"1024L"}. This record does not assert independent reproduction. ### SnapKV NrtvQA Score: 26.57. Result ID: 6a02dc2c47f34ce924e56abd. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"512L"}. This record does not assert independent reproduction. ### ThinK NrtvQA Score: 26.48. Result ID: 6a02dc2c47f34ce924e56abf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"512L"}. This record does not assert independent reproduction. ### RDKV NrtvQA Score: 26.45. Result ID: 6a02dc2c47f34ce924e56abc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"256L"}. This record does not assert independent reproduction. ### SnapKV NrtvQA Score: 25.92. Result ID: 6a02dc2c47f34ce924e56ab8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"256L"}. This record does not assert independent reproduction. ### Snap+Zip NrtvQA Score: 25.47. Result ID: 6a02dc2c47f34ce924e56ac0. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"512L"}. This record does not assert independent reproduction. ### AdaKV NrtvQA Score: 25.46. Result ID: 6a02dc2c47f34ce924e56ab9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"256L"}. This record does not assert independent reproduction. ### ThinK NrtvQA Score: 25.3. Result ID: 6a02dc2c47f34ce924e56aba. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"256L"}. This record does not assert independent reproduction. ### SLM NrtvQA Score: 24.97. Result ID: 6a437359fbc373592c93c405. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Instruct","Cache size":"1024"}. This record does not assert independent reproduction. ### RDKV NrtvQA Score: 24.94. Result ID: 6a02dc2c47f34ce924e56ab2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"64L"}. This record does not assert independent reproduction. ### K-VEC NrtvQA Score: 24.55. Result ID: 6a437359fbc373592c93c412. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct","Budget (B)":"1024"}. This record does not assert independent reproduction. ### Snap+Zip NrtvQA Score: 24.21. Result ID: 6a02dc2c47f34ce924e56abb. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"256L"}. This record does not assert independent reproduction. ### SnapKV NrtvQA Score: 23.88. Result ID: 6a437359fbc373592c93c411. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct","Budget (B)":"1024"}. This record does not assert independent reproduction. ### Snap+Zip NrtvQA Score: 23.84. Result ID: 6a02dc2c47f34ce924e56ab6. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"128L"}. This record does not assert independent reproduction. ### AdaKV NrtvQA Score: 23.78. Result ID: 6a02dc2c47f34ce924e56ab4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"128L"}. This record does not assert independent reproduction. ### K-VEC NrtvQA Score: 23.45. Result ID: 6a437359fbc373592c93c410. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct","Budget (B)":"128"}. This record does not assert independent reproduction. ### ThinK NrtvQA Score: 22.88. Result ID: 6a02dc2c47f34ce924e56ab5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"128L"}. This record does not assert independent reproduction. ### SnapKV NrtvQA Score: 21.46. Result ID: 6a02dc2c47f34ce924e56ab3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"128L"}. This record does not assert independent reproduction. ### GemFilter NrtvQA Score: 20.71. Result ID: 6a437359fbc373592c93c40a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Instruct","Cache size":"1024"}. This record does not assert independent reproduction. ### SnapKV NrtvQA Score: 20.42. Result ID: 6a437359fbc373592c93c40f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct","Budget (B)":"128"}. This record does not assert independent reproduction. ### ThinK NrtvQA Score: 18.65. Result ID: 6a02dc2c47f34ce924e56ab0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"64L"}. This record does not assert independent reproduction. ### Snap+Zip NrtvQA Score: 18.22. Result ID: 6a02dc2c47f34ce924e56ab1. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"64L"}. This record does not assert independent reproduction. ### SnapKV NrtvQA Score: 16.95. Result ID: 6a02dc2c47f34ce924e56aae. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"64L"}. This record does not assert independent reproduction. ### AdaKV NrtvQA Score: 16.83. Result ID: 6a02dc2c47f34ce924e56aaf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Instruct-2507","Cache Budget (Btotal)":"64L"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.