# Long-context Language Understanding on LongBench (Specific Metric Subset including NQA and Delta Avg)

Source: [Long-context Language Understanding on LongBench (Specific Metric Subset including NQA and Delta Avg)](<https://www.sota2.com/research/sota/long-context-language-understanding-on-longbench-specific-metric-subset-including-nqa-and-delta-avg>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-21T08:26:47.154Z

Evaluation of general language understanding capabilities over long contexts using the LongBench dataset, reporting performance across a specific subset of 10 metrics including NQA, LCC, and the average performance change (Δ Avg.).

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Long-context Language Understanding |
| Dataset (including any stated split) | LongBench |
| Metric | NQA |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [NQA](<https://www.sota2.com/research/sota/long-context-language-understanding-on-longbench-specific-metric-subset-including-nqa-and-delta-avg.md?metric=NQA>): Higher is better; unit: PERCENT.

- [MF-en](<https://www.sota2.com/research/sota/long-context-language-understanding-on-longbench-specific-metric-subset-including-nqa-and-delta-avg.md?metric=MF-en>): Higher is better; unit: PERCENT.

- [HP](<https://www.sota2.com/research/sota/long-context-language-understanding-on-longbench-specific-metric-subset-including-nqa-and-delta-avg.md?metric=HP>): Higher is better; unit: PERCENT.

- [MQ](<https://www.sota2.com/research/sota/long-context-language-understanding-on-longbench-specific-metric-subset-including-nqa-and-delta-avg.md?metric=MQ>): Higher is better; unit: PERCENT.

- [DR](<https://www.sota2.com/research/sota/long-context-language-understanding-on-longbench-specific-metric-subset-including-nqa-and-delta-avg.md?metric=DR>): Higher is better; unit: PERCENT.

- [GR](<https://www.sota2.com/research/sota/long-context-language-understanding-on-longbench-specific-metric-subset-including-nqa-and-delta-avg.md?metric=GR>): Higher is better; unit: PERCENT.

- [PR](<https://www.sota2.com/research/sota/long-context-language-understanding-on-longbench-specific-metric-subset-including-nqa-and-delta-avg.md?metric=PR>): Higher is better; unit: PERCENT.

- [LCC](<https://www.sota2.com/research/sota/long-context-language-understanding-on-longbench-specific-metric-subset-including-nqa-and-delta-avg.md?metric=LCC>): Higher is better; unit: PERCENT.

- [Average Score](<https://www.sota2.com/research/sota/long-context-language-understanding-on-longbench-specific-metric-subset-including-nqa-and-delta-avg.md?metric=Average%20Score>): Higher is better; unit: PERCENT.

- [Delta Average Change](<https://www.sota2.com/research/sota/long-context-language-understanding-on-longbench-specific-metric-subset-including-nqa-and-delta-avg.md?metric=Delta%20Average%20Change>): Higher is better; unit: SCALAR.

## Recorded results

| Method | NQA | Result date | Report paper |
| --- | --- | --- | --- |
| H2O | 34.92 | Not recorded | 69e704767d0172140ec7ec01 |
| Full Attention | 34.81 | Not recorded | 69e704767d0172140ec7ec01 |
| SinkRouter | 34.43 | Not recorded | 69e704767d0172140ec7ec01 |
| FastGen | 30.24 | Not recorded | 69e704767d0172140ec7ec01 |
| SinkRouter | 30.08 | Not recorded | 69e704767d0172140ec7ec01 |
| Full Attention | 30 | Not recorded | 69e704767d0172140ec7ec01 |
| H2O | 29.69 | Not recorded | 69e704767d0172140ec7ec01 |
| FastGen | 26.02 | Not recorded | 69e704767d0172140ec7ec01 |
| StreamingLLM | 25.35 | Not recorded | 69e704767d0172140ec7ec01 |
| StreamingLLM | 20.7 | Not recorded | 69e704767d0172140ec7ec01 |
| H2O | 12.25 | Not recorded | 69e704767d0172140ec7ec01 |
| Full Attention | 11.92 | Not recorded | 69e704767d0172140ec7ec01 |
| SinkRouter | 11.66 | Not recorded | 69e704767d0172140ec7ec01 |
| FastGen | 10.65 | Not recorded | 69e704767d0172140ec7ec01 |
| StreamingLLM | 4.29 | Not recorded | 69e704767d0172140ec7ec01 |

### H2O

NQA: 34.92. Result ID: 69e734c147f34ce924cd65b7.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa718c1e209131cea51ead.md>)

[Implementation](<https://github.com/FMInference/H2O>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-70B"}. This record does not assert independent reproduction.

### Full Attention

NQA: 34.81. Result ID: 69e734c147f34ce924cd65b6.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-70B"}. This record does not assert independent reproduction.

### SinkRouter

NQA: 34.43. Result ID: 69e734c147f34ce924cd65ba.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-70B"}. This record does not assert independent reproduction.

### FastGen

NQA: 30.24. Result ID: 69e734c147f34ce924cd65b8.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa757a1e209131cea68385.md>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-70B"}. This record does not assert independent reproduction.

### SinkRouter

NQA: 30.08. Result ID: 69e734c147f34ce924cd65b0.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction.

### Full Attention

NQA: 30. Result ID: 69e734c147f34ce924cd65ac.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction.

### H2O

NQA: 29.69. Result ID: 69e734c147f34ce924cd65ad.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa718c1e209131cea51ead.md>)

[Implementation](<https://github.com/FMInference/H2O>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction.

### FastGen

NQA: 26.02. Result ID: 69e734c147f34ce924cd65ae.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa757a1e209131cea68385.md>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction.

### StreamingLLM

NQA: 25.35. Result ID: 69e734c147f34ce924cd65b9.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72d71e209131cea58d79.md>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-70B"}. This record does not assert independent reproduction.

### StreamingLLM

NQA: 20.7. Result ID: 69e734c147f34ce924cd65af.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72d71e209131cea58d79.md>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction.

### H2O

NQA: 12.25. Result ID: 69e734c147f34ce924cd65b2.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa718c1e209131cea51ead.md>)

[Implementation](<https://github.com/FMInference/H2O>)

Recorded evaluation configuration: {"Backbone":"Yi-9B-200K"}. This record does not assert independent reproduction.

### Full Attention

NQA: 11.92. Result ID: 69e734c147f34ce924cd65b1.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

Recorded evaluation configuration: {"Backbone":"Yi-9B-200K"}. This record does not assert independent reproduction.

### SinkRouter

NQA: 11.66. Result ID: 69e734c147f34ce924cd65b5.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

Recorded evaluation configuration: {"Backbone":"Yi-9B-200K"}. This record does not assert independent reproduction.

### FastGen

NQA: 10.65. Result ID: 69e734c147f34ce924cd65b3.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa757a1e209131cea68385.md>)

Recorded evaluation configuration: {"Backbone":"Yi-9B-200K"}. This record does not assert independent reproduction.

### StreamingLLM

NQA: 4.29. Result ID: 69e734c147f34ce924cd65b4.

[Reporting paper](<https://www.sota2.com/research/paper/69e704767d0172140ec7ec01.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72d71e209131cea58d79.md>)

Recorded evaluation configuration: {"Backbone":"Yi-9B-200K"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.