# Long-context Language Modeling on LongBench 16K context length

Source: [Long-context Language Modeling on LongBench 16K context length](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-20T09:14:12.866Z

Evaluation of language models using the LongBench dataset specifically configured for a 16,000 token context window, reporting performance across a comprehensive set of long-context specific metrics including NrtvQA, Qasper, SAMSum, and LCC.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Long-context Language Modeling |
| Dataset (including any stated split) | LongBench 16K context length |
| Metric | NrtvQA Score |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 6 of 6 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [NrtvQA Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=NrtvQA%20Score>): Higher is better; unit: SCALAR.

- [Qasper Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=Qasper%20Score>): Higher is better; unit: SCALAR.

- [MF (English) Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=MF%20(English)%20Score>): Higher is better; unit: SCALAR.

- [MF (Chinese) Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=MF%20(Chinese)%20Score>): Higher is better; unit: SCALAR.

- [HPQA Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=HPQA%20Score>): Higher is better; unit: SCALAR.

- [2WikiMQA Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=2WikiMQA%20Score>): Higher is better; unit: SCALAR.

- [MuSiQue Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=MuSiQue%20Score>): Higher is better; unit: SCALAR.

- [DuReader Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=DuReader%20Score>): Higher is better; unit: SCALAR.

- [GovReport Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=GovReport%20Score>): Higher is better; unit: SCALAR.

- [QMSum Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=QMSum%20Score>): Higher is better; unit: SCALAR.

- [MultiNews Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=MultiNews%20Score>): Higher is better; unit: SCALAR.

- [VCSum Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=VCSum%20Score>): Higher is better; unit: SCALAR.

- [TREC Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=TREC%20Score>): Higher is better; unit: SCALAR.

- [TriviaQA Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=TriviaQA%20Score>): Higher is better; unit: SCALAR.

- [SAMSum Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=SAMSum%20Score>): Higher is better; unit: SCALAR.

- [LSHT Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=LSHT%20Score>): Higher is better; unit: SCALAR.

- [PassCount](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=PassCount>): Higher is better; unit: SCALAR.

- [PassRe (English) Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=PassRe%20(English)%20Score>): Higher is better; unit: SCALAR.

- [PassRe (Chinese) Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=PassRe%20(Chinese)%20Score>): Higher is better; unit: SCALAR.

- [LCC Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=LCC%20Score>): Higher is better; unit: SCALAR.

- [RB-P Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=RB-P%20Score>): Higher is better; unit: SCALAR.

- [Average Score](<https://www.sota2.com/research/sota/long-context-language-modeling-on-longbench-16k-context-length.md?metric=Average%20Score>): Higher is better; unit: SCALAR.

## Recorded results

| Method | NrtvQA Score | Result date | Report paper |
| --- | --- | --- | --- |
| UltraLLaDA + BA-Att | 13.3 | Not recorded | 6a0d4054f984739ed6737b45 |
| UltraLLaDA | 12.1 | Not recorded | 6a0d4054f984739ed6737b45 |
| LLaDA1.5 + BA-Att | 4.9 | Not recorded | 6a0d4054f984739ed6737b45 |
| LLaDA1.5 + XAtt | 4.3 | Not recorded | 6a0d4054f984739ed6737b45 |
| LLaDA1.5 | 4.1 | Not recorded | 6a0d4054f984739ed6737b45 |
| UltraLLaDA + XAtt | 4.1 | Not recorded | 6a0d4054f984739ed6737b45 |

### UltraLLaDA + BA-Att

NrtvQA Score: 13.3. Result ID: 6a0d7a04fbc373592c05b87a.

[Reporting paper](<https://www.sota2.com/research/paper/6a0d4054f984739ed6737b45.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0d4054f984739ed6737b45.md>)

Recorded evaluation configuration: {"Attention":"BA-Att","Sparsity":"\> 50%"}. This record does not assert independent reproduction.

### UltraLLaDA

NrtvQA Score: 12.1. Result ID: 6a0d7a04fbc373592c05b878.

[Reporting paper](<https://www.sota2.com/research/paper/6a0d4054f984739ed6737b45.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c0a1e209131cea8c08b.md>)

Recorded evaluation configuration: {"Attention":"Dense Attention"}. This record does not assert independent reproduction.

### LLaDA1.5 + BA-Att

NrtvQA Score: 4.9. Result ID: 6a0d7a04fbc373592c05b877.

[Reporting paper](<https://www.sota2.com/research/paper/6a0d4054f984739ed6737b45.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0d4054f984739ed6737b45.md>)

Recorded evaluation configuration: {"Attention":"BA-Att","Sparsity":"\> 50%"}. This record does not assert independent reproduction.

### LLaDA1.5 + XAtt

NrtvQA Score: 4.3. Result ID: 6a0d7a04fbc373592c05b876.

[Reporting paper](<https://www.sota2.com/research/paper/6a0d4054f984739ed6737b45.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77811e209131cea73a62.md>)

Recorded evaluation configuration: {"Attention":"XAtt"}. This record does not assert independent reproduction.

### LLaDA1.5

NrtvQA Score: 4.1. Result ID: 6a0d7a04fbc373592c05b875.

[Reporting paper](<https://www.sota2.com/research/paper/6a0d4054f984739ed6737b45.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c0e1e209131cea8c28e.md>)

Recorded evaluation configuration: {"Attention":"Dense Attention"}. This record does not assert independent reproduction.

### UltraLLaDA + XAtt

NrtvQA Score: 4.1. Result ID: 6a0d7a04fbc373592c05b879.

[Reporting paper](<https://www.sota2.com/research/paper/6a0d4054f984739ed6737b45.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77811e209131cea73a62.md>)

Recorded evaluation configuration: {"Attention":"XAtt"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.