Long-context question-answering on LooGLE (test)
84.2ShortQA ScoreQwen2.5-7B-Instruct-1M
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen2.5-7B-Instruct-1MBackbone=Qwen2.5-7B-Instruct-1M2026.04 | 84.2 | 83 | — | — | — | — | 22.69 | 23.4 | 12.3 | 84.2 | 4.3 | 73.5 | 83.3 | |
| LLMLingua2Backbone=Qwen2.5-7B-Instruct-1M2026.04 | 84.1 | 82.8 | — | — | — | — | 19.56 | 18 | 11.3 | 84.2 | 4.9 | 73.1 | 80.5 | |
| FastVBackbone=Qwen2.5-7B-Instruct-1M2026.04 | 84 | 82.9 | — | — | — | — | 19.78 | 20.2 | 10.6 | 84 | 4 | 73 | 82.4 | |
| DASHBackbone=Qwen2.5-7B-Instruct-1M2026.04 | 83.9 | 82.9 | — | — | — | — | 19.94 | 17.2 | 11.2 | 84.1 | 4 | 73.6 | 83.1 | |
| SnapKV (pr.)Backbone=Qwen2.5-7B-Instruct-1M2026.04 | 83.6 | 82.7 | — | — | — | — | 19.87 | 17.1 | 11.6 | 84.1 | 4.3 | 73.2 | 82.9 | |
| D3Backbone=Qwen2.5-7B-Instruct-1M2026.04 | 83 | 82.2 | — | — | — | — | 19.49 | 12.4 | 7.8 | 83.1 | 3.8 | 72.1 | 81.1 | |
| LIFTBackbone=Llama 22025.02 | 54.65 | 29.26 | 38.53 | 24.35 | 5.68 | 28.87 | — | — | — | — | — | — | — | |
| LLoCOBackbone=Llama-2-7b-instruct2025.02 | 21.04 | 21.92 | 29.82 | 17.1 | 0 | 23.71 | — | — | — | — | — | — | — |