Long-context language understanding on RULER (4k Context)
0FWE RateE2LLM-R
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| E2LLM-RContext Length=4K2024.09 | 0 | — | — | — | — | 3 | — | — | — | 8.28 | 1.5 | 35 | |
| E2LLM-C+RContext Length=4K2024.09 | 2 | — | — | — | — | 7.2 | — | — | — | 66.22 | 32.4 | 37 | |
| LongLLMLinguaContext Length=4K2024.09 | 4.67 | — | — | — | — | 5.2 | — | — | — | 27.44 | 7.6 | 14.5 | |
| LongLoRAContext Length=4K2024.09 | 9.33 | — | — | — | — | 1.6 | — | — | — | 41.07 | 16.6 | 55.5 | |
| Full AttentionAttention mechanism=Full Attention, Budget=Full context, Backbone=Llama3-8B-Instruct-Gradient-1048k2026.02 | 10 | 100 | 89.6 | 87.8 | 79.2 | 17.4 | 79.8 | 56.4 | 63.7 | — | — | — | |
| LycheeDecodeAttention mechanism=Sparse Attention, Budget=4096 tokens, Backbone=Llama3-8B-Instruct-Gradient-1048k2026.02 | 10 | 100 | 89.4 | 88.4 | 78.9 | 17.3 | 80 | 56.2 | 63.7 | — | — | — | |
| E2LLM-CContext Length=4K2024.09 | 13.67 | — | — | — | — | 1.2 | — | — | — | 2.45 | 6.8 | 35.5 | |
| YaRNContext Length=4K2024.09 | 20.33 | — | — | — | — | 9.8 | — | — | — | 44 | 15.2 | 7 | |
| LLoCOContext Length=4K2024.09 | 24.67 | — | — | — | — | 0 | — | — | — | 0.44 | 27.7 | 32.5 | |
| CCA-attentionBackbone=LLaMA2-7B-32K2026.05 | 45.67 | — | — | — | — | — | — | — | — | — | 41.1 | — | |
| StreamingLLMBackbone=LLaMA2-7B-32K2026.05 | 47.83 | — | — | — | — | — | — | — | — | — | 61.8 | — | |
| LLaMA2-7B-32K (Vanilla Self-Attention)Backbone=LLaMA2-7B-32K, Attention=Vanilla Self-Attention2026.05 | 48 | — | — | — | — | — | — | — | — | — | 60.65 | — | |
| MInferenceBackbone=LLaMA2-7B-32K2026.05 | 48.67 | — | — | — | — | — | — | — | — | — | 60.35 | — | |
| DuoAttentionBackbone=LLaMA2-7B-32K2026.05 | 48.83 | — | — | — | — | — | — | — | — | — | 60.15 | — | |
| Training–Inference Consistent Segmented ExecutionBackbone=LLaMA2-7B-32K2026.05 | 53.83 | — | — | — | — | — | — | — | — | — | 70.6 | — | |
| LLama2-7B-chatContext Length=4K2024.09 | 74.33 | — | — | — | — | 27 | — | — | — | 95.87 | 85.6 | 63 |