Long-context language evaluation on LongBench
48.43SQABaseline
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| BaselineModel=Meta-Llama-3.1-8B-Instruct, Strategy=Dense2025.12 | 48.43 | 43.18 | 25.99 | 63.22 | 34.83 | 59.89 | 45.92 | |
| OmniKVModel=Meta-Llama-3.1-8B-Instruct, Strategy=decode-only2025.12 | 48.22 | 43.05 | 25.97 | 63.22 | 34.72 | 59.33 | 45.75 | |
| LessIsMoreModel=Meta-Llama-3.1-8B-Instruct, Strategy=decode-only2025.12 | 48.15 | 42.71 | 25.38 | 63.05 | 34.67 | 59.16 | 45.52 | |
| Kascade (All Heads Pooled)Model=Meta-Llama-3.1-8B-Instruct2025.12 | 47.83 | 40.5 | 25.34 | 63.09 | 34.5 | 62.95 | 45.7 | |
| BaselineModel=Qwen3-8B, Strategy=Dense2025.12 | 47.56 | 41.35 | 24.15 | 64.32 | 34.83 | 65.9 | 46.35 | |
| KascadeModel=Meta-Llama-3.1-8B-Instruct2025.12 | 47.41 | 39.84 | 25.21 | 61.32 | 33.67 | 62.7 | 45.02 | |
| PBS-AttnBackbone=Qwen3-8B2025.10 | 47.04 | 37.17 | 16.66 | 33.88 | 66.33 | 2.8 | 33.98 | |
| QuestModel=Meta-Llama-3.1-8B-Instruct, Strategy=decode-only2025.12 | 46.97 | 42.82 | 25.71 | 62.33 | 34.14 | 54.36 | 44.39 | |
| FullBackbone=Qwen3-8B2025.10 | 46.93 | 37.97 | 16.57 | 34.07 | 66 | 2.93 | 34.08 | |
| MInferenceBackbone=Qwen3-8B2025.10 | 46.92 | 37.11 | 16.53 | 34.15 | 66.17 | 2.98 | 33.98 | |
| FlexPrefillBackbone=Qwen3-8B2025.10 | 46.08 | 37.29 | 16.53 | 33.63 | 49 | 2.52 | 30.84 | |
| MeanPoolingBackbone=Qwen3-8B2025.10 | 45.69 | 37.25 | 16.61 | 37.45 | 62.33 | 2.52 | 33.64 | |
| XAttentionBackbone=Qwen3-8B2025.10 | 45.66 | 37.77 | 16.66 | 36.2 | 64.67 | 2.02 | 33.83 | |
| Kascade (All Heads Pooled)Model=Qwen3-8B2025.12 | 44.87 | 42.34 | 23.74 | 61.99 | 34.5 | 62.71 | 45.02 | |
| QuestModel=Qwen3-8B, Strategy=decode-only2025.12 | 44.71 | 40.46 | 24.34 | 62.72 | 33.94 | 57.53 | 43.95 | |
| KascadeModel=Qwen3-8B2025.12 | 44.19 | 40.38 | 23.02 | 60.83 | 35 | 63.98 | 44.57 | |
| LessIsMoreModel=Qwen3-8B, Strategy=decode-only2025.12 | 40.87 | 38.47 | 23.03 | 62.38 | 34.67 | 63.21 | 43.77 | |
| StreamingLLMModel=Meta-Llama-3.1-8B-Instruct, Sliding Window=30%, Sink Tokens=42025.12 | 24.83 | 25.05 | 22.41 | 56.33 | 12 | 59.89 | 33.42 | |
| StreamingLLMModel=Qwen3-8B, Sliding Window=30%, Sink Tokens=42025.12 | 24.33 | 28.44 | 21.01 | 56.82 | 12.5 | 64.32 | 34.57 |