Long-context language understanding on InfiniteBench
33.01En.SumFull
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FullModel=Qwen3-30B-A3B-Instruct-25072026.03 | 33.01 | 27.35 | 65.94 | 32.5 | 10.05 | 49.24 | 34.86 | 21.19 | 100 | 4.2 | 37.83 | — | — | |
| FlashPrefillModel=Qwen3-30B-A3B-Instruct-25072026.03 | 33.01 | 23.36 | 63.76 | 19.5 | 9.52 | 34.77 | 36.57 | 33.22 | 100 | 8.6 | 36.23 | — | — | |
| Llama-3.1-8B-InstructMem%=100%2026.01 | 32.93 | 27.41 | 65.94 | 19.5 | 34.6 | 25.38 | 24 | 99.66 | 98.64 | — | 47.56 | — | — | |
| HeteroCacheMem%=50%2026.01 | 32.05 | 26.94 | 65.94 | 19.5 | 34.14 | 25.38 | 24 | 99.66 | 98.47 | — | 47.34 | — | — | |
| FullModel=Llama-3.1-8B-Instruct2026.03 | 32.04 | 25.93 | 69 | 21 | 31.75 | 18.02 | 25.14 | 99.32 | 99.66 | 62 | 48.39 | — | — | |
| MInferenceModel=Llama-3.1-8B-Instruct2026.03 | 32.04 | 21.94 | 64.63 | 14.5 | 31.75 | 5.33 | 27.43 | 56.61 | 78.31 | 14 | 34.65 | — | — | |
| FlashPrefillModel=Llama-3.1-8B-Instruct2026.03 | 32.04 | 25.07 | 67.69 | 16.5 | 31.22 | 16.75 | 24.86 | 96.1 | 97.97 | 55 | 46.32 | — | — | |
| CAKEMem%=50%2026.01 | 31.51 | 26.15 | 65.94 | 19 | 33.22 | 25.13 | 24 | 99.66 | 97.46 | — | 46.91 | — | — | |
| OmniKVMem%=50%2026.01 | 31.46 | 27.03 | 65.94 | 19 | 33.98 | 23.1 | 24 | 99.66 | 98.14 | — | 46.92 | — | — | |
| FlexPrefillModel=Qwen3-30B-A3B-Instruct-25072026.03 | 31.07 | 22.22 | 61.57 | 22.5 | 8.47 | 37.31 | 33.14 | 27.46 | 98.14 | 5.4 | 34.73 | — | — | |
| APBBackbone=Llama-3.1-8B-instruct2025.02 | 30.63 | 29.25 | 63.32 | 11 | 36.99 | 30.46 | 26.86 | 100 | 98.81 | 81.8 | 50.91 | — | — | |
| FULLATTNBackbone=Llama-3.1-8B-instruct2025.02 | 30.59 | 29.04 | 63.76 | 11 | 36.18 | 24.62 | 28.82 | 100 | 99.49 | 51 | 47.45 | — | — | |
| STARATTNBackbone=Llama-3.1-8B-instruct2025.02 | 30.55 | 30.66 | 61.57 | 15 | 36.26 | 26.65 | 24.57 | 100 | 98.98 | 40.6 | 46.48 | — | — | |
| FlexPrefillModel=Llama-3.1-8B-Instruct2026.03 | 30.42 | 24.82 | 68.41 | 15.5 | 32.46 | 16.75 | 31.14 | 95.64 | 99.83 | 44 | 45.9 | — | — | |
| XAttentionModel=Llama-3.1-8B-Instruct2026.03 | 30.1 | 24.79 | 68.56 | 15.5 | 32.28 | 13.71 | 27.14 | 92.54 | 93.9 | 39 | 43.75 | — | — | |
| MINFERENCEBackbone=Llama-3.1-8B-instruct2025.02 | 30.06 | 26.42 | 55.46 | 12.5 | 36.51 | 28.17 | 32.29 | 98.47 | 98.81 | 17.4 | 43.61 | — | — | |
| QuestMem%=100%2026.01 | 28.86 | 26.55 | 64 | 18.5 | 31.78 | 24.65 | 22.86 | 99.15 | 98.01 | — | 46.68 | — | — | |
| MInferenceBackbone=GLM-4-9B-1M, Attention=Sparse2024.07 | 28.8 | 9.6 | 68.6 | 38.5 | 12 | 30.7 | 39.1 | 100 | 100 | 43 | 47 | — | — | |
| GLM-4-9B-1MBackbone=GLM-4-9B-1M, Attention=Full Attention2024.07 | 28.3 | 9.7 | 68.6 | 39.5 | 12.1 | 29.4 | 38.9 | 100 | 100 | 41 | 46.7 | — | — | |
| InfLLMBackbone=GLM-4-9B-1M, Attention=Sparse2024.07 | 28 | 7.3 | 45 | 14 | 10.7 | 27.9 | 39.4 | 98 | 100 | 2.6 | 37.3 | — | — | |
| FULLATTNBackbone=Qwen-2.5-14B-instruct2025.02 | 27.8 | 10.4 | 52.84 | 28 | 10.21 | 38.07 | 42.57 | 100 | 100 | 17.8 | 42.68 | — | — | |
| StreamingLLMBackbone=GLM-4-9B-1M, Attention=Sparse2024.07 | 27.7 | 6.4 | 40.2 | 12.5 | 10.8 | 27.7 | 21.1 | 97.1 | 25.6 | 0.6 | 27 | — | — | |
| STARATTNBackbone=Qwen-2.5-14B-instruct2025.02 | 27.48 | 12.44 | 62.88 | 23.5 | 11.03 | 43.4 | 37.71 | 100 | 100 | 18 | 43.69 | — | — | |
| Llama-3.1-8BTokens=128K, Base Model=Llama-3.1-8B2025.05 | 26.8 | 14.8 | 65.9 | 19 | 13.3 | 22.8 | 34 | 100 | 99.3 | — | 44 | — | — | |
| APBBackbone=Qwen-2.5-14B-instruct2025.02 | 26.51 | 12.74 | 66.38 | 25 | 11.22 | 37.31 | 35.71 | 100 | 100 | 62.2 | 47.34 | — | — | |
| EntropyInferModel=Llama-3.1-8B-Instruct, Latent Decoding=Disabled2026.06 | 26.48 | 14.14 | 62.01 | 19 | 12.7 | 25.38 | 35.14 | 100 | 95.42 | — | 43.36 | — | — | |
| BaseModel=Llama-3.1-8B-Instruct2026.06 | 26.32 | 14.44 | 66.81 | 21 | 13.22 | 22.08 | 33.14 | 100 | 99.49 | — | 44.06 | — | — | |
| MINFERENCEBackbone=Qwen-2.5-14B-instruct2025.02 | 25.63 | 12.04 | 61.57 | 16.5 | 11.06 | 41.62 | 48.57 | 100 | 100 | 5.2 | 42.22 | — | — | |
| MInferenceModel=Qwen3-30B-A3B-Instruct-25072026.03 | 25.24 | 22.79 | 59.83 | 17.5 | 7.41 | 33.5 | 30.29 | 22.37 | 92.37 | 6.8 | 31.81 | — | — | |
| XAttentionModel=Qwen3-30B-A3B-Instruct-25072026.03 | 25.24 | 24.79 | 64.19 | 21 | 5.29 | 32.23 | 34 | 29.32 | 96.1 | 7.2 | 33.94 | — | — | |
| CritiPrefillModel=Llama-3.1-8B-Instruct2026.06 | 24.97 | 12.92 | 56.77 | 11 | 11.03 | 26.65 | 34.86 | 100 | 99.15 | — | 41.93 | — | — | |
| InfLLMBackbone=Llama-3-8B-262K, Attention=Sparse2024.07 | 24.1 | 7.8 | 45 | 6 | 11.4 | 19.5 | 32.9 | 100 | 100 | 1.2 | 34.8 | — | — | |
| InfLLMBackbone=LLaMA-3-70B-Instruct-262K2024.07 | 24.1 | 8.1 | 57 | 10 | 12.9 | 27.4 | 52.3 | 100 | 100 | 0 | 39.2 | — | — | |
| TailorKV-2Tokens=128+(896), Base Model=Llama-3.1-8B2025.05 | 24.1 | 14.4 | 66.8 | 18.5 | 12.9 | 22.8 | 34 | 100 | 89.4 | — | 42.6 | — | — | |
| ShadowKVMem%=50%2026.01 | 23.59 | 18.62 | 64.19 | 15.5 | 30.55 | 25.38 | 19.71 | 98.31 | 98.14 | — | 44.26 | — | — | |
| CriticalKVModel=Meta-Llama-3.1-8B-Instruct, Cache budget (Btotal)=1024L2026.05 | 23.34 | 9.31 | 70 | 8 | — | 22 | 53 | 100 | 93 | 0 | 38.07 | — | 2 | |
| AdaKVModel=Meta-Llama-3.1-8B-Instruct, Cache budget (Btotal)=1024L2026.05 | 23.32 | 8.66 | 71 | 9 | — | 22 | 53 | 100 | 94 | 2 | 38.4 | — | 1 | |
| TailorKV-1Tokens=128+(896), Base Model=Llama-3.1-8B2025.05 | 22.8 | 13.6 | 66.4 | 18 | 13 | 22.8 | 34 | 99.8 | 73.2 | — | 40.4 | — | — | |
| SnapKVTokens=1024, Base Model=Llama-3.1-8B2025.05 | 22.4 | 10.4 | 65.5 | 9.5 | 11.3 | 22.8 | 34 | 100 | 93.2 | — | 41 | — | — | |
| SnapKVModel=Llama-3.1-8B-Instruct2026.06 | 22.35 | 11.77 | 66.81 | 11.5 | 12.03 | 22.08 | 33.14 | 100 | 85.59 | — | 40.59 | — | — | |
| EntropyInferModel=Llama-3.1-8B-Instruct, Latent Decoding=Enabled2026.06 | 22.02 | 14.02 | 62.01 | 16.5 | 12.46 | 25.38 | 35.14 | 100 | 95.42 | — | 42.55 | — | — | |
| AdaKVModel=Llama-3.1-8B-Instruct2026.06 | 22 | 12.21 | 66.38 | 15 | 12.05 | 22.08 | 32.86 | 100 | 82.37 | — | 40.55 | — | — | |
| BaseModel=Qwen2.5-7B-Instruct2026.06 | 21.95 | 4.81 | 46.72 | 16 | 9.12 | 26.14 | 38.57 | 100 | 93.22 | — | 39.62 | — | — | |
| LaProxModel=Meta-Llama-3.1-8B-Instruct, Cache budget (Btotal)=1024L2026.05 | 21.64 | 9.83 | 71 | 10 | — | 22 | 53 | 100 | 96 | 4 | 38.95 | — | 2 | |
| StreamingLLMBackbone=Llama-3-8B-262K, Attention=Sparse2024.07 | 21 | 8.2 | 40.2 | 10 | 10.4 | 25.9 | 30 | 86.8 | 5.1 | 0.8 | 23.8 | — | — | |
| LLaMA-3-70B-Instruct-262KBackbone=LLaMA-3-70B-Instruct-262K, Attention=Full Attention2024.07 | 20.7 | 10.3 | 84.2 | 9.5 | 14 | 33.2 | 61.7 | 97 | 100 | 34 | 46.5 | — | — | |
| AdaKVModel=Qwen2.5-7B-Instruct2026.06 | 20.62 | 4.71 | 46.29 | 12 | 9.18 | 25.63 | 35.71 | 83.56 | 4.07 | — | 26.86 | — | — | |
| MInferenceBackbone=LLaMA-3-70B-Instruct-262K2024.07 | 20.6 | 10.1 | 83.4 | 10 | 14.1 | 34.1 | 61.9 | 100 | 100 | 39 | 47.3 | — | — | |
| MInferenceBackbone=Llama-3-8B-262K, Attention=Sparse2024.07 | 20.5 | 12.9 | 65.9 | 7.5 | 12.5 | 22.3 | 33.1 | 100 | 100 | 12.8 | 38.8 | — | — | |
| StreamingLLMBackbone=LLaMA-3-70B-Instruct-262K2024.07 | 20.5 | 8.5 | 52 | 10 | 12.6 | 27.4 | 61.1 | 14 | 10 | 0 | 21.6 | — | — | |
| FlashPrefillModel=Qwen2.5-7B-Instruct2026.03 | 20.39 | 4.84 | 47.16 | 10.5 | 8.99 | 13.2 | 37.43 | 11.19 | 95.59 | 0 | 24.93 | — | — | |
| DenseModel=Llama3.1, Context Window=Truncated to fit2025.06 | 20.36 | 38.33 | 57.64 | 18.5 | 27.57 | 22.59 | 23.71 | 100 | — | — | 45.17 | 97.8 | — | |
| Llama-3-8B-262KBackbone=Llama-3-8B-262K, Attention=Full Attention2024.07 | 20.2 | 12.4 | 67.3 | 6 | 12.9 | 22.1 | 26.6 | 100 | 100 | 14.4 | 38.2 | — | — | |
| StreamingLLM w/ dilatedBackbone=Llama-3-8B-262K, Attention=Sparse2024.07 | 20.1 | 9.4 | 44.5 | 15.5 | 11.2 | 20.5 | 27.5 | 5 | 87.5 | 0.5 | 24.2 | — | — | |
| SnapKVModel=Qwen2.5-7B-Instruct2026.06 | 19.95 | 4.6 | 46.72 | 11 | 9.32 | 25.63 | 35.14 | 85.42 | 4.41 | — | 26.91 | — | — | |
| MInference (static)Backbone=Llama-3-8B-262K, Attention=Sparse2024.07 | 19.9 | 8.6 | 43.2 | 3.5 | 8.9 | 20.6 | 25.1 | 92.4 | 96.3 | 0.2 | 31.9 | — | — | |
| CritiPrefillModel=Qwen2.5-7B-Instruct2026.06 | 19.84 | 4.43 | 37.99 | 10.5 | 8.39 | 24.11 | 34.57 | 99.15 | 89.49 | — | 36.5 | — | — | |
| EntropyInferModel=Qwen2.5-7B-Instruct, Latent Decoding=Enabled2026.06 | 19.75 | 5.26 | 44.1 | 12 | 9.02 | 23.35 | 44.57 | 96.44 | 86.78 | — | 37.92 | — | — | |
| MTrainingTraining strategy=MTraining, Inference strategy=Dense2025.10 | 19.5 | 6.8 | 65.3 | 3.5 | — | 34.1 | — | — | — | — | 25.8 | — | — | |
| MTrainingTraining strategy=MTraining, Inference strategy=MInference2025.10 | 19.5 | 6.7 | 63.3 | 5 | — | 15.5 | — | — | — | — | 22 | — | — | |
| EntropyInferModel=Qwen2.5-7B-Instruct, Latent Decoding=Disabled2026.06 | 19.4 | 4.84 | 44.98 | 15 | 8.64 | 23.35 | 44.57 | 96.44 | 86.78 | — | 38.22 | — | — | |
| HATAModel=Llama3.1, Sparse token budget=2048, Context Window=Truncated to fit2025.06 | 19.27 | 37.52 | 57.64 | 18.5 | 27.27 | 22.59 | 23.71 | 100 | — | — | 44.77 | 96.44 | — | |
| PQCacheTokens=1024, Base Model=Llama-3.1-8B2025.05 | 18.9 | 12.6 | 65.9 | 15 | 12.6 | 23.3 | 34 | 8.6 | 2.5 | — | 21.5 | — | — | |
| DenseTraining strategy=Dense, Inference strategy=Dense2025.10 | 18.5 | 8.2 | 63.5 | 6 | — | 26.3 | — | — | — | — | 24.5 | — | — | |
| FlexPrefillModel=Qwen2.5-7B-Instruct2026.03 | 18.45 | 4.84 | 48.47 | 9.5 | 7.94 | 17.51 | 29.71 | 4.41 | 95.59 | 0 | 23.64 | — | — | |
| FullModel=Qwen2.5-7B-Instruct2026.03 | 17.48 | 5.13 | 44.54 | 15.5 | 8.99 | 18.53 | 34 | 0.34 | 94.24 | 0 | 23.87 | — | — | |
| XAttentionModel=Qwen2.5-7B-Instruct2026.03 | 17.48 | 3.99 | 49.78 | 13.5 | 6.88 | 18.78 | 30 | 1.36 | 93.73 | 0 | 23.55 | — | — | |
| DenseTraining strategy=Dense, Inference strategy=MInference2025.10 | 17.4 | 6.7 | 63 | 4.4 | — | 17 | — | — | — | — | 21.7 | — | — | |
| StreamingLLM w/ stridedBackbone=Llama-3-8B-262K, Attention=Sparse2024.07 | 17.3 | 8.2 | 27.5 | 14.5 | 11.2 | 19.5 | 27.5 | 4 | 2.1 | 1 | 13.3 | — | — | |
| MInferenceModel=Qwen2.5-7B-Instruct2026.03 | 16.5 | 3.7 | 39.74 | 12.5 | 7.94 | 16.24 | 24.29 | 6.78 | 91.36 | 0 | 21.9 | — | — | |
| FlashMoBAModel=Llama-3.1-8B-Instruct2026.03 | 14.56 | 12.82 | 26.2 | 5 | 14.29 | 2.03 | 11.14 | 44.24 | 43.05 | 0 | 17.33 | — | — | |
| StreamLLMTokens=1024, Base Model=Llama-3.1-8B2025.05 | 12.7 | 5.9 | 66.3 | 7 | 9.7 | 22.8 | 34 | 3.3 | 3 | — | 18.3 | — | — | |
| QuestTokens=1024, Base Model=Llama-3.1-8B2025.05 | 12.2 | 9.2 | 69.8 | 14 | 11.4 | 25.1 | 34 | 100 | 28.9 | — | 33.8 | — | — | |
| PQCacheTokens=1024, Base Model=Yi-9B2025.05 | 8.7 | 11.2 | 66.3 | 2 | 14.9 | 25.6 | 22.2 | 9.6 | 5.9 | — | 18.5 | — | — | |
| SnapKVTokens=1024, Base Model=Yi-9B2025.05 | 8.6 | 8.4 | 67.6 | 3 | 14.9 | 26.6 | 22.5 | 99.8 | 18.3 | — | 30 | — | — | |
| Yi-9B-200KBackbone=Yi-9B-200K, Attention=Full Attention2024.07 | 8.2 | 10.6 | 64.2 | 1 | 17.3 | 21.3 | 23.4 | 99.8 | 100 | 28.8 | 37.5 | — | — | |
| Yi-9BTokens=200K, Base Model=Yi-9B2025.05 | 8.2 | 10.8 | 65 | 2.5 | 16.7 | 26.3 | 23.4 | 100 | 100 | — | 39.2 | — | — | |
| TailorKV-2Tokens=128+(896), Base Model=Yi-9B2025.05 | 8.2 | 11.2 | 65.1 | 4.5 | 16.6 | 24.9 | 24 | 100 | 99.7 | — | 39.4 | — | — | |
| MInferenceBackbone=Yi-9B-200K, Attention=Sparse2024.07 | 7.9 | 11.2 | 64.2 | 1 | 17.9 | 24.1 | 23.1 | 99.5 | 100 | 27.6 | 37.7 | — | — | |
| TailorKV-1Tokens=128+(896), Base Model=Yi-9B2025.05 | 7.3 | 15.1 | 64.2 | 2.5 | 19.7 | 21.3 | 24 | 100 | 98.3 | — | 39.2 | — | — | |
| MINFERENCEBackbone=Yi-34B-200K2025.02 | 7.19 | 20.07 | 63.32 | 2.5 | 25.4 | 28.17 | 31.4 | 100 | 100 | 52.4 | 43.05 | — | — | |
| FlashMoBAModel=Qwen3-30B-A3B-Instruct-25072026.03 | 6.8 | 8.26 | 27.95 | 12 | 2.65 | 7.61 | 13.14 | 19.15 | 44.24 | 0 | 14.18 | — | — | |
| StreamLLMTokens=1024, Base Model=Yi-9B2025.05 | 6.4 | 8.8 | 66.8 | 2.5 | 15 | 21.3 | 23.7 | 2.5 | 0.5 | — | 16.4 | — | — | |
| InfLLMBackbone=Yi-9B-200K, Attention=Sparse2024.07 | 6.3 | 13 | 45.9 | 2.5 | 21.5 | 20.6 | 34.6 | 85.3 | 88.1 | 1.4 | 31.9 | — | — | |
| StreamingLLM w/ stridedBackbone=Yi-9B-200K, Attention=Sparse2024.07 | 6.1 | 4.5 | 9.8 | 0 | 16.9 | 0 | 3.1 | 1.5 | 0 | 0 | 4.6 | — | — | |
| APBBackbone=Yi-34B-200K2025.02 | 5.96 | 19.81 | 62.45 | 1 | 25.42 | 27.92 | 30 | 100 | 100 | 65.2 | 43.78 | — | — | |
| FULLATTNBackbone=Yi-34B-200K2025.02 | 5.83 | 17.57 | 47.6 | 2 | 18.77 | 25.13 | 28 | 100 | 100 | 49 | 39.39 | — | — | |
| MInference (static)Backbone=Yi-9B-200K, Attention=Sparse2024.07 | 5.8 | 12.6 | 48.5 | 3 | 12.6 | 20.8 | 25.1 | 60.9 | 38.5 | 1 | 22.9 | — | — | |
| StreamingLLM w/ dilatedBackbone=Yi-9B-200K, Attention=Sparse2024.07 | 5.7 | 4.2 | 15 | 0 | 18.2 | 0 | 2.9 | 0 | 0 | 0 | 4.2 | — | — | |
| StreamingLLMBackbone=Yi-9B-200K, Attention=Sparse2024.07 | 5.4 | 14.2 | 38 | 4 | 18.8 | 18.8 | 22.3 | 39.2 | 6.1 | 1.6 | 16.8 | — | — | |
| PQCacheTokens=1024, Base Model=Yi-6B2025.05 | 4.4 | 16.4 | 51.5 | 1.5 | 26.2 | 26.9 | 5.7 | 10.1 | 3.7 | — | 16.3 | — | — | |
| TailorKV-1Tokens=128+(896), Base Model=Yi-6B2025.05 | 4.4 | 17.4 | 53.3 | 2.5 | 26 | 26.4 | 7.7 | 100 | 97.5 | — | 37.2 | — | — | |
| STARATTNBackbone=Yi-34B-200K2025.02 | 4.38 | 19.71 | 60.26 | 0 | 23.4 | 28.17 | 21.14 | 100 | 100 | 24.6 | 38.17 | — | — | |
| TailorKV-2Tokens=128+(896), Base Model=Yi-6B2025.05 | 4 | 18 | 53.3 | 3 | 25.8 | 26.7 | 8 | 100 | 97 | — | 37.3 | — | — | |
| StreamLLMTokens=1024, Base Model=Yi-6B2025.05 | 3.9 | 10.2 | 52.8 | 0 | 20.2 | 25.8 | 4.8 | 3.3 | 0.6 | — | 13.5 | — | — | |
| QuestTokens=1024, Base Model=Yi-6B2025.05 | 3.6 | 13.4 | 52.4 | 3 | 18.8 | 26.6 | 5.1 | 100 | 99.1 | — | 35.8 | — | — | |
| QuestTokens=1024, Base Model=Yi-9B2025.05 | 3.4 | 12.5 | 58.5 | 4 | 12.7 | 18.7 | 18.2 | 100 | 96.9 | — | 36.1 | — | — | |
| Yi-6BTokens=200K, Base Model=Yi-6B2025.05 | 3.4 | 18.2 | 53.3 | 1 | 26 | 26.9 | 6.8 | 100 | 98.4 | — | 37.1 | — | — | |
| SnapKVTokens=1024, Base Model=Yi-6B2025.05 | 3.1 | 14.1 | 53.2 | 2.5 | 22.2 | 26.9 | 4.5 | 100 | 11.5 | — | 26.4 | — | — | |
| FlashMoBAModel=Qwen2.5-7B-Instruct2026.03 | 2.91 | 1.14 | 11.79 | 6 | 2.12 | 7.11 | 13.14 | 0 | 41.19 | 0 | 8.54 | — | — |