Multi-choice Question Answering on LongBench v2
46.5Overall AccuracyQwen2.5-14B-1M-LongRLVR
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen2.5-14B-1M-LongRLVRBackbone=Qwen2.5-14B-1M, Protocol=LongRLVR2026.03 | 46.5 | 55.6 | 43.3 | 38 | |
| Qwen3-14B (Thinking)Backbone=Qwen3-14B, Thinking Mode=true2026.03 | 44.9 | 51.7 | 42.3 | 38.9 | |
| Qwen2.5-72B-YaRNBackbone=Qwen2.5-72B, Extension=YaRN2026.03 | 43.5 | 43.5 | 48.9 | 40.9 | |
| QwenLong-L1-32BBackbone=QwenLong-L1-32B2026.03 | 41 | 53.3 | 34.4 | 33.3 | |
| Qwen2.5-14B-1MBackbone=Qwen2.5-14B-1M2026.03 | 40.2 | 51.7 | 34 | 33.3 | |
| Qwen2.5-14B-1M-RLVRBackbone=Qwen2.5-14B-1M, Protocol=RLVR2026.03 | 39.8 | 48.3 | 36.7 | 31.5 | |
| Qwen2.5-14B-1M-SFTBackbone=Qwen2.5-14B-1M, Protocol=SFT2026.03 | 39.6 | 48.9 | 34.9 | 33.3 | |
| Qwen2.5-7B-1M-LongRLVRBackbone=Qwen2.5-7B-1M, Protocol=LongRLVR2026.03 | 38.6 | 45.6 | 35.8 | 32.4 | |
| Qwen3-8B (Thinking)Backbone=Qwen3-8B, Thinking Mode=true2026.03 | 37.6 | 43.3 | 28.8 | 32.4 | |
| LLaMA-3.1-8B-LongRLVRBackbone=LLaMA-3.1-8B, Protocol=LongRLVR2026.03 | 36.2 | 41.1 | 30.7 | 38.9 | |
| ShadowKVBackbone=Qwen-2.5-14B-Instruct, KV Cache Management Method=ShadowKV, KV Budget/Sparsity=20482025.05 | 34.79 | 40.56 | 34.88 | 25 | |
| RazorBackbone=Qwen-2.5-14B-Instruct, KV Cache Management Method=Razor, KV Budget/Sparsity=0.152025.05 | 34.19 | 43.33 | 30.7 | 25.93 | |
| ArkValeBackbone=Qwen-2.5-14B-Instruct, KV Cache Management Method=ArkVale, KV Budget/Sparsity=20482025.05 | 34.19 | 41.11 | 33.49 | 24.07 | |
| FreeKVBackbone=Qwen-2.5-14B-Instruct, KV Cache Management Method=FreeKV, KV Budget/Sparsity=20482025.05 | 34.19 | 41.11 | 33.49 | 24.07 | |
| QuestBackbone=Qwen-2.5-14B-Instruct, KV Cache Management Method=Quest, KV Budget/Sparsity=20482025.05 | 33.8 | 40 | 33.49 | 24.07 | |
| FullBackbone=Qwen-2.5-14B-Instruct, KV Cache Management Method=Full, KV Budget/Sparsity=Full Cache2025.05 | 33.4 | 41.11 | 31.16 | 25 | |
| Qwen2.5-7B-1M-SFTBackbone=Qwen2.5-7B-1M, Protocol=SFT2026.03 | 33.2 | 36.7 | 32.6 | 28.7 | |
| Qwen2.5-7B-1MBackbone=Qwen2.5-7B-1M2026.03 | 33 | 37.8 | 31.2 | 28.7 | |
| RaaSBackbone=Qwen-2.5-14B-Instruct, KV Cache Management Method=RaaS, KV Budget/Sparsity=20482025.05 | 32.6 | 40.56 | 32.09 | 20.37 | |
| LLaMA-3.1-8B-RLVRBackbone=LLaMA-3.1-8B, Protocol=RLVR2026.03 | 32.4 | 35.6 | 31.2 | 29.6 | |
| Qwen2.5-7B-1M-RLVRBackbone=Qwen2.5-7B-1M, Protocol=RLVR2026.03 | 32.4 | 37.2 | 29.3 | 30.6 | |
| InfiniGenBackbone=Qwen-2.5-14B-Instruct, KV Cache Management Method=InfiniGen, KV Budget/Sparsity=20482025.05 | 32.31 | 40.56 | 29.89 | 23.95 | |
| LLaMA-3.1-8B-SFTBackbone=LLaMA-3.1-8B, Protocol=SFT2026.03 | 31.2 | 36.1 | 28.4 | 28.7 | |
| LLaMA-3.1-8BBackbone=LLaMA-3.1-8B2026.03 | 30.4 | 34.4 | 31.6 | 21.3 | |
| FullBackbone=Llama-3.1-8B-Instruct, KV Cache Management Method=Full, KV Budget/Sparsity=Full Cache2025.05 | 29.22 | 34.44 | 27.91 | 23.15 | |
| FreeKVBackbone=Llama-3.1-8B-Instruct, KV Cache Management Method=FreeKV, KV Budget/Sparsity=20482025.05 | 29.22 | 35 | 27.44 | 23.15 | |
| ArkValeBackbone=Llama-3.1-8B-Instruct, KV Cache Management Method=ArkVale, KV Budget/Sparsity=20482025.05 | 28.63 | 33.89 | 26.98 | 23.15 | |
| InfiniGenBackbone=Llama-3.1-8B-Instruct, KV Cache Management Method=InfiniGen, KV Budget/Sparsity=20482025.05 | 28.56 | 32.28 | 26.05 | 24.07 | |
| QuestBackbone=Llama-3.1-8B-Instruct, KV Cache Management Method=Quest, KV Budget/Sparsity=20482025.05 | 28.43 | 33.33 | 27.44 | 22.22 | |
| RaaSBackbone=Llama-3.1-8B-Instruct, KV Cache Management Method=RaaS, KV Budget/Sparsity=20482025.05 | 28.23 | 33.89 | 26.51 | 22.02 | |
| QuestBackbone=Qwen-2.5-7B-Instruct, KV Cache Management Method=Quest, KV Budget/Sparsity=20482025.05 | 27.63 | 36.67 | 22.79 | 22.22 | |
| RazorBackbone=Llama-3.1-8B-Instruct, KV Cache Management Method=Razor, KV Budget/Sparsity=0.152025.05 | 27.44 | 33.89 | 25.12 | 21.3 | |
| FullBackbone=Qwen-2.5-7B-Instruct, KV Cache Management Method=Full, KV Budget/Sparsity=Full Cache2025.05 | 27.44 | 36.11 | 23.72 | 20.37 | |
| ArkValeBackbone=Qwen-2.5-7B-Instruct, KV Cache Management Method=ArkVale, KV Budget/Sparsity=20482025.05 | 26.84 | 36.11 | 22.33 | 20.37 | |
| FreeKVBackbone=Qwen-2.5-7B-Instruct, KV Cache Management Method=FreeKV, KV Budget/Sparsity=20482025.05 | 26.84 | 34.44 | 22.33 | 23.15 | |
| InfiniGenBackbone=Qwen-2.5-7B-Instruct, KV Cache Management Method=InfiniGen, KV Budget/Sparsity=20482025.05 | 26.44 | 32.22 | 23.26 | 23.15 | |
| RaaSBackbone=Qwen-2.5-7B-Instruct, KV Cache Management Method=RaaS, KV Budget/Sparsity=20482025.05 | 26.24 | 35.56 | 21.86 | 19.44 | |
| LLaMA-3.1-70BBackbone=LLaMA-3.1-70B2026.03 | 25.9 | 36.2 | 45 | 34 | |
| ShadowKVBackbone=Qwen-2.5-7B-Instruct, KV Cache Management Method=ShadowKV, KV Budget/Sparsity=20482025.05 | 25.84 | 32.22 | 20 | 26.85 | |
| ShadowKVBackbone=Llama-3.1-8B-Instruct, KV Cache Management Method=ShadowKV, KV Budget/Sparsity=20482025.05 | 25.45 | 32.78 | 22.79 | 18.52 | |
| RazorBackbone=Qwen-2.5-7B-Instruct, KV Cache Management Method=Razor, KV Budget/Sparsity=0.152025.05 | 25.25 | 32.78 | 21.86 | 19.44 |