Long-context reasoning on LongReason 64K-input 70K context
71.25AccuracyKVZip
Evaluation results
| Method | Links | |
|---|---|---|
| KVZipBackbone=Qwen-3-4B-Thinking, Sparsity level (sp)=0.22025.10 | 71.25 | |
| FullBackbone=Qwen-3-4B-Thinking2025.10 | 70.25 | |
| DuoAttentionBackbone=Qwen-3-4B-Thinking, Sparsity level (sp)=0.22025.10 | 69.75 | |
| RLKV (Ours)Backbone=Qwen-3-4B-Thinking, Sparsity level (sp)=0.22025.10 | 68.5 | |
| RLKV (Ours)Backbone=Qwen-3-4B-Thinking, Sparsity level (sp)=0.42025.10 | 68.25 | |
| KVZipBackbone=Qwen-3-4B-Thinking, Sparsity level (sp)=0.42025.10 | 67.25 | |
| DuoAttentionBackbone=Qwen-3-4B-Thinking, Sparsity level (sp)=0.42025.10 | 63.25 | |
| RLKV (Ours)Backbone=Qwen-3-4B-Thinking, Sparsity level (sp)=0.62025.10 | 58.25 | |
| DuoAttentionBackbone=Qwen-3-4B-Thinking, Sparsity level (sp)=0.62025.10 | 55.5 | |
| KVZipBackbone=Qwen-3-4B-Thinking, Sparsity level (sp)=0.62025.10 | 53 | |
| RLKV (Ours)Backbone=Llama-3.1-8B-R1, Sparsity level (sp)=0.42025.10 | 52.5 | |
| RLKV (Ours)Backbone=Llama-3.1-8B-R1, Sparsity level (sp)=0.22025.10 | 50.5 | |
| DuoAttentionBackbone=Llama-3.1-8B-R1, Sparsity level (sp)=0.22025.10 | 49.5 | |
| FullBackbone=Llama-3.1-8B-R12025.10 | 49.25 | |
| KVZipBackbone=Llama-3.1-8B-R1, Sparsity level (sp)=0.42025.10 | 49 | |
| DuoAttentionBackbone=Llama-3.1-8B-R1, Sparsity level (sp)=0.42025.10 | 48.75 | |
| KVZipBackbone=Llama-3.1-8B-R1, Sparsity level (sp)=0.22025.10 | 48 | |
| RLKV (Ours)Backbone=Llama-3.1-8B-R1, Sparsity level (sp)=0.62025.10 | 45.25 | |
| KVZipBackbone=Llama-3.1-8B-R1, Sparsity level (sp)=0.62025.10 | 36 | |
| DuoAttentionBackbone=Llama-3.1-8B-R1, Sparsity level (sp)=0.62025.10 | 35.25 | |
| RLKV (Ours)Backbone=Llama-3.1-8B-R1, Sparsity level (sp)=0.82025.10 | 15 | |
| KVZipBackbone=Llama-3.1-8B-R1, Sparsity level (sp)=0.82025.10 | 4.75 | |
| RLKV (Ours)Backbone=Qwen-3-4B-Thinking, Sparsity level (sp)=0.82025.10 | 3 | |
| DuoAttentionBackbone=Llama-3.1-8B-R1, Sparsity level (sp)=0.82025.10 | 1.5 | |
| R-KVBackbone=Qwen-3-4B-Thinking, Sparsity level (sp)=0.22025.10 | 1.25 | |
| DuoAttentionBackbone=Qwen-3-4B-Thinking, Sparsity level (sp)=0.82025.10 | 1 | |
| R-KVBackbone=Qwen-3-4B-Thinking, Sparsity level (sp)=0.42025.10 | 0.75 | |
| R-KVBackbone=Qwen-3-4B-Thinking, Sparsity level (sp)=0.62025.10 | 0.5 | |
| R-KVBackbone=Qwen-3-4B-Thinking, Sparsity level (sp)=0.82025.10 | 0.5 | |
| KVZipBackbone=Qwen-3-4B-Thinking, Sparsity level (sp)=0.82025.10 | 0.25 | |
| R-KVBackbone=Llama-3.1-8B-R1, Sparsity level (sp)=0.22025.10 | 0 | |
| R-KVBackbone=Llama-3.1-8B-R1, Sparsity level (sp)=0.42025.10 | 0 | |
| R-KVBackbone=Llama-3.1-8B-R1, Sparsity level (sp)=0.62025.10 | 0 | |
| R-KVBackbone=Llama-3.1-8B-R1, Sparsity level (sp)=0.82025.10 | 0 |