Long-context language understanding on LongBench (test)
51.87Average ScoreFull-context
Evaluation results
| Method | Links | ||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Full-contextModel=Ministral-8B-Instruct, Prefill=100%, KV=100%, Execution strategy=Full-context2025.02 | 51.87 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 41.58 | 49.45 | 27.72 | 70.83 | 67.13 | 54.5 | — | — | — | — | — | — | — | — | — | |
| SnapKVModel=Ministral-8B-Instruct, Prefill=100%, KV=20%, Execution strategy=Decoding-Only2025.02 | 51.35 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 40.63 | 49.59 | 25.67 | 70.64 | 67.07 | 54.5 | — | — | — | — | — | — | — | — | — | |
| FastKVModel=Ministral-8B-Instruct, Prefill=60%, KV=20%, Execution strategy=Prefill-Aware2025.02 | 51.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 41.08 | 49.2 | 25.02 | 71.41 | 64.68 | 55 | — | — | — | — | — | — | — | — | — | |
| SnapKVModel=Ministral-8B-Instruct, Prefill=100%, KV=10%, Execution strategy=Decoding-Only2025.02 | 50.68 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39.97 | 49.43 | 23.92 | 70.03 | 65.75 | 55 | — | — | — | — | — | — | — | — | — | |
| FastKVModel=Ministral-8B-Instruct, Prefill=60%, KV=10%, Execution strategy=Prefill-Aware2025.02 | 50.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39.71 | 49.63 | 23.38 | 70.71 | 64 | 55 | — | — | — | — | — | — | — | — | — | |
| Full-contextModel=LLaMA-3.1-8B-Instruct, Prefill=100%, KV=100%, Execution strategy=Full-context2025.02 | 50.19 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 43.58 | 44.65 | 29.22 | 69.48 | 60.01 | 54.21 | — | — | — | — | — | — | — | — | — | |
| AdmTreeBackbone=Qwen-2-7B, Method category=Compression-based Methods, Latency=7.0, Speedup=3.4x2025.12 | 49.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 41.6 | 45.9 | 30.2 | 69.9 | 66.8 | — | — | — | — | — | — | — | — | — | — | |
| SnapKVModel=LLaMA-3.1-8B-Instruct, Prefill=100%, KV=20%, Execution strategy=Decoding-Only2025.02 | 49.43 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 43.27 | 43.92 | 26.75 | 68.21 | 60.73 | 53.73 | — | — | — | — | — | — | — | — | — | |
| FastKVModel=LLaMA-3.1-8B-Instruct, Prefill=60%, KV=20%, Execution strategy=Prefill-Aware2025.02 | 49.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 42.93 | 43.74 | 26.03 | 69.51 | 68.64 | 53.56 | — | — | — | — | — | — | — | — | — | |
| FullBackbone=Llama-3.1-8B, Budget=Full2026.01 | 49.06 | 31.06 | 45.43 | 53.78 | 55.04 | 47.14 | 31.29 | 34.87 | 25.33 | 27.49 | 72.5 | 91.25 | 43.81 | 6 | 99.5 | 63.36 | 56.65 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FullBackbone=Qwen2.5-7B, Budget=Full2026.01 | 48.87 | 29.05 | 43.34 | 52.52 | 57.59 | 47.05 | 30.24 | 31.78 | 23.64 | 23.96 | 72.5 | 89.47 | 45.61 | 8.5 | 100 | 59.61 | 67.12 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TAPPABackbone=Llama-3.1-8B, Budget=20482026.01 | 48.73 | 30.7 | 45.69 | 53.06 | 55.49 | 46.68 | 30.94 | 30.54 | 24.65 | 27.12 | 71 | 91.65 | 43 | 6 | 99.5 | 64.93 | 58.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SnapKVModel=LLaMA-3.1-8B-Instruct, Prefill=100%, KV=10%, Execution strategy=Decoding-Only2025.02 | 48.73 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 41.9 | 44.11 | 25.34 | 68.17 | 59.53 | 53.72 | — | — | — | — | — | — | — | — | — | |
| PyramidKVBackbone=Llama-3.1-8B, Budget=20482026.01 | 48.51 | 31.13 | 45.06 | 53.8 | 55.78 | 46.59 | 30.89 | 30.25 | 24.82 | 27.35 | 71 | 91.65 | 42.62 | 6 | 99.5 | 63.27 | 56.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Full-KVModel=Qwen2.5-32B, Compression ratio=0%2026.02 | 48.51 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 42.91 | 54.15 | 27.33 | 68.91 | 42.35 | 55.75 | — | — | — | — | — | — | — | — | — | |
| FastKVModel=LLaMA-3.1-8B-Instruct, Prefill=60%, KV=10%, Execution strategy=Prefill-Aware2025.02 | 48.47 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 41.95 | 43.73 | 24.78 | 69.45 | 57.45 | 53.43 | — | — | — | — | — | — | — | — | — | |
| TAPPABackbone=Qwen2.5-7B, Budget=20482026.01 | 48.46 | 29.18 | 44.03 | 52.24 | 57.36 | 45.77 | 30.19 | 29.14 | 23.31 | 23.62 | 69 | 89.47 | 44.99 | 8 | 100 | 60.95 | 68.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TAPPABackbone=Llama-3.1-8B, Budget=10242026.01 | 48.43 | 30.77 | 44.94 | 52.14 | 55.43 | 46.99 | 31.16 | 28.72 | 24.9 | 26.65 | 69.5 | 91.95 | 42.38 | 6 | 99.5 | 64.99 | 58.84 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CAKEBackbone=Llama-3.1-8B, Budget=20482026.01 | 48.43 | 30.79 | 45.83 | 53.57 | 55.5 | 46.6 | 30.47 | 31.12 | 24.67 | 27.16 | 70.5 | 91.48 | 43.48 | 6 | 99.5 | 63.23 | 56.64 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SnapKVBackbone=Llama-3.1-8B, Budget=20482026.01 | 48.32 | 30.99 | 45.06 | 53.15 | 55.25 | 46.56 | 30.78 | 30.24 | 24.63 | 27.32 | 70.5 | 91.48 | 42.37 | 6 | 99.5 | 63.28 | 56.86 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CAKEBackbone=Qwen2.5-7B, Budget=20482026.01 | 48.31 | 29.08 | 43.35 | 51.92 | 57.2 | 45.77 | 30.26 | 29.35 | 23.46 | 23.59 | 69 | 89.37 | 45.37 | 8 | 100 | 59.35 | 67.88 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LU-KVModel=Qwen2.5-32B, Eviction Metric=KeyDiff Metric (π2), Compression ratio=80%2026.02 | 48.26 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 41.58 | 54.23 | 26.61 | 67.86 | 46.91 | 53.75 | — | — | — | — | — | — | — | — | — | |
| PyramidKVBackbone=Qwen2.5-7B, Budget=20482026.01 | 48.17 | 28.39 | 43.36 | 51.83 | 56.75 | 45.6 | 30.5 | 26.9 | 23.03 | 23.38 | 71 | 88.22 | 45.01 | 8 | 100 | 61.36 | 67.37 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CAKEBackbone=Llama-3.1-8B, Budget=10242026.01 | 48.13 | 30.88 | 44.95 | 52.38 | 55.49 | 46.99 | 30.82 | 28.68 | 24.91 | 26.39 | 69 | 91.94 | 42.6 | 6 | 99.5 | 62.65 | 56.89 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SnapKVBackbone=Qwen2.5-7B, Budget=20482026.01 | 48.12 | 29.11 | 41.53 | 52.05 | 57.17 | 46.26 | 30.69 | 29.49 | 23.23 | 23.64 | 71.5 | 89.17 | 45.49 | 8 | 100 | 60.92 | 67.94 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SnapKVBackbone=Llama-3.1-8B, Budget=10242026.01 | 47.95 | 30.95 | 44.74 | 52.58 | 55.09 | 46.83 | 30.37 | 27.87 | 24.57 | 25.99 | 68.5 | 92.03 | 42.6 | 6.5 | 99.5 | 63 | 56.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LU-KVModel=Qwen2.5-32B, Eviction Metric=SnapKV Metric (π1), Compression ratio=80%2026.02 | 47.95 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39.84 | 53.55 | 26.19 | 67.32 | 48.46 | 54.75 | — | — | — | — | — | — | — | — | — | |
| TAPPABackbone=Qwen2.5-7B, Budget=10242026.01 | 47.72 | 29.64 | 43.59 | 51.53 | 56.9 | 45.86 | 29.43 | 26.64 | 22.57 | 23 | 65.5 | 89.48 | 45.24 | 8 | 100 | 60.04 | 66.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CAKEBackbone=Qwen2.5-7B, Budget=10242026.01 | 47.7 | 29.47 | 42.71 | 52.12 | 56.11 | 46.41 | 29.13 | 26.86 | 23.12 | 22.72 | 67.5 | 89.23 | 45.46 | 8.5 | 100 | 59.11 | 64.79 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PyramidKVBackbone=Llama-3.1-8B, Budget=10242026.01 | 47.69 | 30.54 | 43.64 | 52.73 | 55.29 | 46.29 | 31.28 | 27.53 | 24.5 | 26 | 68 | 92.09 | 41.75 | 6.05 | 99.5 | 62.35 | 55.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Original LLM-FTBackbone=Qwen-2-7B, Latency=23.52025.12 | 47.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 41 | 40.6 | 26.8 | 68.5 | 66.1 | — | — | — | — | — | — | — | — | — | — | |
| H2OBackbone=Llama-3.1-8B, Budget=20482026.01 | 47.3 | 29.65 | 39.53 | 48.64 | 54.23 | 46.5 | 29.28 | 29.97 | 23.68 | 27.21 | 68.5 | 91.48 | 43.06 | 6.11 | 99.5 | 63.06 | 56.91 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Full-KVModel=Mistral-7B-v0.3, Compression ratio=0%2026.02 | 47.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 38.36 | 37.83 | 28.86 | 70.86 | 63.26 | 51.25 | — | — | — | — | — | — | — | — | — | |
| SnapKVBackbone=Qwen2.5-7B, Budget=10242026.01 | 47.27 | 29.24 | 41.61 | 50.93 | 57.6 | 45.5 | 29.39 | 25.63 | 23.06 | 22.26 | 65.5 | 88.92 | 44.65 | 8.5 | 100 | 58.16 | 65.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TAPPABackbone=Llama-3.1-8B, Budget=5122026.01 | 47.21 | 29.47 | 42.66 | 51.63 | 54.53 | 46.64 | 30.81 | 25.48 | 24.57 | 24.71 | 62.5 | 92.35 | 42.42 | 6.25 | 99.5 | 64.56 | 57.35 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BeaconBackbone=Qwen-2-7B, Method category=Compression-based Methods, Latency=7.3, Speedup=3.2x2025.12 | 47.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 40.5 | 40.3 | 26.8 | 68.4 | 66.4 | — | — | — | — | — | — | — | — | — | — | |
| CAKEBackbone=Llama-3.1-8B, Budget=5122026.01 | 47.19 | 31.82 | 42.99 | 51.65 | 54.37 | 46.89 | 30.73 | 26.36 | 24.94 | 25.27 | 63.5 | 91.54 | 42.52 | 6.33 | 99.5 | 62.3 | 54.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PyramidInferModel=Ministral-8B-Instruct, Prefill=60%, KV=60%, Execution strategy=Prefill-Aware2025.02 | 47.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 34.29 | 37.34 | 24.16 | 68.19 | 65.83 | 52.25 | — | — | — | — | — | — | — | — | — | |
| H2OBackbone=Qwen2.5-7B, Budget=20482026.01 | 46.95 | 28.55 | 40.2 | 47.45 | 53.49 | 44.44 | 27 | 28.93 | 22.66 | 23.79 | 63.5 | 88.5 | 46.08 | 8 | 100 | 61.06 | 67.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| H2OModel=Ministral-8B-Instruct, Prefill=100%, KV=20%, Execution strategy=Decoding-Only2025.02 | 46.95 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 38.93 | 44.27 | 26.38 | 69.3 | 63.58 | 39.25 | — | — | — | — | — | — | — | — | — | |
| SnapKVBackbone=Llama-3.1-8B, Budget=5122026.01 | 46.92 | 30.76 | 42.03 | 52.13 | 54.15 | 46.14 | 30.51 | 24.98 | 24.24 | 24.65 | 64 | 92.05 | 42.04 | 6.08 | 99.5 | 62.62 | 54.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PyramidKVBackbone=Llama-3.1-8B, Budget=5122026.01 | 46.59 | 30.47 | 42.15 | 52.17 | 54.67 | 45.25 | 30.6 | 25 | 24.33 | 24.51 | 62.5 | 91.24 | 41.67 | 5.95 | 99.5 | 61.58 | 53.89 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GemFilterModel=Ministral-8B-Instruct, Prefill=70%, KV=20%, Execution strategy=Prefill-Aware2025.02 | 46.37 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 40.84 | 54.06 | 25.31 | 68.2 | 38.56 | 51.25 | — | — | — | — | — | — | — | — | — | |
| LU-KVModel=Mistral-7B-v0.3, Eviction Metric=KeyDiff Metric (π2), Compression ratio=80%2026.02 | 46.21 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39.8 | 38.09 | 28.22 | 68.32 | 56.04 | 52.02 | — | — | — | — | — | — | — | — | — | |
| H2OBackbone=Llama-3.1-8B, Budget=10242026.01 | 46.11 | 29.57 | 36.15 | 45.94 | 54.43 | 44.81 | 29.04 | 27.64 | 23.31 | 26.47 | 62 | 91.43 | 43.14 | 6.36 | 99 | 62.24 | 55.74 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| H2OModel=Ministral-8B-Instruct, Prefill=100%, KV=10%, Execution strategy=Decoding-Only2025.02 | 45.94 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.47 | 43.88 | 25.96 | 68.64 | 60.95 | 38.75 | — | — | — | — | — | — | — | — | — | |
| PyramidKVBackbone=Qwen2.5-7B, Budget=10242026.01 | 45.85 | 29.34 | 38.6 | 50.17 | 55.67 | 45.12 | 27.82 | 23.26 | 22.16 | 20.55 | 62.5 | 86.85 | 43.26 | 8.5 | 100 | 57.76 | 61.99 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LU-KVModel=Mistral-7B-v0.3, Eviction Metric=SnapKV Metric (π1), Compression ratio=80%2026.02 | 45.79 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.16 | 36.59 | 27.97 | 69.81 | 57.69 | 51.35 | — | — | — | — | — | — | — | — | — | |
| TAPPABackbone=Qwen2.5-7B, Budget=5122026.01 | 45.78 | 28.97 | 39.4 | 50.46 | 55.48 | 44.47 | 28.02 | 23.99 | 22.87 | 20.72 | 55 | 87.02 | 44.66 | 8 | 100 | 59.04 | 64.39 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Original LLMBackbone=Qwen-2-7B, Latency=23.5, Speedup=1.0x2025.12 | 45.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 38.8 | 37.5 | 26.7 | 70.1 | 60.3 | — | — | — | — | — | — | — | — | — | — | |
| PyramidKVBackbone=Qwen2.5-7B, Budget=5122026.01 | 45.58 | 27.33 | 38.04 | 50.38 | 55.73 | 44.28 | 27.12 | 22.24 | 21.86 | 19.54 | 66 | 86.36 | 43.69 | 8 | 99 | 57.59 | 62.09 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CAKEBackbone=Qwen2.5-7B, Budget=5122026.01 | 45.56 | 28.97 | 39.46 | 50.4 | 54.8 | 44.7 | 28.02 | 23.9 | 22.35 | 20.74 | 55 | 86.91 | 44.92 | 8 | 99.5 | 57.06 | 64.26 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SnapKVBackbone=Qwen2.5-7B, Budget=5122026.01 | 45.51 | 28.94 | 40.7 | 50.4 | 55.8 | 44.21 | 27.83 | 24.42 | 22.74 | 21.07 | 66.5 | 86.56 | 44.14 | 8 | 99.5 | 59.17 | 64.22 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SnapKVBackbone=Qwen-2-7B, Method category=Compression-based Methods, Latency=7.8, Speedup=3.0x2025.12 | 45 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 38.7 | 37.6 | 26.2 | 67.1 | 60.3 | — | — | — | — | — | — | — | — | — | — | |
| DensePer-Query Budget=-, Backbone=Trado-8B-Instruct2026.04 | 44.86 | 19.04 | 17.75 | — | 49.45 | — | — | — | — | — | — | 84.79 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 53.29 | |
| H2OBackbone=Llama-3.1-8B, Budget=5122026.01 | 44.39 | 27.76 | 29.01 | 44.75 | 52.78 | 44.31 | 29.22 | 24.71 | 23.11 | 24.56 | 54.5 | 91.38 | 42.1 | 6.36 | 99 | 62.3 | 54.33 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| StreamingLLMBackbone=Llama-3.1-8B, Budget=20482026.01 | 44.39 | 27.4 | 36.91 | 37.85 | 49.23 | 44.66 | 24.31 | 28.57 | 21.67 | 27.12 | 67.5 | 90.98 | 42.49 | 6.12 | 87 | 63.06 | 55.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| StreamingLLMModel=LLaMA-3.1-8B-Instruct, Prefill=100%, KV=20%, Execution strategy=Decoding-Only2025.02 | 44.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 30.43 | 38.89 | 24.19 | 65.41 | 59.44 | 46.83 | — | — | — | — | — | — | — | — | — | |
| H2OBackbone=Qwen2.5-7B, Budget=10242026.01 | 44.14 | 26.45 | 34.94 | 40.49 | 48.63 | 42.02 | 22.27 | 25.67 | 20.9 | 22.41 | 59 | 87.83 | 45.07 | 8.5 | 98.48 | 59.77 | 63.88 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AdmTreeBackbone=Llama-2-7B, Method category=Compression-based Methods, Latency=7.8, Speedup=3.3x2025.12 | 44.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 36.5 | 36.3 | 26.9 | 65.5 | 60.9 | — | — | — | — | — | — | — | — | — | — | |
| H2OBackbone=Qwen2.5-7B, Budget=5122026.01 | 44.07 | 26.83 | 34.17 | 41.43 | 50.8 | 41.83 | 22.82 | 25.57 | 21.35 | 22.03 | 60.5 | 84.67 | 45.86 | 8 | 95.5 | 59.11 | 64.66 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LoSAPer-Query Budget=1024, Backbone=Trado-8B-Instruct2026.04 | 42.88 | 18.89 | 15.78 | — | 48.45 | — | — | — | — | — | — | 82.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 48.94 | |
| StreamingLLMModel=LLaMA-3.1-8B-Instruct, Prefill=100%, KV=10%, Execution strategy=Decoding-Only2025.02 | 42.85 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 28.67 | 37.01 | 22.28 | 63.19 | 58.01 | 47.96 | — | — | — | — | — | — | — | — | — | |
| GemFilterModel=Ministral-8B-Instruct, Prefill=60%, KV=10%, Execution strategy=Prefill-Aware2025.02 | 42.82 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.92 | 55.22 | 23.3 | 65.05 | 30.88 | 46 | — | — | — | — | — | — | — | — | — | |
| StreamingLLMBackbone=Llama-3.1-8B, Budget=10242026.01 | 42.73 | 26.64 | 30.77 | 35.59 | 47.31 | 42.03 | 24.17 | 25.81 | 21.31 | 25.66 | 63.5 | 88.84 | 42.76 | 6.5 | 88 | 61.31 | 53.47 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GemFilterModel=LLaMA-3.1-8B-Instruct, Prefill=61%, KV=20%, Execution strategy=Prefill-Aware2025.02 | 42.39 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 34.63 | 40.77 | 24.39 | 66.39 | 36.56 | 51.59 | — | — | — | — | — | — | — | — | — | |
| AdaModel=Qwen2.5-32B, Eviction Metric=KeyDiff Metric (π2), Compression ratio=80%2026.02 | 42.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 32.16 | 46.44 | 23.68 | 66.33 | 36.71 | 48.92 | — | — | — | — | — | — | — | — | — | |
| LoSAPer-Query Budget=512, Backbone=Trado-8B-Instruct2026.04 | 42.2 | 18.39 | 13.3 | — | 44.19 | — | — | — | — | — | — | 84.97 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 50.14 | |
| LoSAPer-Query Budget=256, Backbone=Trado-8B-Instruct2026.04 | 42.14 | 19.42 | 17.11 | — | 44.53 | — | — | — | — | — | — | 81.82 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 47.81 | |
| LoSAPer-Query Budget=128, Backbone=Trado-8B-Instruct2026.04 | 41.97 | 17.19 | 15.58 | — | 48.27 | — | — | — | — | — | — | 83.79 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 45 | |
| AdaModel=Mistral-7B-v0.3, Eviction Metric=SnapKV Metric (π1), Compression ratio=80%2026.02 | 41.89 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 26.21 | 31.88 | 24.42 | 66.52 | 62.05 | 49.5 | — | — | — | — | — | — | — | — | — | |
| StreamingLLMBackbone=Llama-3.1-8B, Budget=5122026.01 | 41.75 | 25.64 | 27.48 | 33.3 | 47.36 | 40.06 | 24.8 | 23.16 | 20.8 | 22.85 | 57.5 | 87.6 | 42.08 | 6.5 | 97 | 60.51 | 51.28 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AdaModel=Qwen2.5-32B, Eviction Metric=SnapKV Metric (π1), Compression ratio=80%2026.02 | 41.58 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25.75 | 43.6 | 23.34 | 65.7 | 44.97 | 50.13 | — | — | — | — | — | — | — | — | — | |
| UniformModel=Qwen2.5-32B, Eviction Metric=SnapKV Metric (π1), Compression ratio=80%2026.02 | 41.21 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25.02 | 44.5 | 23.33 | 64.45 | 44.99 | 48.75 | — | — | — | — | — | — | — | — | — | |
| StreamingLLMModel=Ministral-8B-Instruct, Prefill=100%, KV=10%, Execution strategy=Decoding-Only2025.02 | 41.21 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 27.74 | 35.87 | 21 | 65.44 | 62.7 | 34.5 | — | — | — | — | — | — | — | — | — | |
| PyramidModel=Mistral-7B-v0.3, Eviction Metric=SnapKV Metric (π1), Compression ratio=80%2026.02 | 40.94 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25.36 | 31.95 | 23.98 | 63.35 | 61.8 | 48.75 | — | — | — | — | — | — | — | — | — | |
| UniformModel=Mistral-7B-v0.3, Eviction Metric=SnapKV Metric (π1), Compression ratio=80%2026.02 | 40.67 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.7 | 31.78 | 24.15 | 63.95 | 60.98 | 47.5 | — | — | — | — | — | — | — | — | — | |
| AdaModel=Mistral-7B-v0.3, Eviction Metric=KeyDiff Metric (π2), Compression ratio=80%2026.02 | 40.54 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 31.94 | 33.78 | 25.23 | 59.6 | 57.71 | 40.79 | — | — | — | — | — | — | — | — | — | |
| Dense FlashAttention-2Sparsity=All2025.12 | 40.24 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 1 | — | — | — | — | — | — | — | — | |
| BeaconBackbone=Llama-2-7B, Method category=Compression-based Methods, Latency=8.0, Speedup=3.2x2025.12 | 40.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 34.9 | 27.5 | 25 | 61.4 | 57.8 | — | — | — | — | — | — | — | — | — | — | |
| BSFASparsity=k=1282025.12 | 40.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 1.08 | — | — | — | — | — | — | — | — | |
| BSFASparsity=k=962025.12 | 39.88 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 1.1 | — | — | — | — | — | — | — | — | |
| Original LLM-FTBackbone=Llama-2-7B, Latency=25.62025.12 | 39.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 34.8 | 27.5 | 23.2 | 61.8 | 57.8 | — | — | — | — | — | — | — | — | — | — | |
| H2OModel=LLaMA-3.1-8B-Instruct, Prefill=100%, KV=20%, Execution strategy=Decoding-Only2025.02 | 39.79 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 30.66 | 19.69 | 26.05 | 68 | 58.05 | 36.28 | — | — | — | — | — | — | — | — | — | |
| BSFASparsity=k=642025.12 | 39.78 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 1.13 | — | — | — | — | — | — | — | — | |
| BSFASparsity=k=322025.12 | 39.39 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 1.16 | — | — | — | — | — | — | — | — | |
| PyramidInferModel=LLaMA-3.1-8B-Instruct, Prefill=60%, KV=60%, Execution strategy=Prefill-Aware2025.02 | 39.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 28.41 | 19.31 | 23.5 | 66.4 | 61.41 | 36.87 | — | — | — | — | — | — | — | — | — | |
| PyramidModel=Qwen2.5-32B, Eviction Metric=SnapKV Metric (π1), Compression ratio=80%2026.02 | 38.68 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 19.44 | 40.23 | 21.84 | 60.24 | 46.34 | 50.46 | — | — | — | — | — | — | — | — | — | |
| GemFilterModel=LLaMA-3.1-8B-Instruct, Prefill=51%, KV=10%, Execution strategy=Prefill-Aware2025.02 | 38.61 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 28.39 | 40.33 | 21.78 | 64.31 | 30.75 | 46.13 | — | — | — | — | — | — | — | — | — | |
| UniformModel=Qwen2.5-32B, Eviction Metric=KeyDiff Metric (π2), Compression ratio=80%2026.02 | 38.33 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 26.85 | 44.3 | 22.32 | 65.82 | 29.32 | 38.36 | — | — | — | — | — | — | — | — | — | |
| H2OModel=LLaMA-3.1-8B-Instruct, Prefill=100%, KV=10%, Execution strategy=Decoding-Only2025.02 | 38.15 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 27.12 | 19.74 | 25.96 | 66.65 | 54.88 | 34.58 | — | — | — | — | — | — | — | — | — | |
| QUESTPer-Query Budget=1024, Backbone=Trado-8B-Instruct2026.04 | 37.69 | 7.37 | 17.35 | — | 39.88 | — | — | — | — | — | — | 78.84 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 45 | |
| StreamingLLMBackbone=Qwen2.5-7B, Budget=10242026.01 | 37.38 | 22.72 | 29.42 | 31.47 | 43.57 | 38.18 | 17.99 | 24.33 | 19.47 | 22.46 | 61 | 87.53 | 43.79 | 8.5 | 34 | 55.17 | 58.43 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Original LLMBackbone=Llama-2-7B, Latency=6.4, Speedup=4.0x2025.12 | 37.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.7 | 22.4 | 24.6 | 63.2 | 57.7 | — | — | — | — | — | — | — | — | — | — | |
| StreamingLLMBackbone=Qwen2.5-7B, Budget=20482026.01 | 37.07 | 23.18 | 36.93 | 45.64 | 45.3 | 40.1 | 19.74 | 28.49 | 20.57 | 23.5 | 68 | 74.41 | 33.06 | 8 | 18 | 54.5 | 53.73 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| UniformModel=Mistral-7B-v0.3, Eviction Metric=KeyDiff Metric (π2), Compression ratio=80%2026.02 | 36.83 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 28.53 | 30.86 | 24.71 | 59 | 46.89 | 33.08 | — | — | — | — | — | — | — | — | — | |
| QUESTPer-Query Budget=512, Backbone=Trado-8B-Instruct2026.04 | 36.7 | 8.99 | 16.57 | — | 34.58 | — | — | — | — | — | — | 79.33 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 44.04 | |
| PyramidModel=Mistral-7B-v0.3, Eviction Metric=KeyDiff Metric (π2), Compression ratio=80%2026.02 | 36.59 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 29.44 | 30.53 | 24.5 | 59.33 | 42.95 | 34.06 | — | — | — | — | — | — | — | — | — | |
| SpargeAttentionSparsity=τ=0.62025.12 | 35.87 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.98 | — | — | — | — | — | — | — | — | |
| QUESTPer-Query Budget=256, Backbone=Trado-8B-Instruct2026.04 | 34.64 | 8.5 | 14.04 | — | 32.95 | — | — | — | — | — | — | 75.23 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 42.46 |