Multi-Document Question Answering on LongBench
57.67HotpotQA AccFullKV
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| FullKVBackbone=Qwen3-8B, KV Cache Size=Full2026.03 | 57.67 | 44.73 | — | — | |
| FullKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=Full2026.03 | 55.91 | 44.7 | — | — | |
| DapQBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 54.79 | 44.47 | — | — | |
| SnapKVBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 54.38 | 41.4 | — | — | |
| DapQBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 53.84 | 43 | — | — | |
| DapQBackbone=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 51.92 | 41.46 | — | — | |
| SnapKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 51.76 | 40.78 | — | — | |
| H2OBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 51.37 | 40.5 | — | — | |
| SnapKVBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 51.2 | 41.66 | — | — | |
| PyramidKVBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 51 | 41.37 | — | — | |
| DapQBackbone=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 50.67 | 34.35 | — | — | |
| PyramidKVBackbone=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 50.2 | 33.88 | — | — | |
| FullKVBackbone=Llama3-8B-Instruct, KV Cache Size=Full2026.03 | 50.17 | 34.88 | — | — | |
| DapQBackbone=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 50.04 | 33.59 | — | — | |
| PyramidKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 49.89 | 40.77 | — | — | |
| DapQBackbone=Qwen3-8B, KV Cache Size=642026.03 | 49.84 | 41.28 | — | — | |
| SnapKVBackbone=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 49.75 | 33.8 | — | — | |
| DapQBackbone=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 49.66 | 41.25 | — | — | |
| DapQBackbone=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 49.62 | 41.36 | — | — | |
| PyramidKVBackbone=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 49.52 | 31.6 | — | — | |
| SnapKVBackbone=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 49.36 | 32.4 | — | — | |
| DapQBackbone=Llama3-8B-Instruct, KV Cache Size=642026.03 | 49.11 | 32.88 | — | — | |
| H2OBackbone=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 48.65 | 31.9 | — | — | |
| H2OBackbone=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 48.62 | 31.5 | — | — | |
| SnapKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 48.62 | 41.4 | — | — | |
| PyramidKVBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 48.41 | 39.46 | — | — | |
| H2OBackbone=Llama3-8B-Instruct, KV Cache Size=642026.03 | 48.27 | 31.7 | — | — | |
| H2OBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 48.1 | 39.85 | — | — | |
| H2OBackbone=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 47.89 | 40.97 | — | — | |
| SnapKVBackbone=Llama3-8B-Instruct, KV Cache Size=642026.03 | 47.16 | 31.71 | — | — | |
| PyramidKVBackbone=Llama3-8B-Instruct, KV Cache Size=642026.03 | 47.01 | 31.54 | — | — | |
| H2OBackbone=Qwen3-8B, KV Cache Size=642026.03 | 46.66 | 39.27 | — | — | |
| PyramidKVBackbone=Qwen3-8B, KV Cache Size=642026.03 | 46.61 | 39.2 | — | — | |
| SnapKVBackbone=Qwen3-8B, KV Cache Size=642026.03 | 46.58 | 39.38 | — | — | |
| PyramidKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 46.57 | 39.07 | — | — | |
| SnapKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 45.56 | 40.28 | — | — | |
| H2OBackbone=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 45.14 | 39.77 | — | — | |
| H2OBackbone=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 44.94 | 39.06 | — | — | |
| PyramidKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 43.51 | 38.08 | — | — | |
| FullKVModel=LLaMA-3-8B-Instruct, KV cache size=Full2026.02 | 43.09 | 36.15 | 21.77 | — | |
| CompilerKVModel=LLaMA-3-8B-Instruct, KV cache size=1282026.02 | 41.37 | 28.48 | 21.25 | — | |
| StreamingLLMModel=LLaMA-3-8B-Instruct, KV cache size=1282026.02 | 37.84 | 29.02 | 16.77 | — | |
| CompilerKVModel=Mistral-7B-Instruct-v0.2, KV cache size=1282026.02 | 36.42 | 22.91 | 16.47 | — | |
| CompilerKVModel=InternLM-2.5-Chat-1M, KV cache size=1282026.02 | 32.17 | 19.75 | 20.95 | — | |
| CompilerKVModel=Qwen2-7B-Instruct, KV cache size=1282026.02 | 14.85 | 16.45 | 8.4 | — | |
| baselinePrecision=16-bit, Backbone=Llama3.1-8B2025.03 | — | — | — | 39.96 | |
| KiViPrecision=2-bit, Backbone=Llama3.1-8B2025.03 | — | — | — | 34.02 | |
| KiViPrecision=4-bit, Backbone=Llama3.1-8B2025.03 | — | — | — | 39.74 | |
| LogQuantPrecision=2-bit, Backbone=Llama3.1-8B2025.03 | — | — | — | 36.08 | |
| LogQuantPrecision=4-bit, Backbone=Llama3.1-8B2025.03 | — | — | — | 39.93 |