Long-context Question Answering on Qasper 128K context
39F1 ScoreFP16
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FP16Model=Qwen2.5-32B2026.03 | 39 | 2,350 | |
| TTKVModel=Qwen2.5-32B2026.03 | 38.9 | 570 | |
| FP16Model=Llama-3.1-70B2026.03 | 38 | 2,450 | |
| DiffKVModel=Qwen2.5-32B2026.03 | 38 | 880 | |
| ShadowKVModel=Qwen2.5-32B2026.03 | 37.9 | 860 | |
| TTKVModel=Llama-3.1-70B2026.03 | 37.8 | 610 | |
| KVQuantModel=Qwen2.5-32B2026.03 | 37.8 | 960 | |
| FP16Model=DeepSeek-R1-14B2026.03 | 37.5 | 2,550 | |
| TTKVModel=DeepSeek-R1-14B2026.03 | 37.3 | 480 | |
| DiffKVModel=Llama-3.1-70B2026.03 | 37 | 880 | |
| KVQuantModel=Llama-3.1-70B2026.03 | 36.8 | 970 | |
| ShadowKVModel=Llama-3.1-70B2026.03 | 36.8 | 850 | |
| KIVIModel=Qwen2.5-32B2026.03 | 36.5 | 1,020 | |
| DiffKVModel=DeepSeek-R1-14B2026.03 | 36.5 | 910 | |
| ShadowKVModel=DeepSeek-R1-14B2026.03 | 36.3 | 880 | |
| KVQuantModel=DeepSeek-R1-14B2026.03 | 36.2 | 980 | |
| KIVIModel=Llama-3.1-70B2026.03 | 35.8 | 1,020 | |
| KIVIModel=DeepSeek-R1-14B2026.03 | 35 | 1,050 |