Long-context understanding on L-Eval 32K
215P95 Latency (ms)TTKV
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TTKVModel=Mistral-7B2026.03 | 215 | 0.95 | 59.1 | |
| TTKVModel=Llama-3.1-8B2026.03 | 245 | 1.05 | 65 | |
| KIVIModel=Mistral-7B, Quantization=2-bit2026.03 | 325 | 2.75 | 58.8 | |
| KVQuantModel=Mistral-7B, Quantization=3-bit2026.03 | 332 | 2.95 | 59 | |
| DiffKVModel=Mistral-7B2026.03 | 338 | 3.05 | 58.8 | |
| FP16Model=Mistral-7B2026.03 | 340 | 3.2 | 59.2 | |
| ShadowKVModel=Mistral-7B2026.03 | 345 | 3.15 | 58.9 | |
| KIVIModel=Llama-3.1-8B, Quantization=2-bit2026.03 | 362 | 3.1 | 64.9 | |
| KVQuantModel=Llama-3.1-8B, Quantization=3-bit2026.03 | 370 | 3.35 | 65.2 | |
| DiffKVModel=Llama-3.1-8B2026.03 | 378 | 3.45 | 65.1 | |
| FP16Model=Llama-3.1-8B2026.03 | 380 | 3.6 | 65.4 | |
| ShadowKVModel=Llama-3.1-8B2026.03 | 385 | 3.55 | 65.3 |