Long-context retrieval on Needle-in-a-Haystack (test)
100AccuracyFullKV
Evaluation results
| Method | Links | |
|---|---|---|
| FullKVLLM=Llama3-8B-Instruct, KV Cache Size=Full2026.03 | 100 | |
| YOCO2026.04 | 100 | |
| YOCO-U2026.04 | 100 | |
| DapQLLM=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 99.46 | |
| RINS2026.04 | 99 | |
| FullKVLLM=Llama3.1-8B-Instruct, KV Cache Size=Full2026.03 | 98.02 | |
| FullKVLLM=Qwen3-8B, KV Cache Size=Full2026.03 | 96.52 | |
| DapQLLM=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 95.75 | |
| FullKVLLM=Qwen2.5-7B-Instruct, KV Cache Size=Full2026.03 | 94.23 | |
| PyramidKVLLM=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 93.94 | |
| DapQLLM=Qwen3-8B, KV Cache Size=2562026.03 | 91.73 | |
| SnapKVLLM=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 90.97 | |
| SnapKVLLM=Qwen3-8B, KV Cache Size=2562026.03 | 90.41 | |
| PyramidKVLLM=Qwen3-8B, KV Cache Size=2562026.03 | 88.5 | |
| Transformer2026.04 | 87 | |
| DapQLLM=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 85.11 | |
| DapQLLM=Llama3.1-8B-Instruct, KV Cache Size=2562026.03 | 84.7 | |
| SnapKVLLM=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 84.3 | |
| PyramidKVLLM=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 83.8 | |
| PyramidKVLLM=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 79.67 | |
| PyramidKVLLM=Llama3.1-8B-Instruct, KV Cache Size=2562026.03 | 78.3 | |
| DapQLLM=Qwen3-8B, KV Cache Size=1282026.03 | 77.89 | |
| DapQLLM=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 76.25 | |
| DapQLLM=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 75.75 | |
| H2OLLM=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 75.64 | |
| SnapKVLLM=Qwen3-8B, KV Cache Size=1282026.03 | 75.39 | |
| SnapKVLLM=Llama3.1-8B-Instruct, KV Cache Size=2562026.03 | 74.84 | |
| PyramidKVLLM=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 74.8 | |
| SnapKVLLM=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 74.67 | |
| H2OLLM=Qwen3-8B, KV Cache Size=2562026.03 | 74.55 | |
| SnapKVLLM=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 73.64 | |
| SnapKVLLM=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 72.84 | |
| PyramidKVLLM=Qwen3-8B, KV Cache Size=1282026.03 | 72.36 | |
| H2OLLM=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 70.45 | |
| DapQLLM=Llama3.1-8B-Instruct, KV Cache Size=1282026.03 | 70.34 | |
| DapQLLM=Llama3-8B-Instruct, KV Cache Size=642026.03 | 68.34 | |
| H2OLLM=Qwen3-8B, KV Cache Size=1282026.03 | 67.5 | |
| H2OLLM=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 66.81 | |
| PyramidKVLLM=Llama3.1-8B-Instruct, KV Cache Size=1282026.03 | 65.23 | |
| H2OLLM=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 63.7 | |
| H2OLLM=Qwen3-8B, KV Cache Size=642026.03 | 62.7 | |
| DapQLLM=Llama3.1-8B-Instruct, KV Cache Size=642026.03 | 62.2 | |
| DapQLLM=Qwen3-8B, KV Cache Size=642026.03 | 61.98 | |
| SnapKVLLM=Llama3-8B-Instruct, KV Cache Size=642026.03 | 61.7 | |
| SnapKVLLM=Llama3.1-8B-Instruct, KV Cache Size=1282026.03 | 61.45 | |
| PyramidKVLLM=Qwen3-8B, KV Cache Size=642026.03 | 61.32 | |
| H2OLLM=Llama3.1-8B-Instruct, KV Cache Size=2562026.03 | 61.18 | |
| SnapKVLLM=Qwen3-8B, KV Cache Size=642026.03 | 59.73 | |
| SnapKVLLM=Llama3.1-8B-Instruct, KV Cache Size=642026.03 | 56.5 | |
| PyramidKVLLM=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 56.11 | |
| PyramidKVLLM=Llama3-8B-Instruct, KV Cache Size=642026.03 | 55.45 | |
| PyramidKVLLM=Llama3.1-8B-Instruct, KV Cache Size=642026.03 | 52.25 | |
| H2OLLM=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 50.92 | |
| H2OLLM=Llama3.1-8B-Instruct, KV Cache Size=1282026.03 | 47.61 | |
| H2OLLM=Llama3-8B-Instruct, KV Cache Size=642026.03 | 42.37 | |
| H2OLLM=Llama3.1-8B-Instruct, KV Cache Size=642026.03 | 40.36 |