Document Question Answering on HotpotQA (Accuracy)
45.55AccuracyFullKV
Evaluation results
| Method | Links | |
|---|---|---|
| FullKVBackbone=LLaMA-3-8B-Instruct, Compression Ratio=100%2025.02 | 45.55 | |
| PyramidKVBackbone=LLaMA-3-8B-Instruct, Compression Ratio=30%2025.02 | 44.56 | |
| ShotKVBackbone=LLaMA-3-8B-Instruct, Compression Ratio=30%2025.02 | 44.38 | |
| SnapKVBackbone=LLaMA-3-8B-Instruct, Compression Ratio=30%2025.02 | 44.12 | |
| ChunkKVBackbone=LLaMA-3-8B-Instruct, Compression Ratio=30%2025.02 | 44.05 | |
| PyramidKVBackbone=LLaMA-3-8B-Instruct, Compression Ratio=10%2025.02 | 43.8 | |
| ShotKVBackbone=LLaMA-3-8B-Instruct, Compression Ratio=10%2025.02 | 43.6 | |
| SnapKVBackbone=LLaMA-3-8B-Instruct, Compression Ratio=10%2025.02 | 43.36 | |
| ChunkKVBackbone=LLaMA-3-8B-Instruct, Compression Ratio=10%2025.02 | 43.27 | |
| H2OBackbone=LLaMA-3-8B-Instruct, Compression Ratio=30%2025.02 | 42.75 | |
| StreamingLLMBackbone=LLaMA-3-8B-Instruct, Compression Ratio=30%2025.02 | 42.18 | |
| H2OBackbone=LLaMA-3-8B-Instruct, Compression Ratio=10%2025.02 | 40.84 | |
| StreamingLLMBackbone=LLaMA-3-8B-Instruct, Compression Ratio=10%2025.02 | 40.27 |