Question Answering on Babilong 16k context length
58QA1 AccuracyFull Attention
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Full AttentionBase Model=Llama-3.1-8b-Instruct, Context Length=16k2026.02 | 58 | 19 | 33 | 90 | 68 | |
| Exact+MP(0.75)Base Model=Llama-3.1-8b-Instruct, Context Length=16k, LASER-KV Policy=Hybrid Exact-LSH + MagicPIG, Exact weight=0.752026.02 | 54 | 18 | 31 | 91 | 59 | |
| Exact+MP(0.25)Base Model=Llama-3.1-8b-Instruct, Context Length=16k, LASER-KV Policy=Hybrid Exact-LSH + MagicPIG, Exact weight=0.252026.02 | 53 | 15 | 31 | 88 | 58 | |
| Exact+MP(0.5)Base Model=Llama-3.1-8b-Instruct, Context Length=16k, LASER-KV Policy=Hybrid Exact-LSH + MagicPIG, Exact weight=0.52026.02 | 53 | 15 | 32 | 89 | 56 | |
| FINCHBase Model=Llama-3.1-8b-Instruct, Context Length=16k2026.02 | 51 | 24 | 25 | 85 | 63 | |
| ExactBase Model=Llama-3.1-8b-Instruct, Context Length=16k, LASER-KV Policy=Exact-LSH2026.02 | 49 | 20 | 31 | 89 | 57 | |
| PyramidKVBase Model=Llama-3.1-8b-Instruct, Context Length=16k2026.02 | 40 | 17 | 14 | 78 | 80 | |
| SnapKVBase Model=Llama-3.1-8b-Instruct, Context Length=16k2026.02 | 34 | 17 | 13 | 75 | 78 | |
| MagicPIGBase Model=Llama-3.1-8b-Instruct, Context Length=16k2026.02 | 12 | 2 | 11 | 35 | 28 |