Long-context understanding on LongBench (Average Score)
47.26Average ScoreLKV
Evaluation Results
| Method | Links | |
|---|---|---|
| LKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.42026.04 | 47.26 | |
| LKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.32026.04 | 47.16 | |
| LKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.22026.04 | 47.11 | |
| LKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.52026.04 | 46.99 | |
| LKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.152026.04 | 46.73 | |
| DuoAttentionBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.52026.04 | 46.62 | |
| Ada-SnapKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.52026.04 | 46.04 | |
| SnapKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.52026.04 | 45.49 | |
| DuoAttentionBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.42026.04 | 45.28 | |
| DuoAttentionBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.32026.04 | 45.21 | |
| Ada-SnapKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.42026.04 | 45.02 | |
| SnapKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.42026.04 | 44.33 | |
| Ada-SnapKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.32026.04 | 44.01 | |
| LKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.12026.04 | 43.93 | |
| SnapKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.32026.04 | 43.16 | |
| Ada-SnapKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.22026.04 | 41.32 | |
| SnapKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.22026.04 | 40.22 | |
| Ada-SnapKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.152026.04 | 39.06 | |
| SnapKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.152026.04 | 38.05 | |
| PyramidKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.52026.04 | 38.01 | |
| PyramidKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.42026.04 | 37.55 | |
| DuoAttentionBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.22026.04 | 37.53 | |
| Ada-SnapKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.12026.04 | 36.28 | |
| PyramidKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.32026.04 | 35.59 | |
| SnapKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.12026.04 | 34.62 | |
| PyramidKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.22026.04 | 33.41 | |
| PyramidKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.152026.04 | 32.12 | |
| PyramidKVBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.12026.04 | 31.97 | |
| DuoAttentionBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.152026.04 | 31.03 | |
| DuoAttentionBase Model=Llama-3.1-8B-Instruct, Retention Ratio (R)=0.12026.04 | 29.72 |