Long-context language understanding on LongBench (Custom Metric Subset)
33.15LCCFull
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FullModel=LLaMA-3.1-8B, Compression=None2026.07 | 33.15 | 44.26 | 92.29 | 24.23 | 32.51 | 15.9 | 40.82 | 31.37 | 28.02 | 26.77 | 24.22 | 8.57 | 33.51 | |
| MosaicKVModel=LLaMA-3.1-8B, Compression=MosaicKV, Block Size=8, Tokens Selected=10%, Channel Retention=25%2026.07 | 32.87 | 43.71 | 90.16 | 24.62 | 37.78 | 18.96 | 35.6 | 30.87 | 24.34 | 25.56 | 23.66 | 6 | 32.84 | |
| QuestModel=LLaMA-3.1-8B, Compression=Sequence-only, Block Size=16, Tokens Selected=10%2026.07 | 32.37 | 44.25 | 92.28 | 23.37 | 34.63 | 16.44 | 40.65 | 30.66 | 20.94 | 25.21 | 23.63 | 7.06 | 32.62 | |
| QuestModel=Ministral-3-8B, Compression=Sequence-only, Block Size=16, Tokens Selected=10%2026.07 | 26.43 | 25.54 | 89.87 | 12.77 | 10.83 | 8.23 | 35.26 | 13.76 | 16.03 | 12.18 | 24.12 | 4 | 23.25 | |
| FullModel=Ministral-3-8B, Compression=None2026.07 | 25.9 | 25.89 | 89.87 | 11.82 | 10.61 | 8.32 | 35.02 | 13.53 | 15.23 | 12.87 | 23.71 | 4.5 | 23.11 | |
| MosaicKVModel=Ministral-3-8B, Compression=MosaicKV, Block Size=8, Tokens Selected=10%, Channel Retention=25%2026.07 | 25.8 | 24.96 | 89.53 | 11.51 | 9.91 | 7.38 | 34.31 | 14.17 | 20.3 | 17.55 | 23.26 | 2.5 | 23.43 | |
| FullModel=Qwen3-4B, Compression=None2026.07 | 5.25 | 46.96 | 88.01 | 11.38 | 12.44 | 7.14 | 28.19 | 9.08 | 18.87 | 21.91 | 18.5 | 3.6 | 22.61 | |
| MosaicKVModel=Qwen3-4B, Compression=MosaicKV, Block Size=8, Tokens Selected=10%, Channel Retention=25%2026.07 | 5.1 | 45.48 | 88.58 | — | 11.65 | 6.69 | 28.46 | 10.89 | 14.23 | 21.29 | 18.67 | 3.37 | 22.17 | |
| QuestModel=Qwen3-4B, Compression=Sequence-only, Block Size=16, Tokens Selected=10%2026.07 | 5.07 | 45.75 | 88.44 | 12.08 | 12.17 | 6.74 | 29.32 | 11.47 | 20.51 | 21.44 | 18.68 | 3.56 | 22.94 |