Long-context language understanding on LongBench (Qasper, HotpotQA, LCC, and 9 other metrics subset)
30.19QasperStandard
Evaluation Results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| StandardBase Model=Qwen1.5-MoE-14.3B-A2.7B2024.10 | 30.19 | 38.47 | 10.17 | 11.92 | 24.77 | 68 | 86.35 | 38.09 | 1.67 | 48.33 | 40.89 | 36.26 | |
| LightTransferBase Model=Qwen1.5-MoE-14.3B-A2.7B2024.10 | 26.92 | 37.23 | 9.12 | 12.78 | 24.63 | 65.5 | 85.35 | 37.74 | 3.1 | 48.68 | 38.33 | 35.43 | |
| SqueezeAttnBase Model=Qwen1.5-MoE-14.3B-A2.7B2024.10 | 23.48 | 29.58 | 8.56 | 11.35 | 20.94 | 64.5 | 84.8 | 38.22 | 3.3 | 44.9 | 36.62 | 33.3 | |
| MiniCacheBase Model=Qwen1.5-MoE-14.3B-A2.7B2024.10 | 18.66 | 24.14 | 5.57 | 7.64 | 20.09 | 66 | 70.44 | 24.87 | 2.04 | 33.38 | 22.09 | 26.81 |