Long-context language understanding on LongBench 15 English
18.26NarrativeQABaseline
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BaselineModel=Llama-3.1-8B-Instruct, Context Length=4K, Quantization=None2026.03 | 18.26 | 12.01 | 25.96 | 13.76 | 7.87 | 14.95 | 32.79 | 21.43 | 26.95 | 51.93 | 47 | 87.76 | 44.72 | 70 | 37.5 | 34.19 | |
| W4A8+GlowQModel=Llama-3.1-8B-Instruct, Context Length=4K, Weight Quantization=4-bit, Activation Quantization=8-bit, Low-rank correction=GlowQ2026.03 | 15.56 | 11.77 | 23.71 | 14.39 | 8.41 | 14.92 | 32 | 21.19 | 27.03 | 51.5 | 35.97 | 84.1 | 42.62 | 68.5 | 37.08 | 32.58 | |
| W4A16+GlowQModel=Llama-3.1-8B-Instruct, Context Length=4K, Weight Quantization=4-bit, Activation Quantization=16-bit, Low-rank correction=GlowQ2026.03 | 15.46 | 11.82 | 23.68 | 14.39 | 7.77 | 14.53 | 32.32 | 21.2 | 26.86 | 50.46 | 35.59 | 84.3 | 42.67 | 68.5 | 37.17 | 32.45 | |
| W4A4+GlowQModel=Llama-3.1-8B-Instruct, Context Length=4K, Weight Quantization=4-bit, Activation Quantization=4-bit, Low-rank correction=GlowQ2026.03 | 14.68 | 10.8 | 24.95 | 14.21 | 8.39 | 14.2 | 32.01 | 22.01 | 26.43 | 47.5 | 37.51 | 85.54 | 42.05 | 69 | 36.36 | 32.38 |