Long-context language understanding on LongBench English subsets
25.38NarrativeQAW4A8+GlowQ
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| W4A8+GlowQQuantization Scheme=W4A8, Backbone Model=Llama-3.1-8B-Instruct, Context Length=8K, Evaluation Protocol=Official LongBench2026.03 | 25.38 | 12.61 | 25.71 | 15.37 | 9.93 | 15.3 | 34.07 | 22.67 | 27.14 | 52.06 | 38.55 | 88.49 | 43.6 | 71 | 72.73 | 36.97 | |
| W4A16+GlowQQuantization Scheme=W4A16, Backbone Model=Llama-3.1-8B-Instruct, Context Length=8K, Evaluation Protocol=Official LongBench2026.03 | 25.36 | 12.61 | 25.62 | 15.13 | 9.82 | 15.2 | 34 | 22.59 | 26.96 | 51.12 | 38.84 | 88.67 | 43.44 | 71 | 73.5 | 36.92 | |
| BaselineBackbone Model=Llama-3.1-8B-Instruct, Context Length=8K, Evaluation Protocol=Official LongBench2026.03 | 23.5 | 13.54 | 27.87 | 16.83 | 10.94 | 16.44 | 34.27 | 22.87 | 26.87 | 52.81 | 48.04 | 90.77 | 43.94 | 71 | 73.13 | 38.19 | |
| W4A4+GlowQQuantization Scheme=W4A4, Backbone Model=Llama-3.1-8B-Instruct, Context Length=8K, Evaluation Protocol=Official LongBench2026.03 | 23.45 | 12.2 | 27.41 | 15.34 | 9.21 | 16.15 | 33.87 | 22.78 | 26.39 | 48.73 | 38.83 | 88.78 | 42.43 | 70.5 | 70.52 | 36.44 |