Single-Document QA on LongBench
25.4NarrativeQAE2-LLM-Llama2-13B-16k
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| E2-LLM-Llama2-13B-16kBackbone=Llama2-13B, Context Window=16k2024.01 | 25.4 | 35.3 | 46.5 | 49.1 | — | |
| E2-LLM-Llama2-13B-32kBackbone=Llama2-13B, Context Window=32k2024.01 | 24.1 | 36.2 | 49 | 52.5 | — | |
| GPT-3.5-Turbo-16kContext Window=16k2024.01 | 23.6 | 43.3 | 52.3 | 61.2 | — | |
| LongLora-7B-16kContext Window=16k2024.01 | 19.8 | 29.1 | 37.2 | 8.5 | — | |
| Vicuna-v1.5-7B-16kContext Window=16k2024.01 | 19.4 | 26.1 | 38.5 | 43 | — | |
| Llama2-13B-chat-4kContext Window=4k2024.01 | 19.2 | 25.8 | 36.9 | 33.3 | — | |
| PI-Llama2-13B-16kContext Window=16k2024.01 | 19.2 | 33.3 | 42.7 | 47.9 | — | |
| Vicuna-v1.5-13B-16kContext Window=16k2024.01 | 18.9 | 29.9 | 46.2 | 28.4 | — | |
| Llama2-7B-chat-4kContext Window=4k2024.01 | 18.7 | 19.2 | 36.8 | 11.9 | — | |
| LongChat-v1.5-7B-32kContext Window=32k2024.01 | 16.9 | 27.7 | 41.4 | 29.1 | — | |
| E2-LLM-Llama2-7B-16kBackbone=Llama2-7B, Context Window=16k2024.01 | 16.4 | 34.7 | 39.1 | 43.6 | — | |
| E2-LLM-Llama2-7B-32kBackbone=Llama2-7B, Context Window=32k2024.01 | 12.3 | 35.6 | 40.4 | 46.6 | — | |
| baselinePrecision=16-bit, Backbone=Llama3.1-8B2025.03 | — | — | — | — | 47.71 | |
| KiViPrecision=2-bit, Backbone=Llama3.1-8B2025.03 | — | — | — | — | 38.89 | |
| KiViPrecision=4-bit, Backbone=Llama3.1-8B2025.03 | — | — | — | — | 47.75 | |
| LogQuantPrecision=2-bit, Backbone=Llama3.1-8B2025.03 | — | — | — | — | 41.91 | |
| LogQuantPrecision=4-bit, Backbone=Llama3.1-8B2025.03 | — | — | — | — | 47.73 |