Multi-Doc Question Answering on LongBench
51.6HotpotQAGPT-3.5-Turbo-16k
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-3.5-Turbo-16kContext Window=16k2024.01 | 51.6 | 37.7 | 26.9 | 28.7 | |
| E2-LLM-Llama2-13B-32kBackbone=Llama2-13B, Context Window=32k2024.01 | 49.2 | 37.6 | 23.1 | 20.4 | |
| E2-LLM-Llama2-13B-16kBackbone=Llama2-13B, Context Window=16k2024.01 | 46.4 | 38.3 | 25.2 | 19.3 | |
| PI-Llama2-13B-16kContext Window=16k2024.01 | 44.9 | 34.8 | 19.5 | 17.4 | |
| E2-LLM-Llama2-7B-32kBackbone=Llama2-7B, Context Window=32k2024.01 | 43.7 | 34.8 | 22 | 22.6 | |
| Vicuna-v1.5-13B-16kContext Window=16k2024.01 | 38.1 | 36 | 10.7 | 20.9 | |
| E2-LLM-Llama2-7B-16kBackbone=Llama2-7B, Context Window=16k2024.01 | 37.1 | 34.4 | 17.9 | 18.6 | |
| LongLora-7B-16kContext Window=16k2024.01 | 37 | 30.3 | 17.1 | 15.3 | |
| Llama2-13B-chat-4kContext Window=4k2024.01 | 36.1 | 32.4 | 14.5 | 26.8 | |
| LongChat-v1.5-7B-32kContext Window=32k2024.01 | 31.5 | 20.6 | 9.7 | 19.5 | |
| Llama2-7B-chat-4kContext Window=4k2024.01 | 25.4 | 32.8 | 9.4 | 5.2 | |
| Vicuna-v1.5-7B-16kContext Window=16k2024.01 | 25.3 | 20.8 | 9.8 | 19.3 |