Multi-hop Question Answering on MuSiQue 2022
51.2F1 ScoreLongLLMLingua
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LongLLMLinguaTarget LLM=GPT-3.5-Turbo, Compression constraint=2x2023.10 | 51.2 | 1,077 | 2.3 | |
| LongLLMLingua (w/o Subsequence Recovery)Target LLM=GPT-3.5-Turbo, Compression constraint=2x2023.10 | 50.7 | 1,077 | 2.3 | |
| LongLLMLingua (w/o restrict)Target LLM=GPT-3.5-Turbo, Compression constraint=2x, Module=Question-aware Coarse-grained2023.10 | 49.2 | 1,078 | 2.3 | |
| LongLLMLingua (w/o Document Reordering)Target LLM=GPT-3.5-Turbo, Compression constraint=2x2023.10 | 49.2 | 1,077 | 2.3 | |
| LongLLMLingua (w/o Question-aware Fine-grained)Target LLM=GPT-3.5-Turbo, Compression constraint=2x2023.10 | 48.4 | 1,118 | 2.2 | |
| LongLLMLingua (w/ GPT2-small)Target LLM=GPT-3.5-Turbo, Compression constraint=2x, Model=GPT2-small2023.10 | 48.4 | 1,095 | 2.2 | |
| LongLLMLingua (w/o Dynamic Compression Ratio)Target LLM=GPT-3.5-Turbo, Compression constraint=2x2023.10 | 48.2 | 1,090 | 2.2 | |
| LongLLMLingua (w/ SBERT)Target LLM=GPT-3.5-Turbo, Compression constraint=2x, Module=Question-aware Coarse-grained2023.10 | 47.3 | 1,070 | 2.3 | |
| LongLLMLingua r_kTarget LLM=GPT-3.5-Turbo, Compression constraint=2x2023.10 | 46.3 | 1,295 | 1.9 | |
| Original PromptTarget LLM=GPT-3.5-Turbo2023.10 | 45.8 | 2,427 | — | |
| LongLLMLingua (w/ p(x_doc|x_que, restrict))Target LLM=GPT-3.5-Turbo, Compression constraint=2x, Module=Question-aware Coarse-grained2023.10 | 44 | 1,066 | 2.3 | |
| LongLLMLingua (w/o Question-awareness)Target LLM=GPT-3.5-Turbo, Compression constraint=2x, Module=Question-aware Coarse-grained2023.10 | 43.2 | 1,076 | 2.3 | |
| LLMLinguaTarget LLM=GPT-3.5-Turbo, Compression constraint=2x2023.10 | 40.1 | 1,110 | 2.2 | |
| SBERTTarget LLM=GPT-3.5-Turbo, Compression constraint=2x2023.10 | 36.2 | 1,288 | 1.9 | |
| BM25Target LLM=GPT-3.5-Turbo, Compression constraint=2x2023.10 | 28.5 | 1,295 | 1.9 | |
| Selective-ContextTarget LLM=GPT-3.5-Turbo, Compression constraint=2x2023.10 | 19.6 | 1,141 | 2.1 |