Question Answering on NaturalQuestions processed
83.05AccuracyOracle
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| OracleBackbone=LongChat-13B, Evaluation Protocol=complete ground-truth documents2024.06 | 83.05 | 59.2 | — | |
| OracleBackbone=LLAMA-2-7B, Evaluation Protocol=complete ground-truth documents2024.06 | 73.45 | 59.2 | — | |
| QGCBackbone=LongChat-13B, Method Category=Compression-based2024.06 | 69.19 | 15.2 | 0.356 | |
| LongLLMLinguaBackbone=LongChat-13B, Method Category=Compression-based2024.06 | 67.01 | 4.1 | 0.118 | |
| LongLLMLinguaBackbone=LongChat-13B, Method Category=Compression-based, Data Source=Cited from Jiang et al. (2023)2024.06 | 66.7 | 3.9 | — | |
| Cond.PPLBackbone=LongChat-13B, Method Category=Reranker-based2024.06 | 65.91 | 4.1 | 0.128 | |
| BGE-RerankerBackbone=LongChat-13B, Method Category=Reranker-based2024.06 | 64.33 | 4.1 | 0.138 | |
| QGCBackbone=LLAMA-2-7B, Method Category=Compression-based2024.06 | 60.9 | 15.2 | 0.313 | |
| Sentence-BERTBackbone=LongChat-13B, Method Category=Reranker-based2024.06 | 60.75 | 4.1 | 0.137 | |
| QGCBackbone=LLAMA-2-7B, Method Category=Compression-based, epsilon=0.422024.06 | 57.62 | 20.6 | — | |
| ICAEBackbone=LLAMA-2-7B, Method Category=Compression-based2024.06 | 53.34 | 21.5 | — | |
| Original PromptBackbone=LongChat-13B, Evaluation Protocol=all retrieved documents2024.06 | 53.11 | 1 | — | |
| AutoCompressorBackbone=LLAMA-2-7B, Method Category=Compression-based2024.06 | 49.23 | 13.9 | 0.302 | |
| LongLLMLinguaBackbone=LLAMA-2-7B, Method Category=Compression-based2024.06 | 41.13 | 4.1 | 0.108 | |
| Selective-ContextBackbone=LongChat-13B, Method Category=Compression-based2024.06 | 35.44 | 2.5 | 0.077 | |
| Closed-bookBackbone=LongChat-13B, Evaluation Protocol=query only2024.06 | 34.84 | — | — | |
| Closed-bookBackbone=LLAMA-2-7B, Evaluation Protocol=query only2024.06 | 32.35 | — | — | |
| Selective-ContextBackbone=LLAMA-2-7B, Method Category=Compression-based2024.06 | 31.79 | 2.6 | 0.082 | |
| Cond.PPLBackbone=LLAMA-2-7B, Method Category=Reranker-based2024.06 | 27.87 | 4.1 | 0.123 | |
| Original PromptBackbone=LLAMA-2-7B, Evaluation Protocol=all retrieved documents2024.06 | 27.53 | 1 | — | |
| BGE-RerankerBackbone=LLAMA-2-7B, Method Category=Reranker-based2024.06 | 25.08 | 4.1 | 0.13 | |
| Sentence-BERTBackbone=LLAMA-2-7B, Method Category=Reranker-based2024.06 | 24.26 | 4.1 | 0.133 |