Question Answering on 2WikiMultiHopQA December 2018 Wikipedia dump (test)
28.6EMEXIT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| EXITReader Model=Llama3.1-70B-Instruct, Compressor Type=Extractive2024.12 | 28.6 | 34.5 | 3.5 | |
| LongLLMLinguaReader Model=Llama3.1-70B-Instruct, Compressor Type=Extractive2024.12 | 28.3 | 34.8 | 4.3 | |
| EXITReader Model=Llama3.1-8B-Instruct, Compressor Type=Extractive2024.12 | 24.2 | 30.8 | 0.9 | |
| RECOMP-AbstReader Model=Llama3.1-70B-Instruct, Compressor Type=Abstractive2024.12 | 24.2 | 30.3 | 4.2 | |
| RECOMP-AbstReader Model=Llama3.1-8B-Instruct, Compressor Type=Abstractive2024.12 | 22.7 | 29.1 | 2.1 | |
| LongLLMLinguaReader Model=Llama3.1-8B-Instruct, Compressor Type=Extractive2024.12 | 21.5 | 27.4 | 0.9 | |
| RefinerReader Model=Llama3.1-70B-Instruct, Compressor Type=Abstractive2024.12 | 21.2 | 28 | 11.2 | |
| Original DocsReader Model=Llama3.1-70B-Instruct, Compressor Type=None2024.12 | 20.8 | 28.3 | 9.1 | |
| CompActReader Model=Llama3.1-70B-Instruct, Compressor Type=Abstractive2024.12 | 20.5 | 27.4 | 11.6 | |
| CompActReader Model=Llama3.1-8B-Instruct, Compressor Type=Abstractive2024.12 | 16.8 | 26 | 8.1 | |
| RefinerReader Model=Llama3.1-8B-Instruct, Compressor Type=Abstractive2024.12 | 16.8 | 25.4 | 6.4 | |
| Original DocsReader Model=Llama3.1-8B-Instruct, Compressor Type=None2024.12 | 16.1 | 24.9 | 1.1 | |
| RECOMP-ExtrReader Model=Llama3.1-70B-Instruct, Compressor Type=Extractive2024.12 | 13.8 | 19.3 | 3.3 | |
| RECOMP-ExtrReader Model=Llama3.1-8B-Instruct, Compressor Type=Extractive2024.12 | 11.2 | 19.6 | 0.6 |