Question Answering on TriviaQA Wikipedia dump December 2018 (test)
67.3EMEXIT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| EXITReader Model=Llama3.1-70B-Instruct, Compressor Type=Extractive2024.12 | 67.3 | 75.9 | 3.1 | |
| LongLLMLinguaReader Model=Llama3.1-70B-Instruct, Compressor Type=Extractive2024.12 | 66.7 | 75.2 | 3.9 | |
| Original DocsReader Model=Llama3.1-70B-Instruct, Compressor Type=None2024.12 | 65.1 | 73.9 | 7.7 | |
| RefinerReader Model=Llama3.1-70B-Instruct, Compressor Type=Abstractive2024.12 | 64.3 | 73 | 18.3 | |
| RECOMP-ExtrReader Model=Llama3.1-70B-Instruct, Compressor Type=Extractive2024.12 | 63.5 | 71 | 2.2 | |
| CompActReader Model=Llama3.1-70B-Instruct, Compressor Type=Abstractive2024.12 | 62.6 | 71.1 | 11.7 | |
| RECOMP-AbstReader Model=Llama3.1-70B-Instruct, Compressor Type=Abstractive2024.12 | 61.3 | 70.6 | 3.3 | |
| EXITReader Model=Llama3.1-8B-Instruct, Compressor Type=Extractive2024.12 | 60.8 | 69.9 | 0.7 | |
| LongLLMLinguaReader Model=Llama3.1-8B-Instruct, Compressor Type=Extractive2024.12 | 59.4 | 68 | 0.8 | |
| RefinerReader Model=Llama3.1-8B-Instruct, Compressor Type=Abstractive2024.12 | 59.2 | 68.9 | 10.9 | |
| Original DocsReader Model=Llama3.1-8B-Instruct, Compressor Type=None2024.12 | 58.8 | 68.6 | 0.9 | |
| CompActReader Model=Llama3.1-8B-Instruct, Compressor Type=Abstractive2024.12 | 58.1 | 67.7 | 8.8 | |
| RECOMP-ExtrReader Model=Llama3.1-8B-Instruct, Compressor Type=Extractive2024.12 | 56.5 | 65.1 | 0.4 | |
| RECOMP-AbstReader Model=Llama3.1-8B-Instruct, Compressor Type=Abstractive2024.12 | 55.9 | 65.7 | 1.4 |