Question Answering on AVG. Aggregate of NQ, TQA, HQA, 2WIKI (test)
42.5EMEXIT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| EXITReader Model=Llama3.1-70B-Instruct, Compressor Type=Extractive2024.12 | 42.5 | 52 | 3.5 | |
| LongLLMLinguaReader Model=Llama3.1-70B-Instruct, Compressor Type=Extractive2024.12 | 40.3 | 49.8 | 4.1 | |
| Original DocsReader Model=Llama3.1-70B-Instruct, Compressor Type=None2024.12 | 38.8 | 48.7 | 8.4 | |
| RefinerReader Model=Llama3.1-70B-Instruct, Compressor Type=Abstractive2024.12 | 38.7 | 48.2 | 21.6 | |
| EXITReader Model=Llama3.1-8B-Instruct, Compressor Type=Extractive2024.12 | 37.9 | 47.5 | 0.8 | |
| CompActReader Model=Llama3.1-70B-Instruct, Compressor Type=Abstractive2024.12 | 37.8 | 47 | 11.5 | |
| RECOMP-AbstReader Model=Llama3.1-70B-Instruct, Compressor Type=Abstractive2024.12 | 37.5 | 47.2 | 4.1 | |
| RECOMP-ExtrReader Model=Llama3.1-70B-Instruct, Compressor Type=Extractive2024.12 | 35.2 | 43.1 | 2.7 | |
| LongLLMLinguaReader Model=Llama3.1-8B-Instruct, Compressor Type=Extractive2024.12 | 34.8 | 43.7 | 0.9 | |
| Original DocsReader Model=Llama3.1-8B-Instruct, Compressor Type=None2024.12 | 34.4 | 44.8 | 1 | |
| RefinerReader Model=Llama3.1-8B-Instruct, Compressor Type=Abstractive2024.12 | 34.4 | 44.8 | 13.1 | |
| CompActReader Model=Llama3.1-8B-Instruct, Compressor Type=Abstractive2024.12 | 34.2 | 44.5 | 8.4 | |
| RECOMP-AbstReader Model=Llama3.1-8B-Instruct, Compressor Type=Abstractive2024.12 | 34.1 | 43.7 | 1.8 | |
| RECOMP-ExtrReader Model=Llama3.1-8B-Instruct, Compressor Type=Extractive2024.12 | 31.4 | 40.5 | 0.5 |