Question Answering on 2WIKI (out-of-domain)
40EMOriginal Docs
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Original DocsType=N/A, Retrieval depth=Top-20, Reader model=GPT-4o2024.12 | 40 | 43.8 | 3,066.2 | 100 | |
| EXITType=Ext., Retrieval depth=Top-20, Reader model=GPT-4o2024.12 | 35.6 | 40.3 | 346.7 | 11.3 | |
| LongLLMLinguaType=Ext., Retrieval depth=Top-20, Reader model=GPT-4o2024.12 | 35.4 | 39.6 | 581.5 | 19 | |
| RECOMP-AbstType=Abs., Retrieval depth=Top-20, Reader model=GPT-4o2024.12 | 33.6 | 35.5 | 48.9 | 1.6 | |
| EXITType=Ext., Retrieval depth=Top-5, Reader model=GPT-4o2024.12 | 31.8 | 35.8 | 145.6 | 19 | |
| Original DocsType=N/A, Retrieval depth=Top-5, Reader model=GPT-4o2024.12 | 31.2 | 35.3 | 764.5 | 100 | |
| RefinerType=Abs., Retrieval depth=Top-20, Reader model=GPT-4o2024.12 | 31 | 35.5 | 100.3 | 3.3 | |
| LongLLMLinguaType=Ext., Retrieval depth=Top-5, Reader model=GPT-4o2024.12 | 30.6 | 34.5 | 230.5 | 30.2 | |
| CompActType=Abs., Retrieval depth=Top-5, Reader model=GPT-4o2024.12 | 30 | 33.6 | 67.6 | 8.8 | |
| RefinerType=Abs., Retrieval depth=Top-5, Reader model=GPT-4o2024.12 | 27.8 | 32.6 | 54.2 | 7.1 | |
| RECOMP-ExtrType=Ext., Retrieval depth=Top-5, Reader model=GPT-4o2024.12 | 25.6 | 28.6 | 93.6 | 12.2 | |
| RECOMP-AbstType=Abs., Retrieval depth=Top-5, Reader model=GPT-4o2024.12 | 23.8 | 27.8 | 53.5 | 7 | |
| RECOMP-ExtrType=Ext., Retrieval depth=Top-20, Reader model=GPT-4o2024.12 | 23.2 | 27.2 | 91 | 3 | |
| CompActType=Abs., Retrieval depth=Top-20, Reader model=GPT-4o2024.12 | 23 | 27.3 | 105.1 | 3.4 |