Question Answering on PopQA longtail (EM, F1, Acc)
45.96EMStandardRAG
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| StandardRAGRetrieval setting=w/ retrieval, Compression ratio=none2026.01 | 45.96 | 23.1 | 33.95 | |
| RPDRGenerator=GPT-3.5, Retriever=RPDR2026.02 | 41.4 | — | — | |
| LLMLingua-2Retrieval setting=w/ retrieval, Compression ratio=x32026.01 | 40.17 | 15.66 | 31.09 | |
| RPDRGenerator=GPT-neox 20B, Retriever=RPDR2026.02 | 34.9 | — | — | |
| xRAGRetrieval setting=w/ retrieval, Compression ratio=x1282026.01 | 33.95 | 4.94 | 28.95 | |
| BM25Generator=GPT-3.5, Retriever=BM252026.02 | 30.5 | — | — | |
| ArcAlignerRetrieval setting=w/ retrieval, Compression ratio=x242026.01 | 30.38 | 32.8 | 30.74 | |
| RPDRGenerator=GPT-j-6B, Retriever=RPDR2026.02 | 30.2 | — | — | |
| RPDRGenerator=LLaMA-3 8B, Retriever=RPDR2026.02 | 29.8 | — | — | |
| ContrieverGenerator=GPT-3.5, Retriever=Contriever2026.02 | 29.7 | — | — | |
| BM25Generator=GPT-neox 20B, Retriever=BM252026.02 | 29.1 | — | — | |
| ContrieverGenerator=GPT-neox 20B, Retriever=Contriever2026.02 | 25.6 | — | — | |
| BM25Generator=GPT-j-6B, Retriever=BM252026.02 | 24.9 | — | — | |
| BM25Generator=LLaMA-3 8B, Retriever=BM252026.02 | 24.7 | — | — | |
| COCOMRetrieval setting=w/ retrieval, Compression ratio=x42026.01 | 24.59 | 25.15 | 24.37 | |
| COCOMRetrieval setting=w/ retrieval, Compression ratio=x162026.01 | 24.16 | 25.79 | 24.59 | |
| ContrieverGenerator=LLaMA-3 8B, Retriever=Contriever2026.02 | 23.7 | — | — | |
| Parametric knowledgeGenerator=GPT-3.5, Retriever=Parametric2026.02 | 22.7 | — | — | |
| NaiveRetrieval setting=w/o retrieval, Compression ratio=none2026.01 | 20.01 | 8.9 | 13.3 | |
| ContrieverGenerator=GPT-j-6B, Retriever=Contriever2026.02 | 19.2 | — | — | |
| Parametric knowledgeGenerator=GPT-neox 20B, Retriever=Parametric2026.02 | 13.5 | — | — | |
| Parametric knowledgeGenerator=LLaMA-3 8B, Retriever=Parametric2026.02 | 11.9 | — | — | |
| Parametric knowledgeGenerator=GPT-j-6B, Retriever=Parametric2026.02 | 11.3 | — | — |