Open-domain Question Answering on Natural Questions (F1, AR)
52.2F1 ScoreUMGP-RAG
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| UMGP-RAGRetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 52.2 | 49.84 | |
| LongRAGRetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 51.45 | 49.49 | |
| UMGP-RAGRetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 51.28 | 48.78 | |
| REPLUGRetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 51.2 | 46.42 | |
| LongRAGRetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 51.13 | 49.1 | |
| MS-PoERetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 50.46 | 47.2 | |
| MS-PoERetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 50.05 | 47.09 | |
| HybridRetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 49.86 | 47.66 | |
| UMG-RAGRetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 49.74 | 47.9 | |
| REPLUGRetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 49.62 | 44.59 | |
| RAGRetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 49.41 | 45.93 | |
| RAGRetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 49.39 | 46.21 | |
| RAGRetriever=SPLADEv3, Generator=Llama-3.2-3B-Instruct2026.06 | 49.23 | 47.41 | |
| HybridRetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 49.16 | 47.41 | |
| UMG-RAGRetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 48.73 | 47.16 | |
| UMGP-RAGRetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 48.11 | 45.33 | |
| LongLLMLinguaRetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 46.31 | 44.62 | |
| LongLLMLinguaRetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 45.61 | 43.46 | |
| UMG-RAGRetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 45.14 | 43.04 | |
| HybridRetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 45.04 | 42.93 | |
| PINERetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 44.52 | 46.03 | |
| PINERetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 42.91 | 44.38 | |
| Summarized RAGRetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 35.37 | 31.62 | |
| Summarized RAGRetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 35.2 | 31.27 | |
| LongLLMLinguaRetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 25.73 | 24.29 | |
| MS-PoERetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 24.35 | 21.92 | |
| RAGRetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 24.14 | 21.78 | |
| LongRAGRetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 23.18 | 20.94 | |
| REPLUGRetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 22.54 | 18.96 | |
| PINERetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 20.84 | 20.97 | |
| Summarized RAGRetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 19.72 | 17.03 |