Question Answering on Natural Questions (F1/AR Model Comparison)
50.52Qwen2.5-3B-Instruct F1UMGP-RAG
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| UMGP-RAGRetriever=BGE-M32026.06 | 50.52 | 47.94 | 51.28 | 48.78 | |
| UMGP-RAGRetriever=Qwen3-Embedding-4B2026.06 | 49.61 | 47.48 | 52.2 | 49.84 | |
| HybridRetriever=BGE-M32026.06 | 49.27 | 47.27 | 49.86 | 47.66 | |
| UMG-RAGRetriever=BGE-M32026.06 | 48.09 | 45.93 | 48.73 | 47.16 | |
| UMG-RAGRetriever=Qwen3-Embedding-4B2026.06 | 48.07 | 45.82 | 49.74 | 47.9 | |
| HybridRetriever=Qwen3-Embedding-4B2026.06 | 48.03 | 46.21 | 49.16 | 47.41 | |
| REPLUGRetriever=Qwen3-Embedding-4B2026.06 | 47.96 | 43.67 | 51.2 | 46.42 | |
| RAGRetriever=SPLADEv32026.06 | 47.45 | 45.29 | 49.23 | 47.41 | |
| LongRAGRetriever=Qwen3-Embedding-4B2026.06 | 47.44 | 43.29 | 51.45 | 49.49 | |
| REPLUGRetriever=BGE-M32026.06 | 47.01 | 42.16 | 49.62 | 44.59 | |
| PINERetriever=Qwen3-Embedding-4B2026.06 | 46.28 | 44.66 | 42.91 | 44.38 | |
| LongRAGRetriever=BGE-M32026.06 | 45.98 | 42.19 | 51.13 | 49.1 | |
| LongLLMLinguaRetriever=BGE-M32026.06 | 45.81 | 41.84 | 46.31 | 44.62 | |
| UMGP-RAGRetriever=BERT2026.06 | 45.72 | 43.07 | 48.11 | 45.33 | |
| PINERetriever=BGE-M32026.06 | 45.47 | 44.2 | 44.52 | 46.03 | |
| MS-PoERetriever=Qwen3-Embedding-4B2026.06 | 44.81 | 40.64 | 50.46 | 47.2 | |
| LongLLMLinguaRetriever=Qwen3-Embedding-4B2026.06 | 44.59 | 40.57 | 45.61 | 43.46 | |
| MS-PoERetriever=BGE-M32026.06 | 44.32 | 40.15 | 50.05 | 47.09 | |
| RAGRetriever=Qwen3-Embedding-4B2026.06 | 44.27 | 39.9 | 49.39 | 46.21 | |
| RAGRetriever=BGE-M32026.06 | 44.07 | 39.94 | 49.41 | 45.93 | |
| UMG-RAGRetriever=BERT2026.06 | 43.5 | 41.1 | 45.14 | 43.04 | |
| HybridRetriever=BERT2026.06 | 42.65 | 40.18 | 45.04 | 42.93 | |
| Summarized RAGRetriever=Qwen3-Embedding-4B2026.06 | 33.33 | 29.15 | 35.37 | 31.62 | |
| Summarized RAGRetriever=BGE-M32026.06 | 32.43 | 28.8 | 35.2 | 31.27 | |
| LongLLMLinguaRetriever=BERT2026.06 | 23.04 | 20.8 | 25.73 | 24.29 | |
| RAGRetriever=BERT2026.06 | 22.7 | 20.13 | 24.14 | 21.78 | |
| MS-PoERetriever=BERT2026.06 | 22.39 | 19.74 | 24.35 | 21.92 | |
| LongRAGRetriever=BERT2026.06 | 20.93 | 18.51 | 23.18 | 20.94 | |
| PINERetriever=BERT2026.06 | 19.15 | 18.43 | 20.84 | 20.97 | |
| Summarized RAGRetriever=BERT2026.06 | 17.04 | 14.91 | 19.72 | 17.03 | |
| REPLUGRetriever=BERT2026.06 | 13.73 | 11.74 | 22.54 | 18.96 |