Question Answering on HotPotQA (F1 and AR)
51.72F1 ScoreMS-PoE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MS-PoERetriever=Qwen3-Embedding-4B, Generator Model=Llama-3.2-3B-Instruct2026.06 | 51.72 | 42.95 | |
| MS-PoERetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 51.72 | 42.95 | |
| UMGP-RAGRetriever=BGE-M3, Generator Model=Llama-3.2-3B-Instruct2026.06 | 51.33 | 42.75 | |
| UMGP-RAGRetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 51.33 | 42.75 | |
| UMGP-RAGRetriever=Qwen3-Embedding-4B, Generator Model=Llama-3.2-3B-Instruct2026.06 | 50.68 | 42 | |
| UMGP-RAGRetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 50.68 | 42 | |
| UMGP-RAGRetriever=BGE-M3, Generator=Qwen2.5-3B-Instruct2026.06 | 50.34 | 42.16 | |
| MS-PoERetriever=BGE-M3, Generator Model=Llama-3.2-3B-Instruct2026.06 | 50.01 | 41.23 | |
| MS-PoERetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 50.01 | 41.23 | |
| RAGRetriever=Qwen3-Embedding-4B, Generator Model=Llama-3.2-3B-Instruct2026.06 | 49.91 | 41.32 | |
| RAGRetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 49.91 | 41.32 | |
| HybridRetriever=BGE-M3, Generator Model=Llama-3.2-3B-Instruct2026.06 | 49.86 | 41.75 | |
| HybridRetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 49.86 | 41.75 | |
| MS-PoERetriever=Qwen3-Embedding-4B, Generator=Qwen2.5-3B-Instruct2026.06 | 49.48 | 40.73 | |
| UMGP-RAGRetriever=Qwen3-Embedding-4B, Generator=Qwen2.5-3B-Instruct2026.06 | 49.19 | 40.71 | |
| RAGRetriever=BGE-M3, Generator Model=Llama-3.2-3B-Instruct2026.06 | 49.13 | 40.48 | |
| RAGRetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 49.13 | 40.48 | |
| RAGRetriever=Qwen3-Embedding-4B, Generator=Qwen2.5-3B-Instruct2026.06 | 48.83 | 40.41 | |
| UMG-RAGRetriever=BGE-M3, Generator Model=Llama-3.2-3B-Instruct2026.06 | 48.27 | 40.18 | |
| UMG-RAGRetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 48.27 | 40.18 | |
| LongRAGRetriever=BGE-M3, Generator Model=Llama-3.2-3B-Instruct2026.06 | 47.99 | 39.84 | |
| LongRAGRetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 47.99 | 39.84 | |
| RAGRetriever=SPLADEv3, Generator Model=Llama-3.2-3B-Instruct2026.06 | 47.98 | 39.66 | |
| RAGRetriever=SPLADEv3, Generator=Llama-3.2-3B-Instruct2026.06 | 47.98 | 39.66 | |
| HybridRetriever=BGE-M3, Generator=Qwen2.5-3B-Instruct2026.06 | 47.75 | 39.75 | |
| LongRAGRetriever=Qwen3-Embedding-4B, Generator Model=Llama-3.2-3B-Instruct2026.06 | 47.65 | 39.61 | |
| LongRAGRetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 47.65 | 39.61 | |
| UMG-RAGRetriever=Qwen3-Embedding-4B, Generator Model=Llama-3.2-3B-Instruct2026.06 | 47.52 | 39.29 | |
| UMG-RAGRetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 47.52 | 39.29 | |
| UMG-RAGRetriever=BGE-M3, Generator=Qwen2.5-3B-Instruct2026.06 | 47.49 | 39.57 | |
| HybridRetriever=Qwen3-Embedding-4B, Generator Model=Llama-3.2-3B-Instruct2026.06 | 47.09 | 39.04 | |
| HybridRetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 47.09 | 39.04 | |
| LongLLMLinguaRetriever=BGE-M3, Generator=Qwen2.5-3B-Instruct2026.06 | 47.06 | 38.07 | |
| LongLLMLinguaRetriever=BGE-M3, Generator Model=Llama-3.2-3B-Instruct2026.06 | 46.68 | 37.71 | |
| LongLLMLinguaRetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 46.68 | 37.71 | |
| LongRAGRetriever=BGE-M3, Generator=Qwen2.5-3B-Instruct2026.06 | 46.58 | 38.45 | |
| LongLLMLinguaRetriever=Qwen3-Embedding-4B, Generator Model=Llama-3.2-3B-Instruct2026.06 | 46.44 | 37.38 | |
| LongLLMLinguaRetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 46.44 | 37.38 | |
| HybridRetriever=Qwen3-Embedding-4B, Generator=Qwen2.5-3B-Instruct2026.06 | 46.26 | 38.37 | |
| MS-PoERetriever=BGE-M3, Generator=Qwen2.5-3B-Instruct2026.06 | 46.22 | 38.05 | |
| UMG-RAGRetriever=Qwen3-Embedding-4B, Generator=Qwen2.5-3B-Instruct2026.06 | 46.22 | 38.2 | |
| PINERetriever=Qwen3-Embedding-4B, Generator=Qwen2.5-3B-Instruct2026.06 | 46.21 | 39.8 | |
| RAGRetriever=SPLADEv3, Generator=Qwen2.5-3B-Instruct2026.06 | 46.13 | 38.5 | |
| RAGRetriever=BGE-M3, Generator=Qwen2.5-3B-Instruct2026.06 | 46.13 | 37.98 | |
| LongLLMLinguaRetriever=Qwen3-Embedding-4B, Generator=Qwen2.5-3B-Instruct2026.06 | 46.03 | 36.93 | |
| LongRAGRetriever=Qwen3-Embedding-4B, Generator=Qwen2.5-3B-Instruct2026.06 | 45.24 | 37.48 | |
| PINERetriever=Qwen3-Embedding-4B, Generator Model=Llama-3.2-3B-Instruct2026.06 | 44.63 | 40.27 | |
| PINERetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 44.63 | 40.27 | |
| UMGP-RAGRetriever=BERT, Generator Model=Llama-3.2-3B-Instruct2026.06 | 44.6 | 36.55 | |
| UMGP-RAGRetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 44.6 | 36.55 | |
| UMGP-RAGRetriever=BERT, Generator=Qwen2.5-3B-Instruct2026.06 | 44.53 | 36.45 | |
| PINERetriever=BGE-M3, Generator=Qwen2.5-3B-Instruct2026.06 | 44.42 | 37.87 | |
| PINERetriever=BGE-M3, Generator Model=Llama-3.2-3B-Instruct2026.06 | 44.09 | 39.41 | |
| PINERetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 44.09 | 39.41 | |
| REPLUGRetriever=Qwen3-Embedding-4B, Generator Model=Llama-3.2-3B-Instruct2026.06 | 43.43 | 34.11 | |
| REPLUGRetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 43.43 | 34.11 | |
| Summarized RAGRetriever=Qwen3-Embedding-4B, Generator Model=Llama-3.2-3B-Instruct2026.06 | 43.15 | 35.46 | |
| Summarized RAGRetriever=Qwen3-Embedding-4B, Generator=Llama-3.2-3B-Instruct2026.06 | 43.15 | 35.46 | |
| REPLUGRetriever=BGE-M3, Generator Model=Llama-3.2-3B-Instruct2026.06 | 42.87 | 33.62 | |
| REPLUGRetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 42.87 | 33.62 | |
| HybridRetriever=BERT, Generator Model=Llama-3.2-3B-Instruct2026.06 | 42.45 | 34.79 | |
| HybridRetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 42.45 | 34.79 | |
| UMG-RAGRetriever=BERT, Generator Model=Llama-3.2-3B-Instruct2026.06 | 42.23 | 34.7 | |
| UMG-RAGRetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 42.23 | 34.7 | |
| UMG-RAGRetriever=BERT, Generator=Qwen2.5-3B-Instruct2026.06 | 41.63 | 34.16 | |
| Summarized RAGRetriever=BGE-M3, Generator Model=Llama-3.2-3B-Instruct2026.06 | 41.09 | 33.66 | |
| Summarized RAGRetriever=BGE-M3, Generator=Llama-3.2-3B-Instruct2026.06 | 41.09 | 33.66 | |
| HybridRetriever=BERT, Generator=Qwen2.5-3B-Instruct2026.06 | 40.81 | 33.11 | |
| REPLUGRetriever=Qwen3-Embedding-4B, Generator=Qwen2.5-3B-Instruct2026.06 | 40.75 | 32.52 | |
| Summarized RAGRetriever=Qwen3-Embedding-4B, Generator=Qwen2.5-3B-Instruct2026.06 | 39.76 | 32.61 | |
| REPLUGRetriever=BGE-M3, Generator=Qwen2.5-3B-Instruct2026.06 | 39.41 | 31.5 | |
| Summarized RAGRetriever=BGE-M3, Generator=Qwen2.5-3B-Instruct2026.06 | 38.56 | 31.61 | |
| LongLLMLinguaRetriever=BERT, Generator Model=Llama-3.2-3B-Instruct2026.06 | 30.21 | 23.62 | |
| LongLLMLinguaRetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 30.21 | 23.62 | |
| MS-PoERetriever=BERT, Generator Model=Llama-3.2-3B-Instruct2026.06 | 28.6 | 23.39 | |
| MS-PoERetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 28.6 | 23.39 | |
| LongLLMLinguaRetriever=BERT, Generator=Qwen2.5-3B-Instruct2026.06 | 28.33 | 22.71 | |
| RAGRetriever=BERT, Generator Model=Llama-3.2-3B-Instruct2026.06 | 28.03 | 22.79 | |
| RAGRetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 28.03 | 22.79 | |
| LongRAGRetriever=BERT, Generator Model=Llama-3.2-3B-Instruct2026.06 | 26.25 | 21.04 | |
| LongRAGRetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 26.25 | 21.04 | |
| MS-PoERetriever=BERT, Generator=Qwen2.5-3B-Instruct2026.06 | 26.12 | 21.12 | |
| RAGRetriever=BERT, Generator=Qwen2.5-3B-Instruct2026.06 | 25.99 | 20.87 | |
| REPLUGRetriever=BERT, Generator Model=Llama-3.2-3B-Instruct2026.06 | 25.6 | 19.84 | |
| REPLUGRetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 25.6 | 19.84 | |
| PINERetriever=BERT, Generator Model=Llama-3.2-3B-Instruct2026.06 | 25.57 | 21.8 | |
| PINERetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 25.57 | 21.8 | |
| Summarized RAGRetriever=BERT, Generator Model=Llama-3.2-3B-Instruct2026.06 | 24.69 | 20.25 | |
| Summarized RAGRetriever=BERT, Generator=Llama-3.2-3B-Instruct2026.06 | 24.69 | 20.25 | |
| PINERetriever=BERT, Generator=Qwen2.5-3B-Instruct2026.06 | 24.24 | 20.39 | |
| LongRAGRetriever=BERT, Generator=Qwen2.5-3B-Instruct2026.06 | 22.99 | 18.61 | |
| Summarized RAGRetriever=BERT, Generator=Qwen2.5-3B-Instruct2026.06 | 22.75 | 18.64 | |
| REPLUGRetriever=BERT, Generator=Qwen2.5-3B-Instruct2026.06 | 17.98 | 14.66 |