Question Answering on TriviaQA (test) (Acc and F1)
80.6F1 ScoreIGPO
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| IGPOMethod Category=Step-reward RL-based2025.10 | 80.6 | — | |
| DeepResearcherMethod Category=Outcome-reward RL-based2025.10 | 78.4 | — | |
| R1-searcherMethod Category=Outcome-reward RL-based2025.10 | 73.1 | — | |
| Search-r1-baseMethod Category=Outcome-reward RL-based2025.10 | 71.9 | — | |
| CoT+RAGMethod Category=Prompt-based2025.10 | 68.9 | — | |
| GiGPOMethod Category=Step-reward RL-based2025.10 | 64.7 | — | |
| Search-o1Method Category=Prompt-based2025.10 | 58.9 | — | |
| CoTMethod Category=Prompt-based2025.10 | 45.6 | — | |
| Search-r1-instructMethod Category=Outcome-reward RL-based2025.10 | 44.7 | — | |
| UR-RAGRank-By=Utility, LLM Backbone=Qwen-14B, Retrieval Method=BM252026.01 | 42.39 | 41.86 | |
| RePlugRank-By=Utility, LLM Backbone=Qwen-14B, Retrieval Method=BM252026.01 | 41.25 | 41.12 | |
| LURE-RAGRank-By=Utility, LLM Backbone=Qwen-14B, Retrieval Method=BM252026.01 | 40.47 | 40.57 | |
| LURE-RAG (w/o topics)Rank-By=Utility, LLM Backbone=Qwen-14B, Retrieval Method=BM252026.01 | 40.01 | 40.08 | |
| k-shotRank-By=Relevance, LLM Backbone=Qwen-14B, Retrieval Method=BM252026.01 | 39.78 | 39.76 | |
| 0-shotRank-By=Relevance, LLM Backbone=Qwen-14B, Retrieval Method=BM252026.01 | 35.87 | 35.66 | |
| UR-RAGRank-By=Utility, LLM Backbone=phi-3.8B, Retrieval Method=BM252026.01 | 34.72 | 36.76 | |
| RePlugRank-By=Utility, LLM Backbone=phi-3.8B, Retrieval Method=BM252026.01 | 33.21 | 36.03 | |
| LURE-RAGRank-By=Utility, LLM Backbone=phi-3.8B, Retrieval Method=BM252026.01 | 32.72 | 35.69 | |
| LURE-RAG (w/o topics)Rank-By=Utility, LLM Backbone=phi-3.8B, Retrieval Method=BM252026.01 | 32.61 | 35.44 | |
| k-shotRank-By=Relevance, LLM Backbone=phi-3.8B, Retrieval Method=BM252026.01 | 31.77 | 34.86 | |
| UR-RAGRank-By=Utility, LLM Backbone=llama-1B, Retrieval Method=BM252026.01 | 31.04 | 31.11 | |
| RePlugRank-By=Utility, LLM Backbone=llama-1B, Retrieval Method=BM252026.01 | 30.17 | 30.27 | |
| LURE-RAGRank-By=Utility, LLM Backbone=llama-1B, Retrieval Method=BM252026.01 | 30.1 | 29.68 | |
| LURE-RAG (w/o topics)Rank-By=Utility, LLM Backbone=llama-1B, Retrieval Method=BM252026.01 | 29.73 | 29.47 | |
| k-shotRank-By=Relevance, LLM Backbone=llama-1B, Retrieval Method=BM252026.01 | 26 | 28.41 | |
| 0-shotRank-By=Relevance, LLM Backbone=phi-3.8B, Retrieval Method=BM252026.01 | 22.14 | 27.79 | |
| 0-shotRank-By=Relevance, LLM Backbone=llama-1B, Retrieval Method=BM252026.01 | 21.37 | 22.58 |