Open-domain Question Answering on HotpotQA in-domain
72.4F1 ScoreSearch-R1
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Search-R1Model=Qwen-2.5-7B, Type=RAG-RL2025.08 | 72.4 | 78.8 | |
| R1-SearcherModel=Qwen-2.5-7B, Type=RAG-RL2025.08 | 71.8 | 78 | |
| UR2Model=Qwen-2.5-7B, Type=Our Implementations2025.08 | 71.2 | 79.4 | |
| Vanilla RLModel=Qwen-2.5-7B, Type=Our Implementations2025.08 | 70.9 | 78.8 | |
| R1-SearcherModel=LLaMA-3.1-8B, Type=RAG-RL2025.08 | 70.8 | 76.8 | |
| UR2Model=LLaMA-3.1-8B, Type=Our Implementations2025.08 | 70.1 | 78.8 | |
| Vanilla RLModel=LLaMA-3.1-8B, Type=Our Implementations2025.08 | 70 | 77.6 | |
| UR2Model=Qwen-2.5-3B, Type=Our Implementations2025.08 | 67.7 | 76 | |
| Vanilla RLModel=Qwen-2.5-3B, Type=Our Implementations2025.08 | 65.9 | 73.6 | |
| Search-o1Model=GPT-4o-mini, Type=Advanced RAG2025.08 | 64.3 | 73.4 | |
| Search-R1Model=Qwen-2.5-3B, Type=RAG-RL2025.08 | 63.1 | 69.2 | |
| Standard RAGModel=GPT-4o-mini, Type=Vanilla Methods2025.08 | 59.6 | 69.6 | |
| R1-Searcher++Model=Qwen-2.5-7B, Type=RAG-RL2025.08 | 59 | 64.2 | |
| RATModel=GPT-4o-mini, Type=Advanced RAG2025.08 | 53.8 | 59.2 | |
| Search-o1Model=LLaMA-3.1-8B, Type=Advanced RAG2025.08 | 53 | 59.4 | |
| Search-o1Model=Qwen-2.5-7B, Type=Advanced RAG2025.08 | 50.9 | 61.6 | |
| Standard RAGModel=Qwen-2.5-3B, Type=Vanilla Methods2025.08 | 50.6 | 57 | |
| Standard RAGModel=Qwen-2.5-7B, Type=Vanilla Methods2025.08 | 49.2 | 62.8 | |
| Standard RAGModel=LLaMA-3.1-8B, Type=Vanilla Methods2025.08 | 47.5 | 54.4 | |
| CoTModel=GPT-4o-mini, Type=Vanilla Methods2025.08 | 46.5 | 51.2 | |
| ZeroSearchModel=Qwen-2.5-7B, Type=RAG-RL2025.08 | 46 | 50.4 | |
| Self-AskModel=GPT-4o-mini, Type=Advanced RAG2025.08 | 45 | 50.4 | |
| RATModel=LLaMA-3.1-8B, Type=Advanced RAG2025.08 | 44.5 | 48.8 | |
| Self-AskModel=LLaMA-3.1-8B, Type=Advanced RAG2025.08 | 43 | 50.8 | |
| Zero-SearchModel=Qwen-2.5-3B, Type=RAG-RL2025.08 | 42.7 | 45.8 | |
| RATModel=Qwen-2.5-7B, Type=Advanced RAG2025.08 | 37.9 | 40.6 | |
| Search-o1Model=Qwen-2.5-3B, Type=Advanced RAG2025.08 | 36.4 | 37.6 | |
| Self-AskModel=Qwen-2.5-3B, Type=Advanced RAG2025.08 | 33.8 | 47.2 | |
| RATModel=Qwen-2.5-3B, Type=Advanced RAG2025.08 | 30.1 | 32.2 | |
| Self-AskModel=Qwen-2.5-7B, Type=Advanced RAG2025.08 | 28.8 | 61 | |
| CoTModel=LLaMA-3.1-8B, Type=Vanilla Methods2025.08 | 28.6 | 31.6 | |
| CoTModel=Qwen-2.5-3B, Type=Vanilla Methods2025.08 | 26.6 | 27.2 | |
| CoTModel=Qwen-2.5-7B, Type=Vanilla Methods2025.08 | 24.9 | 31 | |
| Search-R1Backbone=Qwen-2.5-7B, Type=RAG-RL2025.08 | 0.724 | 0.788 | |
| R1-SearcherBackbone=Qwen-2.5-7B, Type=RAG-RL2025.08 | 0.718 | 0.78 | |
| UR2Backbone=Qwen-2.5-7B, Type=Our Implementations2025.08 | 0.712 | 0.794 | |
| Vanilla RLBackbone=Qwen-2.5-7B, Type=Our Implementations2025.08 | 0.709 | 0.788 | |
| R1-SearcherBackbone=LLaMA-3.1-8B, Type=RAG-RL2025.08 | 0.708 | 0.768 | |
| UR2Backbone=LLaMA-3.1-8B, Type=Our Implementations2025.08 | 0.701 | 0.788 | |
| Vanilla RLBackbone=LLaMA-3.1-8B, Type=Our Implementations2025.08 | 0.7 | 0.776 | |
| UR2Backbone=Qwen-2.5-3B, Type=Our Implementations2025.08 | 0.677 | 0.76 | |
| Vanilla RLBackbone=Qwen-2.5-3B, Type=Our Implementations2025.08 | 0.659 | 0.736 | |
| Self-AskBackbone=GPT-4.1-mini, Type=Advanced RAG2025.08 | 0.654 | 0.75 | |
| Search-R1Backbone=Qwen-2.5-3B, Type=RAG-RL2025.08 | 0.631 | 0.692 | |
| R1-Searcher++Backbone=Qwen-2.5-7B, Type=RAG-RL2025.08 | 0.59 | 0.642 | |
| RATBackbone=GPT-4.1-mini, Type=Advanced RAG2025.08 | 0.569 | 0.642 | |
| Standard RAGBackbone=GPT-4.1-mini, Type=Vanilla Methods2025.08 | 0.545 | 0.744 | |
| Search-o1Backbone=GPT-4.1-mini, Type=Advanced RAG2025.08 | 0.531 | 0.74 | |
| Standard RAGBackbone=Qwen-2.5-3B, Type=Vanilla Methods2025.08 | 0.506 | 0.57 | |
| Standard RAGBackbone=Qwen-2.5-7B, Type=Vanilla Methods2025.08 | 0.492 | 0.628 | |
| Standard RAGBackbone=LLaMA-3.1-8B, Type=Vanilla Methods2025.08 | 0.475 | 0.544 | |
| ZeroSearchBackbone=Qwen-2.5-7B, Type=RAG-RL2025.08 | 0.46 | 0.504 | |
| CoTBackbone=GPT-4.1-mini, Type=Vanilla Methods2025.08 | 0.437 | 0.592 | |
| Zero-SearchBackbone=Qwen-2.5-3B, Type=RAG-RL2025.08 | 0.427 | 0.458 | |
| CoTBackbone=LLaMA-3.1-8B, Type=Vanilla Methods2025.08 | 0.286 | 0.316 | |
| CoTBackbone=Qwen-2.5-3B, Type=Vanilla Methods2025.08 | 0.266 | 0.272 | |
| CoTBackbone=Qwen-2.5-7B, Type=Vanilla Methods2025.08 | 0.249 | 0.31 |