Open-domain QA on Bambogle v1 (out-of-domain)
64.7F1 ScoreR1-Searcher
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| R1-SearcherModel=LLaMA-3.1-8B, Type=RAG-RL2025.08 | 64.7 | 62.4 | |
| UR2Model=Qwen-2.5-7B, Type=Our Implementations2025.08 | 64.5 | 62.4 | |
| Vanilla RLModel=Qwen-2.5-7B, Type=Our Implementations2025.08 | 63.3 | 63.2 | |
| R1-Searcher++Model=Qwen-2.5-7B, Type=RAG-RL2025.08 | 60.8 | 59.2 | |
| UR2Model=LLaMA-3.1-8B, Type=Our Implementations2025.08 | 60.7 | 63.2 | |
| Vanilla RLModel=LLaMA-3.1-8B, Type=Our Implementations2025.08 | 60.6 | 63.2 | |
| Self-AskModel=GPT-4o-mini, Type=Advanced RAG2025.08 | 59.3 | 57.6 | |
| Vanilla RLModel=Qwen-2.5-3B, Type=Our Implementations2025.08 | 59 | 57.6 | |
| Search-R1Model=Qwen-2.5-7B, Type=RAG-RL2025.08 | 58.9 | 56.8 | |
| UR2Model=Qwen-2.5-3B, Type=Our Implementations2025.08 | 57.8 | 58.4 | |
| R1-SearcherModel=Qwen-2.5-7B, Type=RAG-RL2025.08 | 56.5 | 53.6 | |
| CoTModel=GPT-4o-mini, Type=Vanilla Methods2025.08 | 55.2 | 62.4 | |
| Search-o1Model=GPT-4o-mini, Type=Advanced RAG2025.08 | 54.5 | 56 | |
| RATModel=GPT-4o-mini, Type=Advanced RAG2025.08 | 53 | 51.2 | |
| Search-R1Model=Qwen-2.5-3B, Type=RAG-RL2025.08 | 48.3 | 48 | |
| Standard RAGModel=GPT-4o-mini, Type=Vanilla Methods2025.08 | 46.7 | 46.4 | |
| CoTModel=LLaMA-3.1-8B, Type=Vanilla Methods2025.08 | 43 | 42.4 | |
| Self-AskModel=LLaMA-3.1-8B, Type=Advanced RAG2025.08 | 41.5 | 44.8 | |
| CoTModel=Qwen-2.5-7B, Type=Vanilla Methods2025.08 | 41.3 | 43.2 | |
| RATModel=LLaMA-3.1-8B, Type=Advanced RAG2025.08 | 39.7 | 39.2 | |
| Standard RAGModel=Qwen-2.5-7B, Type=Vanilla Methods2025.08 | 38.9 | 40 | |
| Search-o1Model=Qwen-2.5-7B, Type=Advanced RAG2025.08 | 37.5 | 39.2 | |
| ZeroSearchModel=Qwen-2.5-7B, Type=RAG-RL2025.08 | 35.8 | 38.4 | |
| Zero-SearchModel=Qwen-2.5-3B, Type=RAG-RL2025.08 | 32.4 | 31.2 | |
| RATModel=Qwen-2.5-7B, Type=Advanced RAG2025.08 | 31.6 | 30.4 | |
| Search-o1Model=Qwen-2.5-3B, Type=Advanced RAG2025.08 | 31.4 | 32 | |
| CoTModel=Qwen-2.5-3B, Type=Vanilla Methods2025.08 | 31.2 | 33.6 | |
| Self-AskModel=Qwen-2.5-3B, Type=Advanced RAG2025.08 | 30.6 | 32 | |
| RATModel=Qwen-2.5-3B, Type=Advanced RAG2025.08 | 30.6 | 28 | |
| Search-o1Model=LLaMA-3.1-8B, Type=Advanced RAG2025.08 | 30 | 30.4 | |
| Self-AskModel=Qwen-2.5-7B, Type=Advanced RAG2025.08 | 28.9 | 42.4 | |
| Standard RAGModel=LLaMA-3.1-8B, Type=Vanilla Methods2025.08 | 26.5 | 28 | |
| Standard RAGModel=Qwen-2.5-3B, Type=Vanilla Methods2025.08 | 26.1 | 27.2 |