Open-domain Question Answering on MusiQue out-of-domain
35.8F1UR2
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| UR2Backbone=Qwen-2.5-7B, Type=Our Implementations2025.08 | 35.8 | 34.6 | |
| UR2Model=Qwen-2.5-7B, Type=Our Implementations2025.08 | 35.8 | 34.6 | |
| Vanilla RLBackbone=Qwen-2.5-7B, Type=Our Implementations2025.08 | 34.4 | 34.4 | |
| Vanilla RLModel=Qwen-2.5-7B, Type=Our Implementations2025.08 | 34.4 | 34.4 | |
| UR2Backbone=LLaMA-3.1-8B, Type=Our Implementations2025.08 | 34.3 | 34 | |
| UR2Model=LLaMA-3.1-8B, Type=Our Implementations2025.08 | 34.3 | 34 | |
| R1-Searcher++Backbone=Qwen-2.5-7B, Type=RAG-RL2025.08 | 33.8 | 32.8 | |
| R1-Searcher++Model=Qwen-2.5-7B, Type=RAG-RL2025.08 | 33.8 | 32.8 | |
| R1-SearcherBackbone=Qwen-2.5-7B, Type=RAG-RL2025.08 | 33.3 | 32.6 | |
| R1-SearcherModel=Qwen-2.5-7B, Type=RAG-RL2025.08 | 33.3 | 32.6 | |
| Vanilla RLBackbone=LLaMA-3.1-8B, Type=Our Implementations2025.08 | 32.7 | 31.8 | |
| Vanilla RLModel=LLaMA-3.1-8B, Type=Our Implementations2025.08 | 32.7 | 31.8 | |
| Search-R1Backbone=Qwen-2.5-7B, Type=RAG-RL2025.08 | 32.2 | 32 | |
| Search-R1Model=Qwen-2.5-7B, Type=RAG-RL2025.08 | 32.2 | 32 | |
| Self-AskBackbone=GPT-4.1-mini, Type=Advanced RAG2025.08 | 31.6 | 35 | |
| R1-SearcherBackbone=LLaMA-3.1-8B, Type=RAG-RL2025.08 | 31.1 | 29.4 | |
| R1-SearcherModel=LLaMA-3.1-8B, Type=RAG-RL2025.08 | 31.1 | 29.4 | |
| UR2Backbone=Qwen-2.5-3B, Type=Our Implementations2025.08 | 30.5 | 31.6 | |
| UR2Model=Qwen-2.5-3B, Type=Our Implementations2025.08 | 30.5 | 31.6 | |
| Vanilla RLBackbone=Qwen-2.5-3B, Type=Our Implementations2025.08 | 30 | 29.6 | |
| Vanilla RLModel=Qwen-2.5-3B, Type=Our Implementations2025.08 | 30 | 29.6 | |
| Search-o1Model=GPT-4o-mini, Type=Advanced RAG2025.08 | 29.6 | 30.2 | |
| RATBackbone=GPT-4.1-mini, Type=Advanced RAG2025.08 | 29 | 31.4 | |
| Search-o1Backbone=GPT-4.1-mini, Type=Advanced RAG2025.08 | 28.6 | 33.4 | |
| CoTBackbone=GPT-4.1-mini, Type=Vanilla Methods2025.08 | 28.3 | 35.4 | |
| Search-R1Backbone=Qwen-2.5-3B, Type=RAG-RL2025.08 | 27.6 | 27.8 | |
| Search-R1Model=Qwen-2.5-3B, Type=RAG-RL2025.08 | 27.6 | 27.8 | |
| Self-AskModel=GPT-4o-mini, Type=Advanced RAG2025.08 | 26.1 | 27.8 | |
| CoTModel=GPT-4o-mini, Type=Vanilla Methods2025.08 | 24.9 | 26.8 | |
| RATModel=GPT-4o-mini, Type=Advanced RAG2025.08 | 24.3 | 24.8 | |
| Standard RAGBackbone=GPT-4.1-mini, Type=Vanilla Methods2025.08 | 21.9 | 28.4 | |
| Search-o1Model=Qwen-2.5-7B, Type=Advanced RAG2025.08 | 20.6 | 19.8 | |
| Standard RAGModel=GPT-4o-mini, Type=Vanilla Methods2025.08 | 19.3 | 21.6 | |
| RATModel=LLaMA-3.1-8B, Type=Advanced RAG2025.08 | 17 | 16 | |
| Zero-SearchBackbone=Qwen-2.5-3B, Type=RAG-RL2025.08 | 16.9 | 17 | |
| Zero-SearchModel=Qwen-2.5-3B, Type=RAG-RL2025.08 | 16.9 | 17 | |
| Self-AskModel=LLaMA-3.1-8B, Type=Advanced RAG2025.08 | 16.8 | 16.4 | |
| Search-o1Model=LLaMA-3.1-8B, Type=Advanced RAG2025.08 | 15.9 | 16.2 | |
| CoTBackbone=Qwen-2.5-7B, Type=Vanilla Methods2025.08 | 14.8 | 12.2 | |
| CoTModel=Qwen-2.5-7B, Type=Vanilla Methods2025.08 | 14.8 | 12.2 | |
| ZeroSearchBackbone=Qwen-2.5-7B, Type=RAG-RL2025.08 | 14.7 | 13.8 | |
| ZeroSearchModel=Qwen-2.5-7B, Type=RAG-RL2025.08 | 14.7 | 13.8 | |
| Self-AskModel=Qwen-2.5-3B, Type=Advanced RAG2025.08 | 14.5 | 14.8 | |
| Standard RAGBackbone=Qwen-2.5-7B, Type=Vanilla Methods2025.08 | 14.4 | 14.6 | |
| Standard RAGModel=Qwen-2.5-7B, Type=Vanilla Methods2025.08 | 14.4 | 14.6 | |
| RATModel=Qwen-2.5-7B, Type=Advanced RAG2025.08 | 14.4 | 12.4 | |
| Self-AskModel=Qwen-2.5-7B, Type=Advanced RAG2025.08 | 13.6 | 19.6 | |
| Search-o1Model=Qwen-2.5-3B, Type=Advanced RAG2025.08 | 12.5 | 10 | |
| CoTBackbone=Qwen-2.5-3B, Type=Vanilla Methods2025.08 | 11.3 | 9.6 | |
| CoTModel=Qwen-2.5-3B, Type=Vanilla Methods2025.08 | 11.3 | 9.6 | |
| RATModel=Qwen-2.5-3B, Type=Advanced RAG2025.08 | 11 | 8.2 | |
| Standard RAGBackbone=LLaMA-3.1-8B, Type=Vanilla Methods2025.08 | 10.1 | 10.2 | |
| Standard RAGModel=LLaMA-3.1-8B, Type=Vanilla Methods2025.08 | 10.1 | 10.2 | |
| CoTBackbone=LLaMA-3.1-8B, Type=Vanilla Methods2025.08 | 9.8 | 10.8 | |
| CoTModel=LLaMA-3.1-8B, Type=Vanilla Methods2025.08 | 9.8 | 10.8 | |
| Standard RAGBackbone=Qwen-2.5-3B, Type=Vanilla Methods2025.08 | 9.7 | 7.4 | |
| Standard RAGModel=Qwen-2.5-3B, Type=Vanilla Methods2025.08 | 9.7 | 7.4 |