Question Answering on MuSiQue (EM, F1, R-1)
39.6EMRouteRAG
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RouteRAGBackbone=Qwen2.5-7B2025.12 | 39.6 | 49.3 | — | |
| HippoRAG 2Backbone=GPT-4o-mini2025.12 | 35 | 49.3 | — | |
| Search-R1Backbone=Qwen2.5-7B2025.12 | 32 | 40.8 | — | |
| RouteRAGBackbone=Qwen2.5-3B2025.12 | 30.7 | 39.3 | — | |
| R1-SearcherBackbone=Qwen2.5-7B2025.12 | 29.3 | 37.6 | — | |
| RAPTORBackbone=GPT-4o-mini2025.12 | 27.7 | 39.2 | — | |
| MemAdapterAgent Model=Qwen2.5-7B-Instruct2026.02 | 27.64 | 40.2 | 41.54 | |
| GraphRAGBackbone=GPT-4o-mini2025.12 | 27 | 42 | — | |
| ITER-RETGENType=Mem.2026.01 | 25.9 | — | — | |
| HippoRAGBackbone=GPT-4o-mini2025.12 | 24 | 35.9 | — | |
| ProRAGMethod Category=Reinforcement Learning-based Methods2026.01 | 23.5 | 34.1 | — | |
| Mem0Agent Model=Qwen2.5-7B-Instruct2026.02 | 23.33 | 36.73 | 37.72 | |
| StepSearchType=RL Training2026.01 | 22.6 | — | — | |
| Search-R1Backbone=Qwen2.5-3B2025.12 | 22.2 | 30.9 | — | |
| MemAdapterAgent Model=Qwen2.5-3B-Instruct2026.02 | 21.93 | 34.81 | 36.38 | |
| HippoRAGType=Mem., Backbone=GPT-3.52026.01 | 21.9 | — | — | |
| Ideal RerankerCategory=Oracle Study, Retrieval=BGE2025.12 | 21.6 | 30.3 | — | |
| Search-R1Method Category=Reinforcement Learning-based Methods2026.01 | 20.6 | 29.5 | — | |
| Classifier+LLM (Structured)Retrieval=BGE, Note=Input reordered for relevant text at end2025.12 | 20.2 | 29.1 | — | |
| Classifier+LLMRetrieval=BGE2025.12 | 19.9 | 28.3 | — | |
| Classifier+LLMRetrieval=MonoT52025.12 | 19.5 | 27.7 | — | |
| StreamingLLMAgent Model=Qwen2.5-7B-Instruct2026.02 | 18.74 | 29.56 | 30.32 | |
| BaselineRetrieval=BGE2025.12 | 17.7 | 26 | — | |
| BaselineRetrieval=MonoT52025.12 | 17.1 | 25.2 | — | |
| Control PipelineRetrieval=BGE2025.12 | 17.1 | 21.2 | — | |
| ACRType=SFT, Reasoner=Qwen-2.5-7B-Instruct2026.01 | 16.67 | — | — | |
| ClassifierRetrieval=BGE2025.12 | 16.2 | 24.3 | — | |
| ClassifierRetrieval=MonoT52025.12 | 15.9 | 23.9 | — | |
| HiPRAGMethod Category=Reinforcement Learning-based Methods2026.01 | 15.6 | 25 | — | |
| Classifier+LLMRetrieval=Dense2025.12 | 15.3 | 23.2 | — | |
| Mem0Agent Model=Qwen2.5-3B-Instruct2026.02 | 14.94 | 24.36 | 25.25 | |
| Search-R1Backbone=Qwen2.5-7B-Instruct2026.02 | 14.6 | — | — | |
| Search-R1Type=RL Training2026.01 | 14.6 | — | — | |
| MemAdapterAgent Model=Qwen2.5-1.5B-Instruct2026.02 | 14.36 | 22.84 | 23.8 | |
| BaselineRetrieval=Dense2025.12 | 14.1 | 22.1 | — | |
| A-MemAgent Model=Qwen2.5-7B-Instruct2026.02 | 13.74 | 23 | 23.53 | |
| Search-o1Method Category=Advanced Methods2026.01 | 13 | 19.9 | — | |
| ReasonRAGMethod Category=Reinforcement Learning-based Methods2026.01 | 12.8 | 20.6 | — | |
| Ideal ClassifierCategory=Oracle Study, Note=Optimal k under dense retrieval2025.12 | 12.8 | 20 | — | |
| ClassifierRetrieval=Dense2025.12 | 12.4 | 19.5 | — | |
| HippoRAG 2Backbone=Qwen2.5-7B2025.12 | 12.3 | 24 | — | |
| HippoRAG 2Backbone=Qwen2.5-3B2025.12 | 12.2 | 20.2 | — | |
| BAR-RAGBackbone=LLaMA-3.1-8B-Instruct2026.02 | 12 | — | — | |
| Vanilla RAGBackbone=Qwen2.5-7B2025.12 | 11.4 | 23 | — | |
| Direct InferenceBackbone=GPT-4o-mini2025.12 | 11.2 | 22 | — | |
| Search-R1Backbone=Qwen2.5-3B-Instruct2026.02 | 10.3 | — | — | |
| Vanilla RAGBackbone=Qwen2.5-3B2025.12 | 10.3 | 17.5 | — | |
| BAR-RAGBackbone=Qwen2.5-7B-Instruct2026.02 | 9.1 | — | — | |
| A-MemAgent Model=Qwen2.5-3B-Instruct2026.02 | 8.44 | 15.89 | 16.55 | |
| StreamingLLMAgent Model=Qwen2.5-3B-Instruct2026.02 | 8.03 | 15.98 | 16.58 | |
| BAR-RAGBackbone=Qwen2.5-3B-Instruct2026.02 | 7.4 | — | — | |
| REPAIR-7BRetrieval Mode=Reranking, Reranking pool size=top-100, Top-k context documents=10, Answer generator model=Llama-3.1-8B-Instruct2026.01 | 7.4 | 14.8 | — | |
| Mem0Agent Model=Qwen2.5-1.5B-Instruct2026.02 | 7.21 | 14.73 | 16.88 | |
| IRCoTType=Prompt2026.01 | 7.2 | — | — | |
| SFTType=SFT2026.01 | 6.6 | — | — | |
| Classifier+LLMRetrieval=BM252025.12 | 6.3 | 10.7 | — | |
| R1-InstructionType=RL Training2026.01 | 6 | — | — | |
| BaselineRetrieval=BM252025.12 | 6 | 10.5 | — | |
| FLAREMethod Category=Advanced Methods2026.01 | 5.8 | 13.5 | — | |
| Search-o1Backbone=Qwen2.5-7B-Instruct2026.02 | 5.8 | — | — | |
| REARANK-7BRetrieval Mode=Reranking, Reranking pool size=top-100, Top-k context documents=10, Answer generator model=Llama-3.1-8B-Instruct2026.01 | 5.6 | 13 | — | |
| Search-o1Backbone=Qwen2.5-3B-Instruct2026.02 | 5.4 | — | — | |
| StreamingLLMAgent Model=Qwen2.5-1.5B-Instruct2026.02 | 5.26 | 10.01 | 12.17 | |
| RankZephyr-7BRetrieval Mode=Reranking, Reranking pool size=top-100, Top-k context documents=10, Answer generator model=Llama-3.1-8B-Instruct2026.01 | 5.2 | 12.1 | — | |
| Rank-R1-7BRetrieval Mode=Reranking, Reranking pool size=top-100, Top-k context documents=10, Answer generator model=Llama-3.1-8B-Instruct2026.01 | 5.2 | 12.3 | — | |
| IRCoTMethod Category=Advanced Methods2026.01 | 5.1 | 13.1 | — | |
| Iter-RetGenMethod Category=Advanced Methods2026.01 | 4.8 | 8.5 | — | |
| A-MemAgent Model=Qwen2.5-1.5B-Instruct2026.02 | 4.68 | 9.88 | 10.41 | |
| ClassifierRetrieval=BM252025.12 | 4.6 | 8.6 | — | |
| Search-o1Backbone=Qwen2.5-3B2025.12 | 3.9 | 10.5 | — | |
| MemoryLLMAgent Model=Qwen2.5-7B-Instruct2026.02 | 3.48 | 7.34 | 8.35 | |
| QwenRerank-7BRetrieval Mode=Reranking, Reranking pool size=top-100, Top-k context documents=10, Answer generator model=Llama-3.1-8B-Instruct2026.01 | 3.4 | 10 | — | |
| Standard RAGMethod Category=Standard Methods2026.01 | 3.3 | 10.3 | — | |
| Direct inferenceType=Prompt2026.01 | 3.1 | — | — | |
| Qwen3-Embedding-0.6BRetrieval Mode=Initial Retrieval, Top-k context documents=10, Answer generator model=Llama-3.1-8B-Instruct2026.01 | 3 | 9.9 | — | |
| Search-o1Backbone=Qwen2.5-7B2025.12 | 2.9 | 7.7 | — | |
| MemoryLLMAgent Model=Qwen2.5-3B-Instruct2026.02 | 2.44 | 5.86 | 6.52 | |
| CoTType=Prompt2026.01 | 2.2 | — | — | |
| Naïve GenerationMethod Category=Standard Methods2026.01 | 2.1 | 9.5 | — | |
| LightRAGBackbone=GPT-4o-mini2025.12 | 2 | 9.3 | — | |
| MemoryLLMAgent Model=Qwen2.5-1.5B-Instruct2026.02 | 1.34 | 3.67 | 3.81 | |
| CAREAgent Model=Qwen2.5-7B-Instruct2026.02 | 0.86 | 6.17 | 6.42 | |
| CAREAgent Model=Qwen2.5-3B-Instruct2026.02 | 0.63 | 5.26 | 5.49 | |
| CAREAgent Model=Qwen2.5-1.5B-Instruct2026.02 | 0.56 | 4.83 | 5.07 |