Question Answering on HotpotQA In-Distribution (Document Accuracy)
93.95F1 ScoreSkillFlow
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SkillFlowMethod Category=Ours2026.05 | 93.95 | — | — | — | — | — | — | — | — | — | — | 92.19 | |
| FlowSteerMethod Category=Agent+RL2026.05 | 91.2 | — | — | — | — | — | — | — | — | — | — | 89.84 | |
| Qwen3.5-AFlowMethod Category=AFlow, Backbone=Qwen3.52026.05 | 90.92 | — | — | — | — | — | — | — | — | — | — | 88.28 | |
| AgentFlowMethod Category=Agent+RL2026.05 | 90.11 | — | — | — | — | — | — | — | — | — | — | 88.28 | |
| SkillRLMethod Category=Agent+RL2026.05 | 89.16 | — | — | — | — | — | — | — | — | — | — | 87.5 | |
| v4-flashMethod Category=Baseline2026.05 | 85.95 | — | — | — | — | — | — | — | — | — | — | 72.66 | |
| Qwen3.5-GRPOMethod Category=GRPO, Backbone=Qwen3.52026.05 | 81.52 | — | — | — | — | — | — | — | — | — | — | 69.53 | |
| Qwen3.5-SFTMethod Category=SFT, Backbone=Qwen3.52026.05 | 79.08 | — | — | — | — | — | — | — | — | — | — | 64.84 | |
| Qwen3.5-9BMethod Category=Baseline2026.05 | 75.7 | — | — | — | — | — | — | — | — | — | — | 60.94 | |
| ReSearch-7BModel Category=7B Models2025.07 | 51 | — | — | — | — | — | — | — | — | 46 | 49 | — | |
| Qwen2.5-VL-7BModel Category=7B Models, Backbone=Qwen2.5-VL-7B2025.07 | 41 | — | — | — | — | — | — | — | — | 33 | 58 | — | |
| Search-R1-7BModel Category=7B Models, Backbone=R1-7B2025.07 | 41 | — | — | — | — | — | — | — | — | 43 | 57 | — | |
| DeepSeek-R1-Distill-70BModel Category=70B Models, Backbone=DeepSeek-R1-Distill-70B2025.07 | 40 | — | — | — | — | — | — | — | — | 50 | 60 | — | |
| InternVL3-78BModel Category=70B Models, Backbone=InternVL3-78B2025.07 | 37 | — | — | — | — | — | — | — | — | 48 | 62 | — | |
| R1-Searcher-7BModel Category=7B Models, Backbone=R1-7B2025.07 | 35 | — | — | — | — | — | — | — | — | 57 | 64 | — | |
| Qwen2.5-VL-72BModel Category=70B Models, Backbone=Qwen2.5-VL-72B2025.07 | 31 | — | — | — | — | — | — | — | — | 54 | 68 | — | |
| GPT-4.1Model Category=Closed-source Models2025.07 | 26 | — | — | — | — | — | — | — | — | 72 | 74 | — | |
| GPT-4oModel Category=Closed-source Models2025.07 | 20 | — | — | — | — | — | — | — | — | 63 | 79 | — | |
| Claude Sonnet 4Model Category=Closed-source Models2025.07 | 16 | — | — | — | — | — | — | — | — | 73 | 83 | — | |
| Deliberative Searcher-72BModel Category=70B Models, Backbone=Qwen2.5-VL-72B2025.07 | 10 | — | — | — | — | — | — | — | — | 67 | 76 | — | |
| Deliberative Searcher-DeepSeek-70BModel Category=70B Models, Backbone=DeepSeek-R1-Distill-70B2025.07 | 9 | — | — | — | — | — | — | — | — | 65 | 76 | — | |
| Deliberative Searcher-7BModel Category=7B Models, Backbone=Qwen2.5-VL-7B2025.07 | 5 | — | — | — | — | — | — | — | — | 62 | 65 | — | |
| Deliberative Searcher-7BModel Category=7B Models, Backbone=Qwen2.5-VL-7B, Reward Function=ECE-based2025.07 | 4 | — | — | — | — | — | — | — | — | 66 | 72 | — | |
| MemAgentScale=3B2025.09 | — | 70.3 | 69.4 | 60.9 | 68.8 | 60.9 | 60.2 | 59.4 | 58.8 | — | — | — | |
| MemAgentScale=7B2025.09 | — | 81.8 | 78.9 | 78.9 | 77 | 79.7 | 72.1 | 74 | 75.8 | — | — | — | |
| Qwen2.5Scale=3B2025.09 | — | 59.4 | 57 | — | — | — | — | — | — | — | — | — | |
| Qwen2.5Scale=7B2025.09 | — | 70.3 | 75 | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-1MScale=7B2025.09 | — | 75.8 | 71.9 | 68 | 67.2 | 69.5 | 54.7 | 22.7 | 0 | — | — | — | |
| R1-DistillScale=7B2025.09 | — | 40.6 | 25.8 | 10.2 | 0.8 | 1.6 | 2.3 | 1.5 | 3.1 | — | — | — | |
| ReMemR1Scale=3B2025.09 | — | 70.9 | 71.7 | 63.8 | 74 | 65.4 | 65 | 65.4 | 66.1 | — | — | — | |
| ReMemR1Scale=7B2025.09 | — | 82.3 | 82.8 | 81.1 | 78.9 | 82 | 79.7 | 80 | 80.8 | — | — | — |