Question Answering on MuSiQue (F1 score)
81.3F1 ScoreTri-RAG
Evaluation Results
| Method | Links | |
|---|---|---|
| Tri-RAGGenerator Backbone=Qwen-Long2026.04 | 81.3 | |
| AflowGenerator Backbone=Qwen-Long2026.04 | 78.9 | |
| Tri-RAGGenerator Backbone=GPT-4o-mini2026.04 | 74.6 | |
| Self-ActGenerator Backbone=Qwen-Long2026.04 | 72.3 | |
| AflowGenerator Backbone=GPT-4o-mini2026.04 | 71.6 | |
| Tri-RAGGenerator Backbone=GPT-4.1-nano2026.04 | 71.2 | |
| AflowGenerator Backbone=GPT-4.1-nano2026.04 | 69.4 | |
| Tri-RAGGenerator Backbone=Qwen-Max2026.04 | 68.3 | |
| Self-ActGenerator Backbone=GPT-4o-mini2026.04 | 67.3 | |
| Tri-RAGGenerator Backbone=Gemma-2-27B2026.04 | 67.1 | |
| AflowGenerator Backbone=Qwen-Max2026.04 | 65.3 | |
| AflowGenerator Backbone=Gemma-2-27B2026.04 | 63.4 | |
| Self-ActGenerator Backbone=Qwen-Max2026.04 | 61.8 | |
| Self-ActGenerator Backbone=GPT-4.1-nano2026.04 | 61.1 | |
| Self-ActGenerator Backbone=Gemma-2-27B2026.04 | 59.7 | |
| ReActGenerator Backbone=Qwen-Long2026.04 | 58.9 | |
| DirectGenerator Backbone=Qwen-Long2026.04 | 55.4 | |
| ReActGenerator Backbone=GPT-4o-mini2026.04 | 53.6 | |
| ReActGenerator Backbone=Qwen-Max2026.04 | 53.6 | |
| ReActGenerator Backbone=Gemma-2-27B2026.04 | 51.7 | |
| DirectGenerator Backbone=Qwen-Max2026.04 | 49.9 | |
| DirectGenerator Backbone=Gemma-2-27B2026.04 | 48.4 | |
| ReActGenerator Backbone=GPT-4.1-nano2026.04 | 47.3 | |
| GPT-4o (2024-11-20)Version=2024-11-202025.08 | 46.04 | |
| DirectGenerator Backbone=GPT-4o-mini2026.04 | 40.1 | |
| AGENT-RADAR2026.05 | 39.72 | |
| DirectGenerator Backbone=GPT-4.1-nano2026.04 | 38.2 | |
| AgentDropout2026.05 | 37.06 | |
| AgentPrune2026.05 | 37.05 | |
| CoT-SCsample_size (n)=82026.05 | 36.33 | |
| Vanilla2026.05 | 35.47 | |
| Compaction2026.05 | 34.43 | |
| StepSearchBackbone=Qwen2.5-7B Instruct2026.05 | 31.2 | |
| PiCABackbone=Qwen2.5-7B Instruct2026.05 | 28.7 | |
| TIPSBackbone=Qwen2.5-7B Instruct2026.05 | 27 | |
| StepSearchBackbone=Qwen2.5-3B Instruct2026.05 | 26.1 | |
| Search-R1Backbone=Qwen2.5-7B Instruct2026.05 | 25.6 | |
| PiCABackbone=Qwen2.5-3B Instruct2026.05 | 23.5 | |
| Llama + LoRRBackbone=Llama3.2-3B, Optimization=LoRR2025.08 | 21.59 | |
| Llama + DPOBackbone=Llama3.2-3B, Optimization=DPO2025.08 | 20.89 | |
| ZeroSearchBackbone=Qwen2.5-7B Instruct2026.05 | 20.4 | |
| Qwen + LoRRBackbone=Qwen2.5-3B, Optimization=LoRR2025.08 | 18.4 | |
| Qwen (Base)Backbone=Qwen2.5-3B, Mode=Base2025.08 | 18.28 | |
| Qwen + DPOBackbone=Qwen2.5-3B, Optimization=DPO2025.08 | 18.03 | |
| Llama (Base)Backbone=Llama3.2-3B, Mode=Base2025.08 | 17.21 | |
| Search-R1Backbone=Qwen2.5-3B Instruct2026.05 | 17.1 | |
| TIPSBackbone=Qwen2.5-3B Instruct2026.05 | 15.9 | |
| ZeroSearchBackbone=Qwen2.5-3B Instruct2026.05 | 14.5 | |
| RAGBackbone=Qwen2.5-7B Instruct2026.05 | 14.2 | |
| RAGBackbone=Qwen2.5-3B Instruct2026.05 | 13.5 | |
| Search-o1Backbone=Qwen2.5-7B Instruct2026.05 | 12.7 | |
| MT-PPOBackbone=Qwen2.5-7B Instruct2026.05 | 12.1 | |
| Search-o1Backbone=Qwen2.5-3B Instruct2026.05 | 11.7 | |
| MT-PPOBackbone=Qwen2.5-3B Instruct2026.05 | 11.1 |