Question Answering on GAIA (Pass@4, Usage Metrics)
51Accuracy (Pass@4)SynPlanResearch-R1
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SynPlanResearch-R1Model Backbone=Qwen3-8B2026.03 | 51 | — | — | — | — | — | |
| SearchGym-RL (Ours)Source=local, Backbone=Qwen-3-8B2026.01 | 50.49 | 5,200.55 | 5.13 | 1.81 | 0 | — | |
| SimpleDeepSearcherModel Backbone=Qwen3-8B2026.03 | 45.2 | — | — | — | — | — | |
| SynPlanResearch-R1Model Backbone=Qwen3-4B2026.03 | 43.9 | — | — | — | — | — | |
| SearchGym-RL (Ours-base)Source=local, Backbone=Qwen-2.5-7B-Base2026.01 | 42.72 | 1,114.48 | 3.71 | 0.9 | 0 | — | |
| Rejection SamplingModel Backbone=Qwen3-8B2026.03 | 41.7 | — | — | — | — | — | |
| SimpleDeepSearcherModel Backbone=Qwen3-4B2026.03 | 41.5 | — | — | — | — | — | |
| Search-R1Model Backbone=Qwen3-8B2026.03 | 40.5 | — | — | — | — | — | |
| ASearcher-base-webSource=web, Backbone=Qwen-2.5-7B-Base2026.01 | 38.83 | 692.28 | 5.92 | 0.07 | — | — | |
| Search-R1Model Backbone=Qwen3-4B2026.03 | 36.2 | — | — | — | — | — | |
| Rejection SamplingModel Backbone=Qwen3-4B2026.03 | 31.2 | — | — | — | — | — | |
| ASearcher-base-localSource=local, Backbone=Qwen-2.5-7B-Base2026.01 | 26.21 | 1,022.38 | 7.03 | 0.03 | 0 | — | |
| Search-o1Model Backbone=Qwen3-8B2026.03 | 25.7 | — | — | — | — | — | |
| Search-o1Model Backbone=Qwen3-4B2026.03 | 25 | — | — | — | — | — | |
| EPC-AWBackbone=Qwen3-Coder-30B2026.05 | 15.22 | — | — | — | — | — | |
| RAGModel Backbone=Qwen3-4B2026.03 | 15 | — | — | — | — | — | |
| Direct InferenceModel Backbone=Qwen3-8B2026.03 | 14.6 | — | — | — | — | — | |
| RAGModel Backbone=Qwen3-8B2026.03 | 13 | — | — | — | — | — | |
| Direct InferenceModel Backbone=Qwen3-4B2026.03 | 11.9 | — | — | — | — | — | |
| RollbackBackbone=Qwen3-Coder-30B2026.05 | 10.76 | — | — | — | — | 4.46 | |
| RetryBackbone=Qwen3-Coder-30B2026.05 | 9.17 | — | — | — | — | 6.05 | |
| No-Repair (AgentFlow)Backbone=Qwen3-Coder-30B2026.05 | 7.09 | — | — | — | — | 8.13 |