Question Answering on GPQA (EM, F1)
53.5EMPruneRAG
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PruneRAGBackbone=Qwen3-8B, Time Cost (ms)=88952026.01 | 53.5 | 53.5 | |
| ConTReGenBackbone=Qwen3-8B, Time Cost (ms)=188302026.01 | 51 | 51 | |
| Standard RAGBackbone=Qwen3-8B, Time Cost (ms)=36732026.01 | 50.5 | 50.5 | |
| RAG-StarBackbone=Qwen3-8B, Time Cost (ms)=128222026.01 | 47.9 | 47.9 | |
| SynPlanResearch-R1Model Backbone=Qwen3-8B2026.03 | 47.5 | — | |
| VanillaBackbone=Qwen3-8B, Time Cost (ms)=34332026.01 | 47.4 | 47.4 | |
| SimpleDeepSearcherModel Backbone=Qwen3-8B2026.03 | 46.6 | — | |
| Search-R1Model Backbone=Qwen3-8B2026.03 | 44.3 | — | |
| Rejection SamplingModel Backbone=Qwen3-8B2026.03 | 42.5 | — | |
| ProbTreeBackbone=Qwen3-8B, Time Cost (ms)=93042026.01 | 37.3 | 37.3 | |
| MemoRAGBackbone=Qwen3-8B, Time Cost (ms)=339662026.01 | 35.8 | 35.8 | |
| SynPlanResearch-R1Model Backbone=Qwen3-4B2026.03 | 33.4 | — | |
| Search-o1Model Backbone=Qwen3-8B2026.03 | 32.4 | — | |
| SimpleDeepSearcherModel Backbone=Qwen3-4B2026.03 | 31.2 | — | |
| Search-o1Backbone=Qwen3-8B, Time Cost (ms)=95272026.01 | 30.3 | 30.3 | |
| PruneRAGBackbone=Llama-3.1-8B-Instruct, Time Cost (ms)=8552026.01 | 29.7 | 29.7 | |
| Search-R1Model Backbone=Qwen3-4B2026.03 | 27.1 | — | |
| Rejection SamplingModel Backbone=Qwen3-4B2026.03 | 24.9 | — | |
| VanillaBackbone=Llama-3.1-8B-Instruct, Time Cost (ms)=1532026.01 | 24.7 | 24.7 | |
| Standard RAGBackbone=Llama-3.1-8B-Instruct, Time Cost (ms)=8132026.01 | 24.2 | 24.2 | |
| Search-o1Model Backbone=Qwen3-4B2026.03 | 23.9 | — | |
| MemoRAGBackbone=Llama-3.1-8B-Instruct, Time Cost (ms)=104052026.01 | 20.2 | 20.3 | |
| Direct InferenceModel Backbone=Qwen3-8B2026.03 | 18.4 | — | |
| Direct InferenceModel Backbone=Qwen3-4B2026.03 | 18.2 | — | |
| RAGModel Backbone=Qwen3-8B2026.03 | 16.3 | — | |
| RAGModel Backbone=Qwen3-4B2026.03 | 14.3 | — | |
| ConTReGenBackbone=Llama-3.1-8B-Instruct, Time Cost (ms)=60222026.01 | 14.1 | 14.5 | |
| ProbTreeBackbone=Llama-3.1-8B-Instruct, Time Cost (ms)=40352026.01 | 13.6 | 13.7 | |
| RAG-StarBackbone=Llama-3.1-8B-Instruct, Time Cost (ms)=312532026.01 | 11.1 | 11.5 | |
| ReactBackbone=Qwen3-8B, Time Cost (ms)=108722026.01 | 7.5 | 8.2 | |
| Search-o1Backbone=Llama-3.1-8B-Instruct, Time Cost (ms)=40242026.01 | 6 | 6.3 | |
| Self-RAGBackbone=Llama-3.1-8B-Instruct, Time Cost (ms)=36342026.01 | 3 | 3.6 | |
| Self-RAGBackbone=Qwen3-8B, Time Cost (ms)=36342026.01 | 3 | 3.6 | |
| ReactBackbone=Llama-3.1-8B-Instruct, Time Cost (ms)=46372026.01 | 0.5 | 0.6 |