Multi-hop Question Answering on HotpotQA (EM, F1, EFR)
56.6EMPruneRAG
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PruneRAGBackbone=Qwen3-8B, Retrieval Type=Multi-Retrieval, Time Cost (ms)=22542026.01 | 56.6 | 63.6 | 23.1 | |
| PruneRAGBackbone=Llama-3.1-8B-Instruct, Retrieval Type=Multi-Retrieval, Time Cost (ms)=4742026.01 | 52.8 | 60.6 | 32.8 | |
| Standard RAGBackbone=Llama-3.1-8B-Instruct, Retrieval Type=Single-Retrieval, Time Cost (ms)=1532026.01 | 49 | 55.1 | 27.6 | |
| Standard RAGBackbone=Qwen3-8B, Retrieval Type=Single-Retrieval, Time Cost (ms)=8242026.01 | 49 | 55.6 | 24.8 | |
| RAG-StarBackbone=Qwen3-8B, Retrieval Type=Multi-Retrieval, Time Cost (ms)=147092026.01 | 48.6 | 57.8 | 52.4 | |
| ConTReGenBackbone=Qwen3-8B, Retrieval Type=Multi-Retrieval, Time Cost (ms)=45072026.01 | 47.6 | 55.4 | 30.5 | |
| RAG-StarBackbone=Llama-3.1-8B-Instruct, Retrieval Type=Multi-Retrieval, Time Cost (ms)=82162026.01 | 45 | 54.6 | 51.3 | |
| ConTReGenBackbone=Llama-3.1-8B-Instruct, Retrieval Type=Multi-Retrieval, Time Cost (ms)=21562026.01 | 43.4 | 49.8 | 35.9 | |
| MemoRAGBackbone=Qwen3-8B, Retrieval Type=Single-Retrieval, Time Cost (ms)=103212026.01 | 39.4 | 50.9 | 35.1 | |
| Search-o1Backbone=Qwen3-8B, Retrieval Type=Multi-Retrieval, Time Cost (ms)=48412026.01 | 36.4 | 42.6 | 56.9 | |
| Search-o1Backbone=Llama-3.1-8B-Instruct, Retrieval Type=Multi-Retrieval, Time Cost (ms)=16962026.01 | 35.4 | 39.4 | 53.1 | |
| ProbTreeBackbone=Llama-3.1-8B-Instruct, Retrieval Type=Multi-Retrieval, Time Cost (ms)=38382026.01 | 32.2 | 39.3 | 54.4 | |
| MemoRAGBackbone=Llama-3.1-8B-Instruct, Retrieval Type=Single-Retrieval, Time Cost (ms)=85662026.01 | 31.2 | 38.8 | 53.1 | |
| Self-RAGBackbone=Llama-3.1-8B-Instruct, Retrieval Type=Multi-Retrieval, Time Cost (ms)=32002026.01 | 30.4 | 37.7 | 57.4 | |
| Self-RAGBackbone=Qwen3-8B, Retrieval Type=Multi-Retrieval, Time Cost (ms)=32002026.01 | 30.4 | 37.7 | 57.4 | |
| ReactBackbone=Qwen3-8B, Retrieval Type=Multi-Retrieval, Time Cost (ms)=44972026.01 | 30 | 36.7 | 46.4 | |
| VanillaBackbone=Qwen3-8B, Retrieval Type=No-Retrieval, Time Cost (ms)=7122026.01 | 26.2 | 34.1 | — | |
| VanillaBackbone=Llama-3.1-8B-Instruct, Retrieval Type=No-Retrieval, Time Cost (ms)=562026.01 | 24.2 | 30.1 | — | |
| ReactBackbone=Llama-3.1-8B-Instruct, Retrieval Type=Multi-Retrieval, Time Cost (ms)=15002026.01 | 24.2 | 27.1 | 65.2 | |
| ProbTreeBackbone=Qwen3-8B, Retrieval Type=Multi-Retrieval, Time Cost (ms)=33002026.01 | 22.6 | 26.4 | 50 |