Question Answering on HotpotQA (EM and F1)
84.98F1FlowSteer
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FlowSteerFramework=Ours, Backbone=4o-mini2026.02 | 84.98 | 78.12 | |
| Router-R1Framework=Agent+RL, Backbone=4o-mini2026.02 | 79.84 | 72 | |
| AFlowFramework=AFlow, Backbone=4o-mini2026.02 | 77.9 | 68.75 | |
| AgentflowFramework=Agent+RL, Backbone=4o-mini2026.02 | 77.88 | 67.19 | |
| OrchestratorFramework=Agent+RL, Backbone=4o-mini2026.02 | 75.61 | 67.97 | |
| TaskFT-SPEED-24+BoSFine-tuning=Task-adaptive, K=24, BoS=true2026.05 | 75.5 | 61.2 | |
| TaskFT-FullFine-tuning=Task-adaptive, Visibility Policy=Full2026.05 | 75.3 | 61 | |
| TaskFT-SPEED-28+BoSFine-tuning=Task-adaptive, K=28, BoS=true2026.05 | 75.3 | 60.9 | |
| SFTFramework=SFT, Backbone=Qwen3-8B2026.02 | 75.25 | 70.31 | |
| ComprExITBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Context Compression2026.02 | 74.15 | 55.92 | |
| TaskFT-SPEED-20+BoSFine-tuning=Task-adaptive, K=20, BoS=true2026.05 | 74.1 | 60.2 | |
| Qwen3-8BFramework=Baseline, Backbone=Qwen3-8B2026.02 | 74.05 | 67.19 | |
| 4o-miniFramework=Baseline, Backbone=4o-mini2026.02 | 73.03 | 63.28 | |
| Prompt tuningBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Prompt tuning, Context Setting=w/ context2026.02 | 72.65 | 55.31 | |
| RouteRAGBackbone=Qwen2.5-7B2025.12 | 72.5 | 60.8 | |
| PANINIRetrieval Category=Proposed Method2026.02 | 71.88 | — | |
| HippoRAG 2Backbone=GPT-4o-mini2025.12 | 71.1 | 56.3 | |
| HippoRAG 2Retrieval Category=Structure-Augmented RAG2026.02 | 71.1 | — | |
| NV-Embed-v2 (7B)Retrieval Category=Dense Retrieval2026.02 | 71 | — | |
| Llama3.1-70B+Hit-RAGFramework=Hit-RAG2026.03 | 69.7 | 55.9 | |
| Prompt tuningBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Prompt tuning, Context Setting=w/ context2026.02 | 69.49 | 53.6 | |
| Qwen3-Embedding (8B)Retrieval Category=Dense Retrieval, Reranker=None2026.02 | 69.2 | — | |
| Search-R1Retrieval Category=Agentic Systems2026.02 | 68.6 | — | |
| ComprExITBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Context Compression2026.02 | 68.4 | 49.84 | |
| Qwen3-Embedding (8B)Retrieval Category=Dense Retrieval, Reranker=Qwen3-Embedding (8B) + reranker2026.02 | 68.2 | — | |
| GraphRAGBackbone=GPT-4o-mini2025.12 | 67.6 | 51.4 | |
| GraphRAGRetrieval Category=Structure-Augmented RAG2026.02 | 67.6 | — | |
| BeaconBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Context Compression2026.02 | 67.41 | 51.55 | |
| ICAEBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Context Compression2026.02 | 66.16 | 47.92 | |
| 500xBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Context Compression2026.02 | 65.71 | 47.28 | |
| RouteRAGBackbone=Qwen2.5-3B2025.12 | 65.1 | 53.2 | |
| GRPOFramework=GRPO, Backbone=Qwen3-8B2026.02 | 64.95 | 59.38 | |
| RAPTORBackbone=GPT-4o-mini2025.12 | 64.7 | 50.6 | |
| RAPTORRetrieval Category=Structure-Augmented RAG2026.02 | 64.7 | — | |
| w/ ContextCompression Rate=1x2026.02 | 64.6 | 51.12 | |
| IRCoTRetrieval Category=Agentic Systems2026.02 | 64.3 | — | |
| Qwen3-8B+Hit-RAGFramework=Hit-RAG2026.03 | 64.1 | 47.6 | |
| Qwen3-4B+Hit-RAGFramework=Hit-RAG2026.03 | 63.6 | 48.2 | |
| BM25Retrieval Category=Sparse Retrieval, Reranker=BM25 + reranker2026.02 | 63.4 | — | |
| Llama3.1-70B+R-InsRetrieval=Yes2026.03 | 62.9 | — | |
| Search-R1Backbone=Qwen2.5-7B2025.12 | 62 | 51 | |
| Llama3.1-8B+Hit-RAGFramework=Hit-RAG2026.03 | 60.3 | 48.1 | |
| HippoRAGBackbone=GPT-4o-mini2025.12 | 60 | 46.3 | |
| HippoRAGRetrieval Category=Structure-Augmented RAG2026.02 | 60 | — | |
| Zero-shotBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Zero-shot, Context Setting=w/ context2026.02 | 57.24 | 38.94 | |
| ICAEBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Context Compression2026.02 | 57.04 | 40.67 | |
| Search-R1Backbone=Qwen2.5-3B2025.12 | 56.9 | 45.2 | |
| R1-SearcherBackbone=Qwen2.5-7B2025.12 | 56.7 | 46.6 | |
| Llama3.1-8B+R-InsRetrieval=Yes2026.03 | 56.4 | — | |
| LLMLingua2Compression Rate=4x, Parameter Size=0.6B2026.02 | 55.72 | 44.18 | |
| RankRAG-70BRetrieval=Yes2026.03 | 55.4 | 42.7 | |
| OURS (PIC)Compression Rate=4x, Parameter Size=0.5B2026.02 | 54.66 | 46.61 | |
| Qwen3-32B+Hit-RAGFramework=Hit-RAG2026.03 | 54.4 | 69.3 | |
| BM25Retrieval Category=Sparse Retrieval, Reranker=None2026.02 | 54.2 | — | |
| Original DocsBackbone=Llama-3.1-70B-Instruct, Top-k Documents=Top-20, Compressor Type=-2024.12 | 51.1 | 39.4 | |
| EXITBackbone=Llama-3.1-70B-Instruct, Top-k Documents=Top-20, Compressor Type=Ext.2024.12 | 51.1 | 39.8 | |
| Qwen3-32BRetrieval=None2026.03 | 50.6 | 38.2 | |
| PCC-LiteCompression Rate=4x, Parameter Size=0.77B2026.02 | 50.37 | 42.2 | |
| EXITBackbone=Llama-3.1-70B-Instruct, Top-k Documents=Top-5, Compressor Type=Ext.2024.12 | 49.4 | 38.1 | |
| LongLLMLinguaBackbone=Llama-3.1-70B-Instruct, Top-k Documents=Top-20, Compressor Type=Ext.2024.12 | 49.4 | 37.9 | |
| PCC-LargeCompression Rate=4x, Parameter Size=8B2026.02 | 48.19 | 39.97 | |
| CompActBackbone=Llama-3.1-70B-Instruct, Top-k Documents=Top-5, Compressor Type=Abs.2024.12 | 47.5 | 36.8 | |
| Vanilla RAGBackbone=Qwen2.5-7B2025.12 | 47.5 | 29.1 | |
| RefinerBackbone=Llama-3.1-70B-Instruct, Top-k Documents=Top-5, Compressor Type=Abs.2024.12 | 46.9 | 36 | |
| LongLLMLinguaBackbone=Llama-3.1-70B-Instruct, Top-k Documents=Top-5, Compressor Type=Ext.2024.12 | 46.8 | 35.6 | |
| Original DocsBackbone=Llama-3.1-70B-Instruct, Top-k Documents=Top-5, Compressor Type=-2024.12 | 46.7 | 36 | |
| RankRAG-8BRetrieval=Yes2026.03 | 46.7 | 35.3 | |
| BeaconBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Context Compression2026.02 | 46.35 | 31.23 | |
| HippoRAG 2Backbone=Qwen2.5-7B2025.12 | 46.1 | 27.4 | |
| HippoRAG 2Backbone=Qwen2.5-3B2025.12 | 45 | 31.2 | |
| REPAIR-7BRetrieval Mode=Reranking, Reranking pool size=top-100, Top-k context documents=10, Answer generator model=Llama-3.1-8B-Instruct2026.01 | 44.3 | 33.4 | |
| RefinerBackbone=Llama-3.1-70B-Instruct, Top-k Documents=Top-20, Compressor Type=Abs.2024.12 | 43.3 | 33.3 | |
| RECOMP-AbstBackbone=Llama-3.1-70B-Instruct, Top-k Documents=Top-20, Compressor Type=Abs.2024.12 | 43.2 | 32.4 | |
| REARANK-7BRetrieval Mode=Reranking, Reranking pool size=top-100, Top-k context documents=10, Answer generator model=Llama-3.1-8B-Instruct2026.01 | 43.2 | 32.4 | |
| EXITBackbone=Llama-3.1-8B-Instruct, Top-k Documents=Top-20, Compressor Type=Ext.2024.12 | 43.1 | 32.2 | |
| CompActBackbone=Llama-3.1-70B-Instruct, Top-k Documents=Top-20, Compressor Type=Abs.2024.12 | 42.7 | 33.6 | |
| Rank-R1-7BRetrieval Mode=Reranking, Reranking pool size=top-100, Top-k context documents=10, Answer generator model=Llama-3.1-8B-Instruct2026.01 | 42.5 | 32.2 | |
| Vanilla RAGBackbone=Qwen2.5-3B2025.12 | 41.8 | 29.5 | |
| RECOMP-AbstBackbone=Llama-3.1-70B-Instruct, Top-k Documents=Top-5, Compressor Type=Abs.2024.12 | 41.2 | 30.8 | |
| LongLLMLinguaBackbone=Llama-3.1-8B-Instruct, Top-k Documents=Top-20, Compressor Type=Ext.2024.12 | 41.1 | 31 | |
| OURS (PIC)Compression Rate=16x, Parameter Size=0.5B2026.02 | 41.05 | 33.24 | |
| Direct InferenceBackbone=GPT-4o-mini2025.12 | 41 | 28.6 | |
| RankZephyr-7BRetrieval Mode=Reranking, Reranking pool size=top-100, Top-k context documents=10, Answer generator model=Llama-3.1-8B-Instruct2026.01 | 41 | 29.2 | |
| NoneRetrieval Category=No Retrieval2026.02 | 41 | — | |
| Qwen3-8BRetrieval=None2026.03 | 40.9 | 30.9 | |
| Original DocsBackbone=Llama-3.1-8B-Instruct, Top-k Documents=Top-20, Compressor Type=-2024.12 | 40.8 | 30.6 | |
| OURS (PIC)Compression Rate=16x, Parameter Size=1B2026.02 | 40.61 | 32.47 | |
| CompActBackbone=Llama-3.1-8B-Instruct, Top-k Documents=Top-5, Compressor Type=Abs.2024.12 | 40.6 | 30.2 | |
| CompActBackbone=Llama-3.1-8B-Instruct, Top-k Documents=Top-20, Compressor Type=Abs.2024.12 | 40.6 | 30.7 | |
| EXITBackbone=Llama-3.1-8B-Instruct, Top-k Documents=Top-5, Compressor Type=Ext.2024.12 | 40.3 | 29.7 | |
| RECOMP-ExtrBackbone=Llama-3.1-70B-Instruct, Top-k Documents=Top-5, Compressor Type=Ext.2024.12 | 40.2 | 30.8 | |
| Llama3.1-70BRetrieval=None2026.03 | 40.1 | 28.6 | |
| COCOM-LiteCompression Rate=4x, Parameter Size=0.1B2026.02 | 40.07 | 32.32 | |
| Original DocsBackbone=Llama-3.1-8B-Instruct, Top-k Documents=Top-5, Compressor Type=-2024.12 | 39.9 | 29.6 | |
| RefinerBackbone=Llama-3.1-8B-Instruct, Top-k Documents=Top-5, Compressor Type=Abs.2024.12 | 39.9 | 29.6 | |
| Zero-shotBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Zero-shot, Context Setting=w/ context2026.02 | 39.24 | 22.51 | |
| Qwen3-Embedding-0.6BRetrieval Mode=Initial Retrieval, Top-k context documents=10, Answer generator model=Llama-3.1-8B-Instruct2026.01 | 38.5 | 27.6 | |
| RECOMP-ExtrBackbone=Llama-3.1-70B-Instruct, Top-k Documents=Top-20, Compressor Type=Ext.2024.12 | 38 | 28.9 | |
| DioRBackbone=LLaMA2-7B-CHAT2025.04 | 37.9 | 27.4 | |
| RefinerBackbone=Llama-3.1-8B-Instruct, Top-k Documents=Top-20, Compressor Type=Abs.2024.12 | 37.9 | 27.8 |