Question Answering on HotpotQA
77.2EMSGIC
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SGICBackbone=Llama2-7B, Tuning=LoRA (PEFT), Retrieved documents=102025.06 | 77.2 | 80.5 | |
| OursBackbone=BERT-base2022.03 | 69.82 | 82.57 | |
| B+C+VcorrBackbone=BERT-base2022.03 | 68.39 | 80.42 | |
| B+CBCLBackbone=BERT-base2022.03 | 67.72 | 79.65 | |
| Supervised SoTA2022.10 | 67.5 | — | |
| B+DRCABackbone=BERT-base2022.03 | 67.07 | 79.32 | |
| B+CL+VBackbone=BERT-base2022.03 | 66.44 | 78.71 | |
| B+C+PredictabilityBackbone=BERT-base2022.03 | 66.36 | 79.75 | |
| B+C+v4Backbone=BERT-base2022.03 | 66.3 | 79.13 | |
| B+C+v3Backbone=BERT-base2022.03 | 65.36 | 78.75 | |
| B+C+ForgettingBackbone=BERT-base2022.03 | 64.75 | 77.62 | |
| B+C+v1Backbone=BERT-base2022.03 | 64.18 | 76.88 | |
| B+C+v2Backbone=BERT-base2022.03 | 64.12 | 77.01 | |
| B+C+DatasetMapBackbone=BERT-base2022.03 | 63.86 | 76.57 | |
| BBackbone=BERT-base2022.03 | 63.53 | 76.42 | |
| B+antiCL+VBackbone=BERT-base2022.03 | 63.01 | 75.03 | |
| Ψ-RAG øType=Tree-RAG, Reranker=Qwen3-Reranker-8B (Q3R), Abstract Type=keyword2026.05 | 62.2 | 74.85 | |
| Ψ-RAG ⍋Type=Tree-RAG, Reranker=Qwen3-Reranker-8B (Q3R), Abstract Type=summative2026.05 | 62.1 | 74.59 | |
| HippoRAG 2Type=Graph-RAG2026.05 | 61.9 | 75.4 | |
| Full-depth LoRAAdaptation strategy=Full-depth LoRA2026.05 | 60.8 | 75.3 | |
| Full-depth LoRA2026.05 | 60.8 | 75.3 | |
| OffShelf-FT-SPEED+BoS-24Adaptation strategy=SPEED+BoS, K=242026.05 | 59.5 | 73.7 | |
| OffShelf-FT-SPEED+BoSSPEED cutoff (K)=242026.05 | 59.5 | 73.7 | |
| OffShelf-FT-SPEED+BoS-20Adaptation strategy=SPEED+BoS, K=202026.05 | 59.4 | 73.5 | |
| OffShelf-FT-SPEED+BoSSPEED cutoff (K)=202026.05 | 59.4 | 73.5 | |
| DPR + IRCoT + Q3RType=Traditional RAG, Reranker=Qwen3-Reranker-8B (Q3R), Iterative Retrieval Protocol=IRCoT2026.05 | 59.1 | 71 | |
| HippoRAG 2 + Q3RType=Graph-RAG, Reranker=Qwen3-Reranker-8B (Q3R)2026.05 | 58.7 | 71.52 | |
| OffShelf-FT-SPEED+BoS-28Adaptation strategy=SPEED+BoS, K=282026.05 | 58.7 | 73.4 | |
| OffShelf-FT-SPEED+BoSSPEED cutoff (K)=282026.05 | 58.7 | 73.4 | |
| VECTORRAGLLM=GPT-OSS-120B2026.05 | 58.6 | 72.9 | |
| BM25 + DPR + Q3RType=Traditional RAG, Reranker=Qwen3-Reranker-8B (Q3R)2026.05 | 58.4 | 70.98 | |
| Llama3.1 8B InstructAdaptation strategy=None2026.05 | 56.9 | 72.7 | |
| Llama3.1 8B Instruct2026.05 | 56.9 | 72.7 | |
| OffShelf-FT-SPEED+BoS-16Adaptation strategy=SPEED+BoS, K=162026.05 | 55 | 69.4 | |
| OffShelf-FT-SPEED+BoSSPEED cutoff (K)=162026.05 | 55 | 69.4 | |
| GraphRAGType=Graph-RAG2026.05 | 54.9 | 66.3 | |
| DPRType=Traditional RAG2026.05 | 52 | 63.1 | |
| HIPPORAGV2LLM=Qwen3-30B-A3B2026.05 | 50 | 64.1 | |
| IRCoT QABackbone=GPT3 (175B)2022.12 | 49.3 | 60.7 | |
| Self-RAGBackbone=Llama2-7B, Tuning=Full parameter fine-tuning2025.06 | 48.7 | 29.9 | |
| OAKLLM=Qwen3-30B-A3B2026.05 | 47 | 58.6 | |
| WIKONTICLLM=Qwen3-30B-A3B2026.05 | 46.4 | 57.7 | |
| OAKLLM=GPT-OSS-120B2026.05 | 46.4 | 57.8 | |
| RAMBackbone=Qwen3-4B-Instruct, Compression=4x2026.02 | 46.37 | 59.28 | |
| RAMBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.02 | 46.24 | 59.13 | |
| OAK+MENDLLM=GPT-OSS-120B2026.05 | 46.1 | 56.8 | |
| SAPO-3B-InstructRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 45.6 | — | |
| OAK+MENDLLM=Qwen3-30B-A3B2026.05 | 45.6 | 56.4 | |
| EXITBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.02 | 45.55 | 58.81 | |
| EXITBackbone=Qwen3-4B-Instruct, Compression=4x2026.02 | 45.24 | 58.13 | |
| SE-SearchRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 45 | — | |
| IN-CONTEXTLLM=Qwen3-30B-A3B2026.05 | 45 | 55.6 | |
| SAPO-3B-BaseRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 44.9 | — | |
| Original PromptBackbone=Qwen3-4B-Instruct2026.02 | 44.33 | 60.47 | |
| BM25Type=Traditional RAG2026.05 | 43.8 | 54.07 | |
| Activation BeaconBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.02 | 43.78 | 57.44 | |
| RAPTOR + Q3RType=Tree-RAG, Reranker=Qwen3-Reranker-8B (Q3R)2026.05 | 43.1 | 52.54 | |
| VECTORRAGLLM=Qwen3-30B-A3B2026.05 | 42.9 | 56.5 | |
| Search-R1Method Category=Reinforcement Learning-based Methods2026.01 | 42.7 | 54.6 | |
| ProvenceBackbone=Qwen3-4B-Instruct, Compression=4x2026.02 | 42.38 | 56.11 | |
| EXSEARCH-BaseRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 42.2 | — | |
| ProRAGMethod Category=Reinforcement Learning-based Methods2026.01 | 41.4 | 52.8 | |
| CriticSearchRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 41.4 | — | |
| InForageRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 40.9 | — | |
| KVIBackbone=Mistral-7B-Instruct-v0.32026.04 | 40.8 | — | |
| ProvenceBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.02 | 40.57 | 57.51 | |
| AutoRefine-BaseRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 40.5 | — | |
| AutoRefine-InstructRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 40.4 | — | |
| WIKONTICLLM=GPT-OSS-120B2026.05 | 40.4 | 50.4 | |
| KGGENLLM=GPT-OSS-120B2026.05 | 38.9 | 48.5 | |
| RAMBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.02 | 38.82 | 50.8 | |
| O2-SearcherRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 38.8 | — | |
| HIPPORAGV2LLM=GPT-OSS-120B2026.05 | 38.8 | 57 | |
| KGGENLLM=Qwen3-30B-A3B2026.05 | 38.5 | 48.7 | |
| ReasonRAGMethod Category=Reinforcement Learning-based Methods2026.01 | 38.4 | 48.9 | |
| GraphRAGBackbone=Mistral-7B-Instruct-v0.32026.04 | 38.3 | — | |
| RAMBackbone=Qwen3-4B-Instruct, Compression=8x2026.02 | 37.9 | 49.15 | |
| ProvenceBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.02 | 37.22 | 49.91 | |
| RECITE2022.12 | 37.1 | 48.4 | |
| HiPRAGMethod Category=Reinforcement Learning-based Methods2026.01 | 36.8 | 49.1 | |
| ProvenceBackbone=Qwen3-4B-Instruct, Compression=8x2026.02 | 36.74 | 48.71 | |
| RAGBackbone=Mistral-7B-Instruct-v0.32026.04 | 36.7 | — | |
| Activation BeaconBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.02 | 36.29 | 48.57 | |
| EXITBackbone=Qwen3-4B-Instruct, Compression=8x2026.02 | 35.75 | 48.4 | |
| ReAct -> CoT-SCBackbone=PaLM-540B2022.10 | 35.1 | — | |
| ReAct2022.12 | 35.1 | — | |
| ReSearch-InstructRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 35.1 | — | |
| LongLLMLinguaBackbone=LLaMA-3.1-8B-Instruct, Compression=4x2026.02 | 34.88 | 51.4 | |
| CoT-SC -> ReActBackbone=PaLM-540B2022.10 | 34.2 | — | |
| ZeroSearch-BaseRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 33.8 | — | |
| LooComp (Ours)Reader=LLama-3.1-8B Instruct, Retriever=BM25, k=10, Time=0.078, Rate=12.22026.03 | 33.6 | 45.3 | |
| CoT-SCBackbone=PaLM-540B2022.10 | 33.4 | — | |
| ZEROSHOTLLM=GPT-OSS-120B2026.05 | 33.4 | 45.1 | |
| KVIBackbone=Qwen2.5-7B-Instruct2026.04 | 33.3 | — | |
| Search-R1-InstructRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 33.1 | — | |
| IN-CONTEXTLLM=GPT-OSS-120B2026.05 | 33.1 | 41.8 | |
| StepSearch-BaseRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 32.9 | — | |
| GraphRAGBackbone=Qwen2.5-7B-Instruct2026.04 | 32.5 | — | |
| RefinerReader=LLama-3.1-8B Instruct, Retriever=BM25, k=10, Time=2.750, Rate=8.32026.03 | 32.4 | 43.3 | |
| EXITBackbone=LLaMA-3.1-8B-Instruct, Compression=8x2026.02 | 32.16 | 50.19 |